Re: FW: I/O Optimization

W dniu 2013-06-11 19:20, John Eells pisze: R.S. wrote: BTW2: It's really strange why IFASMFDP have no option like SPANINC. Is it candidate for z/OS version 3.x in 2024? As Yogi Berra is rumored to have said (there are any number of attributed variations), "Prediction is hard. Especially abo

Re: I/O Optimization

In <2931391569354995.wa.paulgboulderaim@listserv.ua.edu>, on 06/10/2013 at 11:32 PM, Paul Gilmartin said: >You may have asked for 32760, He asked for LRECL=32760, not for BLKSIZE=32760. >but no block is even close to that; something >appears to be overriding to half-track. He specified

Re: FW: I/O Optimization

In <00ef01ce6688$dc592c20$950b8460$@mxg.com>, on 06/11/2013 at 04:48 AM, Barry Merrill said: >The historic use of VBS was precisely for SMF in OS/360, Didn't FORTRAN use VBS before there was VBS? -- Shmuel (Seymour J.) Metz, SysProg and JOAT Atid/2

Re: FW: I/O Optimization

R.S. wrote: BTW2: It's really strange why IFASMFDP have no option like SPANINC. Is it candidate for z/OS version 3.x in 2024? As Yogi Berra is rumored to have said (there are any number of attributed variations), "Prediction is hard. Especially about the future." But in my view, the chanc

Re: FW: I/O Optimization

Been too long. Guess in late sixties we had a 360/50 as play machine. Mostly MVT but some RAX and for 'big' programs PCP. Anyway, we ordered a 'model' tape from Sandia Labs that was supposed to be 'very precise' transistor library for SCEPTRE design(like ECAP and PCAP). One of the checks was

Re: I/O Optimization

On Tue, 11 Jun 2013 09:01:16 -0500, Tom Marchant wrote: > >Clearly, if a record ends, for example, three bytes short of BLKSIZE, >another record cannot be added to that block. The block would >have to be written as is, with a block size of three bytes less than >BLKSIZE or a program reading it

Re: FW: I/O Optimization

On 6/11/2013 8:31 AM, John Gilmore wrote: This is accurate enough if 'The' is replaced by 'a'; but VBS has a very much longer history beginning with unformatted FORTRAN II tape I/O on the IBM 704. Perhaps my memory isn't what it used to be, but I do not remember anything like variable format o

Re: I/O Optimization

In <4f887c92-5a9f-4159-80c5-9dedeced5...@aim.com>, on 06/10/2013 at 10:25 PM, Paul Gilmartin said: >It was a single-part message; no attachment, with: >MIME-Version: 1.0 >Content-Transfer-Encoding: base64 >Content-Type: text/plain; charset="utf-8" >... an unusual combination, but

Re: I/O Optimization

In <0997106197004218.wa.paulgboulderaim@listserv.ua.edu>, on 06/10/2013 at 04:02 PM, Paul Gilmartin said: >Is uniform lengths a requirement? No. >Do any applications rely on being able to calculate a cylinder and >track address within a VBS data set? I hope not; if it does, it's brok

Re: I/O Optimization

In <746510224.45656.1370914746772.javamail.r...@sz0127a.westchester.pa.mail.comcast.net>, on 06/11/2013 at 01:39 AM, DASDBILL2 said: >On output, the blocks are all written with the same blocksize except >possibly the last.  This causes the reading in of such a file with >constant block size ex

I/O Optimization

For a complete discussion about V/VB/VS/VBS and F/FB/FBS file formats, please look at Chapter 20 of the DFSMS Using Data Sets manual. The following for FBS is extracted from that manual. This is the definitive manual for standard files to be processed by BSAM and QSAM. Standard Format During

Re: I/O Optimization

On Mon, 10 Jun 2013 16:02:59 -0500, Paul Gilmartin wrote: >On Mon, 10 Jun 2013 17:43:58 +, DASDBILL2 wrote: >> >>Spanned means that a single logical record might span multiple >>physical blocks, but the way it is implemented results in having >>all block sizes be the same, except for possibl

Re: FW: I/O Optimization

U Subject: Re: FW: I/O Optimization Barry Merrill wrote: The historic use of VBS was precisely for SMF in OS/360, when the NUC was 86K and SMF want to write 32K logical records. This is accurate enough if 'The' is replaced by 'a'; but VBS has a very much longer history be

Re: FW: I/O Optimization

Barry Merrill wrote: The historic use of VBS was precisely for SMF in OS/360, when the NUC was 86K and SMF want to write 32K logical records. This is accurate enough if 'The' is replaced by 'a'; but VBS has a very much longer history beginning with unformatted FORTRAN II tape I/O on the IBM 704

Re: FW: I/O Optimization

W dniu 2013-06-11 12:06, Elardus Engelbrecht pisze: Barry Merrill wrote: NOT PARTICULARLY EFFICIENT, but met the requirments. Of course. It is also not efficient when you need to recover lost SMF data. One broken record or block and the rest of the dataset are lost. [1] Groete / Greetings El

Re: FW: I/O Optimization

t: Re: FW: I/O Optimization Barry Merrill wrote: >NOT PARTICULARLY EFFICIENT, but met the requirments. Of course. It is also not efficient when you need to recover lost SMF data. One broken record or block and the rest of the dataset are lost. [1] Groete / Greetings Elardus Engelbrecht

Re: FW: I/O Optimization

Barry Merrill wrote: >NOT PARTICULARLY EFFICIENT, but met the requirments. Of course. It is also not efficient when you need to recover lost SMF data. One broken record or block and the rest of the dataset are lost. [1] Groete / Greetings Elardus Engelbrecht [1] - I believe there is a program

FW: I/O Optimization

List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Paul Gilmartin Sent: Monday, June 10, 2013 11:33 PM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: I/O Optimization On Mon, 10 Jun 2013 17:18:48 -0500, Barry Merrill wrote: >VBS Blocksize in dumped SMF data with RECFM=VBS,LRECL=32760,BLKSIZE=0 &

Re: I/O Optimization

On Mon, 10 Jun 2013 17:18:48 -0500, Barry Merrill wrote: >VBS Blocksize in dumped SMF data with RECFM=VBS,LRECL=32760,BLKSIZE=0 does NOT >produce interior same-sized blocks, as this small file from my DEV machine >shows: > >LENGTHFREQUENCY PERCENT

Re: I/O Optimization

On 2013-06-10, at 17:20, Ted MacNEIL wrote: > Easy for you to way! > > -Original Message- > From: "Blaicher, Christopher Y." > Date: Mon, 10 Jun 2013 18:31:31 > > ÅjÇßÆqÔµ§‘U˘æ’ã À œ\»œıØjß4Pª¤S0-ë9’1e£Î$*˘ï
çPêA > zÙ¶MQÊãN>j|6¯d‚2Úo¸É&‡z%4wQb~0„¥@ü∫ÎÁ¸7ß

Re: I/O Optimization

Original Message - From: "Paul Gilmartin" To: IBM-MAIN@LISTSERV.UA.EDU Sent: Monday, June 10, 2013 4:02:59 PM Subject: Re: I/O Optimization On Mon, 10 Jun 2013 17:43:58 +, DASDBILL2 wrote: > >Spanned means that a single logical record might span multiple physical

Re: I/O Optimization

Easy for you to way! - Ted MacNEIL eamacn...@yahoo.ca Twitter: @TedMacNEIL -Original Message- From: "Blaicher, Christopher Y." Sender: IBM Mainframe Discussion List Date: Mon, 10 Jun 2013 18:31:31 To: Reply-To: IBM Mainframe Discussion List Subjec

Re: I/O Optimization

cussion List Date: Mon, 10 Jun 2013 17:18:48 To: Reply-To: IBM Mainframe Discussion List Subject: Re: I/O Optimization VBS Blocksize in dumped SMF data with RECFM=VBS,LRECL=32760,BLKSIZE=0 does NOT produce interior same-sized blocks, as this sm

Re: I/O Optimization

Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Paul Gilmartin Sent: Monday, June 10, 2013 4:03 PM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: I/O Optimization On Mon, 10 Jun 2013 17:43:58 +, DASDBILL2 wrote: > >Spanned means that a single logical record migh

Re: I/O Optimization

, June 10, 2013 4:03 PM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: I/O Optimization On Mon, 10 Jun 2013 17:43:58 +, DASDBILL2 wrote: > >Spanned means that a single logical record might span multiple physical >blocks, but the way it is implemented results in having all block sizes be t

Re: I/O Optimization

On Mon, 10 Jun 2013 17:43:58 +, DASDBILL2 wrote: > >Spanned means that a single logical record might span multiple physical >blocks, but the way it is implemented results in having all block sizes be the >same, except for possibly the last one, which might be short. > Is uniform lengths a r

Re: I/O Optimization

: Wednesday, June 5, 2013 9:42:32 PM Subject: Re: I/O Optimization >Where does "spanned" come into play? Why does that make the difference? An acronym conflict! I hate IBM's tendency to use the same letters (in the same position) to mean different things! vbS -- Variabl

Re: I/O Optimization

In <1697813022-1370486555-cardhu_decombobulator_blackberry.rim.net-1466420072-@b12.c1.bise6.blackberry>, on 06/06/2013 at 02:42 AM, Ted MacNEIL said: >ATM -- I thought it meant Automated Teller Machine. At the moment it means asynchronous transfer mode. -- Shmuel (Seymour J.) Metz, Sys

Re: I/O Optimization

W dniu 2013-06-06 13:20, John Gilmore pisze: My last post was mangled/truncated. The overloaded-acronyms problem is not chiefly a combinatorial one. It is better viewed as a variant of Feller's duplicate-birthdays problem Anybody said USS? ;-) -- Radoslaw Skorupka Lodz, Poland -- Tre

Re: I/O Optimization

Gerard Schildberger writes: > CMS - Cross Memory Services --- later renamed to XMS. re: http://www.garlic.com/~lynn/2013h.html#24 I/O Optimization CMS was originally Cambridge Monitor System ... part of cp67. CMS morphed into Conversational Monitor System when CP67 morphed into VM370 as

Re: I/O Optimization

My last post was mangled/truncated. The overloaded-acronyms problem is not chiefly a combinatorial one. It is better viewed as a variant of Feller's duplicate-birthdays problem John Gilmore, Ashland, MA 01721 - USA -- For IBM-MA

Re: I/O Optimization

There is no avoiding overloaded acronyms. The important They are fairly benign when their chief uses are in very different contexts. Here ACM is an acronym for Association for Computing Machinery. In Nashville it mostly means Academy of Country Music On 6/5/13, Shmuel Metz (Seymour J.) wrot

Re: I/O Optimization

In , on 06/05/2013 at 09:33 AM, John McKown said: >As I understand it, the filesystem knows the last byte written. What is in the DSCB1 is the last track and record. There is no support for indicating that part of a block in an FB or FBS data set is unused. -- Shmuel (Seymour J.) Metz,

Re: I/O Optimization

Paul Gilmartin wrote: >It has been pointed out ... Who did that? >... that one of the great challenges to be dealt with in the 21st Century is >that there are only 17,576 possible three letter acronyms. Solution: Enlarge your pool of 26 characters to include numbers [ten] and symbols [how man

Re: I/O Optimization

On Thu, 6 Jun 2013 02:42:32 +, Ted MacNEIL wrote: > >An acronym conflict! > It has been pointed out that one of the great challenges to be dealt with in the 21st Century is that there are only 17,576 possible three letter acronyms. -- gil -

Re: I/O Optimization

>Where does "spanned" come into play? Why does that make the difference? An acronym conflict! I hate IBM's tendency to use the same letters (in the same position) to mean different things! vbS -- Variable Blocked Spanned fbS -- Fixed Blocked Standard (BTW: why "Standard"? What the H-E-Double-T

Re: I/O Optimization

On Wed, 5 Jun 2013 12:49:26 -0400, Gerhard Postpischil wrote: > >It is still possible to add extra code to OPEN MOD to read the last >block, and adjust the buffers, but there is no business case for it. >Whenever direct addressing was convenient, I used BSAM or BDAM, and >handled MOD and short buf

Re: I/O Optimization

On 6/5/2013 9:55 AM, John Gilmore wrote: It has important uses, but FBS is for experienced, highly competent people and only for them.. Unfortunately you may replace FBS with any non-trivial computing concept, and still be correct. What's important is the work environment. With a couple of e

Re: I/O Optimization

Original Message- > From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On > Behalf Of David Crayford > Sent: Wednesday, June 05, 2013 8:19 AM > To: IBM-MAIN@LISTSERV.UA.EDU > Subject: Re: I/O Optimization > >> BTW, is the reason FBS is fast for seek

Re: I/O Optimization

On 6/5/2013 10:12 AM, Paul Gilmartin wrote: How, then, do OSes with native FBA filesystems manage to support appending to files? Couldn't QSAM have been designed to to likewise? Either they fill every block, or they keep a pointer. When xSAM was designed, every byte was precious, and most co

Re: I/O Optimization

The overloaded 'S' is a gratuitous trap for novices, and QSAM could have been designed to write an initial dummy record of standard length when it allocated a DASD extent for a RECFM=FBS dataset. We live in an imperfect world in which, in particular, not everyone is perfectly prescient. John Gi

Re: I/O Optimization

As I understand it, the filesystem knows the last byte written. With an FBA device, the access method reads that block and starts writing after the last used byte within the block. It then rewrites the entire block. I don't know why BSAM could not do something similar with FBS files. Of course, thi

Re: I/O Optimization

On Wed, 5 Jun 2013 19:26:56 +0800, David Crayford wrote: >I've never seen FBS used before. I see it regularly for dumps that are processed by IPCS. -- Tom Marchant -- For IBM-MAIN subscribe / signoff / archive access instructi

Re: I/O Optimization

On Wed, 5 Jun 2013 09:32:18 -0400, Blaicher, Christopher Y. wrote: > >Do not MOD data to a FBS file. A short block in an FBS file is a EOF >condition. This means that if you luck out and fill the last buffer of a >file, you can MOD to it, but 99 44/100 percent of the time it doesn't work and >

Re: I/O Optimization

I agree with Chris Blaicher, but I should put the matter more brutally. It has important uses, but FBS is for experienced, highly competent people and only for them.. John Gilmore, Ashland, MA 01721 - USA -- For IBM-MAIN subscr

Re: I/O Optimization

David Crayford wrote: Where does "spanned" come into play? Why does that make the difference? The short answer is that it does not come into play. For fixed-length records, S means 'standard', without short blocks. For variable-length records, S means 'spanned', a [logical] record may span a

Re: I/O Optimization

"Standard" for FBS, not "spanned". Spanned is for VBS. = > Date: Wed, 5 Jun 2013 21:19:09 +0800 > From: dcrayf...@gmail.com > Subject: Re: I/O Optimization > To: IBM-MAIN@LISTSERV.UA.EDU > > >> BTW, is the reason FBS is fast for seeks beca

Re: I/O Optimization

On 6/5/2013 7:19 AM, David Crayford wrote: BTW, is the reason FBS is fast for seeks because it makes it easier to calculate the track position for a POINT? I've never seen FBS used before. I don't know what IBM uses under the covers, but it's probably the same thing that SAS/C did. Calculate the

Re: I/O Optimization

ncsort.com -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of David Crayford Sent: Wednesday, June 05, 2013 8:19 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: I/O Optimization >> BTW, is the reason FBS is fast for seeks because it make

Re: I/O Optimization

-Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of David Crayford Sent: Wednesday, June 05, 2013 15:19 To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: I/O Optimization >> BTW, is the reason FBS is fast for seeks because it makes it

Re: I/O Optimization

I thought "s" stood for "standard" rather than "spanned". It's the "s" in VBS that's "spanned". But I can't answer your question. :-( Cheers, Martin Martin Packer, zChampion, Principal Systems Investigator, Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.i

Re: I/O Optimization

BTW, is the reason FBS is fast for seeks because it makes it easier to calculate the track position for a POINT? I've never seen FBS used before. I don't know what IBM uses under the covers, but it's probably the same thing that SAS/C did. Calculate the CCHHR from the byte offset and use EXCP to

Re: I/O Optimization

poit...@pobox.com (Don Poitras) writes: > I don't know what IBM uses under the covers, but it's probably the same > thing that SAS/C did. Calculate the CCHHR from the byte offset and use > EXCP to read the block directly. No need to use POINT. FBS is guaranteed > not to have any short blocks, so t

Re: I/O Optimization

In article <51af2080.6080...@gmail.com> you wrote: > One would expect reads to be faster with ZFS because of the caching but > what astonished me was the efficiency of writes. Writes to a ZFS file > system were well over 10x faster than QSAM. I profiled the > tests and QSAM spent most of it's tim

Re: I/O Optimization

One would expect reads to be faster with ZFS because of the caching but what astonished me was the efficiency of writes. Writes to a ZFS file system were well over 10x faster than QSAM. I profiled the tests and QSAM spent most of it's time waiting for I/O to complete. BSAM was worse. Maybe if we

Re: I/O Optimization

Yes, I guess, that the freads on ZFS will be faster, but: - the customer wants the files to be classical z/OS data sets, and - because of the cache (least recently used algoritm), the I/O delays are of no real concern any more - the elapsed time is much better than in the DB2 case, anyway. But:

Re: I/O Optimization

Did you try using a ZFS file system? On my system freads()/fwrites() to a unix file are significantly faster than QSAM/BSAM. On 5/06/2013 9:33 AM, Bernd Oppolzer wrote: Two weeks ago, I told you about tests with our table system, which holds read-only data for our insurance math package. The d

Re: I/O Optimization

Two weeks ago, I told you about tests with our table system, which holds read-only data for our insurance math package. The data was stored in DB2 tables until now, and we tried to get better CPU usage etc. by moving the data to our file based table system, which we have in the Windows and Unix e

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

On Thu, 23 May 2013 11:44:58 -0400, Shmuel Metz (Seymour J.) wrote: >>> >>>In your ADDRESS SYSCALL read, what length do you specify? In your >>>ADDRESS TSO EXECIO, what do you specify for lines? >>> >>Minimal, in order to let the ALLOCATE/OPEN/CLOSE overhead >>dominate. > >Using minimal lengths dri

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

In <9433992692961071.wa.paulgboulderaim@listserv.ua.edu>, on 05/22/2013 at 03:42 PM, Paul Gilmartin said: >On Wed, 22 May 2013 14:25:46 -0400, Shmuel Metz (Seymour J.) wrote: > >> at 07:36 AM, Paul Gilmartin said: >> >>>SYSCALL for the UNIX files; BPXWDYN/EXECIO for the legacy. >> >>In

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

On Wed, 22 May 2013 14:25:46 -0400, Shmuel Metz (Seymour J.) wrote: > > at 07:36 AM, Paul Gilmartin said: > >>SYSCALL for the UNIX files; BPXWDYN/EXECIO for the legacy. > >In your ADDRESS SYSCALL read, what length do you specify? In your >ADDRESS TSO EXECIO, what do you specify for lines? > Mi

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

In <6207625106804329.wa.paulgboulderaim@listserv.ua.edu>, on 05/22/2013 at 07:36 AM, Paul Gilmartin said: >SYSCALL for the UNIX files; BPXWDYN/EXECIO for the legacy. In your ADDRESS SYSCALL read, what length do you specify? In your ADDRESS TSO EXECIO, what do you specify for lines? --

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

On Wed, 22 May 2013 05:02:09 -0400, Shmuel Metz (Seymour J.) wrote: > > at 10:35 AM, Paul Gilmartin said: > >>I should confess (or at least clarify) that my tests were Rexx-based > >Using what interface? > It was a while back, though the code might still be around somewhere. From memory: SYSCAL

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

On 22/05/2013 4:11 PM, Miklos Szigetvari wrote: Hi Maybe some point: - A few years ago in the OMVS newsgroup somebody pointed out that the "fseek" "ftell" etc could decrease the performance very drastically - If you need a random access maybe the VSAM would be an option IIRC, that would be

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

In <1560706747382932.wa.paulgboulderaim@listserv.ua.edu>, on 05/21/2013 at 10:35 AM, Paul Gilmartin said: >I should confess (or at least clarify) that my tests were Rexx-based Using what interface? -- Shmuel (Seymour J.) Metz, SysProg and JOAT Atid/2

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

Hi Maybe some point: - A few years ago in the OMVS newsgroup somebody pointed out that the "fseek" "ftell" etc could decrease the performance very drastically - If you need a random access maybe the VSAM would be an option On 21.05.2013 16:24, Bernd Oppolzer wrote: We already have improv

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

On Tue, 21 May 2013 09:30:41 -0400, John Gilmore wrote: > >My measurements confirm this, but when these small files are moved [as >members] into a single PDSE things change dramatically. The UNIX >times are 1.47-3.07 times higher: To avoid spurious exactitude let us >just say significantly higher.

Re: I/O Optimization

BTW: for another customer (also z/OS based), we put all the data in a data space, using a 3rd party system that the other customer provided. No problem there. For Windows server, the application runs multi-threaded, and the cache data of the table system needs to be serialized; this is done b

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

We already have improvements to cut the open/close/free overhead. The ca. 800 little tables are put into 8 large containers, which have a (sort of) directory at the beginning, so that there is only one open for the container. The open is done only once, and the containers stay open throughout the

Re: I/O Optimization (was: B37 för FTINCL in ISPF ...)

Paul Gilmartin wrote: I've found that for large numbers of small files z/OS UNIX files vastly outperform legacy data sets. The allocate/open/close/free overhead is brutal. My measurements confirm this, but when these small files are moved [as members] into a single PDSE things change dramatica

I/O Optimization (was: B37 för FTINCL in ISPF ...)

On Tue, 21 May 2013 09:55:51 +0200, Bernd Oppolzer wrote: >Slightly drifting topic: > >We use fseek / ftell / fread to do the file I/O. The files are normal >sequential >OS files. > Sounds like the worst of both worlds. Have you tried it with normal z/OS UNIX files? The kernel may do the cachi