On Wed, 2 May 2018 23:27:55 +0100, David W Noon wrote: >>> >> Suppose there is no guarantee that every logical record fit in available >> virtual memory? >> >How do you process such a logical record in a finite address space? The >only way I can think of is to segment the record and process it >piecewise. That would require BSAM, since the buffer pool will fit into >memory, and the record segments will be limited in size to fit into a >buffer. > >You then have an issue of related fields not being in the same segment >or not being in segments close enough to have the related fields >accessed concurrently. > Windowing. If it's feasible to choose a segment size large enough that any two related fields fit in no more than two consecutive segments you need only two buffers. Purge the older one; swap the buffers; and read in a newer segment.
>I would then feel that the data stream design is flawed. I would want it >normalized, when possible. The problem with that is that many "small >platform" people know little or nothing about normal forms. > Tunnel vision. Some real world data naturally arrive in a featureless stream; no neat separation into segments, records, or blocks. The world won't change to accommodate your preferred design, and you can't intimidate it by using cultural pejoratives such as "many 'small platform' people know little ..." > ... [Indeed, it >is mainly DB2 and IMS people who understand it on the mainframe.] The >problem remains if the record layout is already in at least 1NF. > Consider LIGO: https://www.ligo.caltech.edu/ Its data inexorably arrive as a stream. Some amazing science comes from processing that stream. That couldn't have been done if the researchers had spurned those input data because they were in a "flawed design" rather than a normal form. -- gil ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
