> >1) batch oriented - each step of the process produces its own > >output file or data structure and this gets picked up by the > >next stage. Tis usually involved processing data in chunks > >- writing the first dump after every 10th set of input say. > > > I see your point, like a static chain, one calling the next & passing > data, the problem being that the links of the chain will need to > remember their previous state when called again, so their output is a > function of previous data + fresh data. I guess their state could be > written to a file, then re-read.
Yes. Just to expand: the typical processing involves three files: 1) the input which is the output of the preceding stage 2) the output which will form input to the next stage 3) the job log. This will contain references to any input data items that failed to process - typically these will be manually inspected, corrected and a new file created and submitted at the end of the batch run. BUT 3) will also contain the sequence number of the last file and/or last data item processed so that when the next cycle runs it knows where to start. It is this belt and braces approach to data processing and error recovery that makes mainframes so reliable, not just the hardware, but the whole culture there is geared to handling failure and being able to *recover* not just report on it. After all its the mainframes where the really mission critical software of any large enterprise runs! As an ex Unix head I learned an awful lot about reliable computing from the 18 months I spent working on a mainframe project. These guys mostly live in a highly specialised microcosm of their own but they have learned a lot of powerful tricks over the last 40 years that the rest of us ignore at our peril. I strongly recommend that anyone who gets the chance of *a short* contract in mainframe land, with training, to grab the opportunity with both hands! < Steps off soapbox now :-) > Alan G Author of the Learn to Program web tutor http://www.freenetpages.co.uk/hp/alan.gauld _______________________________________________ Tutor maillist - [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/tutor
