On 2/19/2012 4:25 PM, Magen Margalit wrote:
We have a daily betch job that is processing as input records which has been collected all day. volume of records is about 5 millions for 24 hours.
With allowance for growth, you're looking at 100-200 records per second, and that's well within the capacity of modern equipment. Depending on how the records are created, you have overhead for serialization and backup (do you need an audit trail to recreate records if the file is lost?). You would need to do some testing to see whether the serialization and processing overhead for live processing is tolerable, since the batch presumably offers some savings in overhead.
If the records are created by a single source, then instead of an STC program, you could consider a subtask, a PC routine, or (gasp) even a user SVC. For multiple tasks, either an SVC or a PC routine might do. The preferred method depends on how the data are created, how they are placed into the batch input file, and what processing they are used for.
In order to make systems more "online" we are looking for a way to run the process for each record all day long instead of a daily run, and doing so with minimum as possible application changes.
If you have a PUT, WRITE, or similar function to create the record, it's easy enough to replace that with a PC or SVC call. But you will need to develop contingency plans for data sets becoming full, or other occurrences that could prevent live updating.
One idea that came up is to convert the process to a "self developed" STC which will be triggered by a record on an MQ queue and will run as STC all the batch process programs To me it seems like a bad idea because having a "self developed" STC in production create a maintenance gap (and where there is one STC a second one will soon to follow...)...
When I worked at ADR in the sixties, we developed our own data collection and accounting routine long before IBM offered SMF. We had our own tape library software long before UCC-1 (or TLS). We had our own security software long before RACF (in addition we used IBM passwords, but only for critical and SYS1 data sets). In each case the staff contained more than one person familiar with the software and the ability to maintain it. While IBM is trying to make the system more accessible (i.e., dumbed down) your installation should have enough staff familiar with internals to write and maintain the code. Basically you need whatever the batch process is doing, and add provision for backup and serialization. Of course it's always possible to overcomplicate things to the point where nobody can maintain it.....<g>
Gerhard Postpischil Bradford, VT ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
