Chris,
Our favourite footer (at a previous project) was:
#EOF#<count_of_detail_records><sum_of_numerical_column1><sum_of_numerical_column2>...
Sometimes there were no numerical columns, sometimes 1 and sometimes 12 or
more. In most cases we were the recipients of the files, so we wrote a
utility that created the footer based on an input specification. When we
received a file we would trim the header and footer into separate files,
then run our footer generation script against the detail records. We then
used diff to compare our footer with the supplied footer. If we didn't get
the same result as the source system then something was wrong. Since we
didn't know and didn't care about how the source system created the
original footer it also meant that effectively the data was verified twice.
For our situation the utility was worthwhile - we ended up with over 100
files using this validation technique from all sorts of source systems and
in each case we just created a 1 line text file detailing the location of
the numerical columns and some info or whether the file was fixed width or
csv.
cheers,
Mark
"Grabowy,
Chris" To: Multiple recipients of list ORACLE-L
<[EMAIL PROTECTED]>
<cgrabowy@fcg. cc:
com> Subject: RE: Flat file generation
integrity ideas...
Sent by:
[EMAIL PROTECTED]
om
25/10/2002
05:01
Please respond
to ORACLE-L
I will have to keep those in mind, if I ever get back onto a UNIX
platform.
Right now, I'm sticking to Tom's suggestion because I religiously follow
KISS.
Thanks!!
-----Original Message-----
Sent: Thursday, October 24, 2002 2:19 PM
To: Multiple recipients of list ORACLE-L
Chris - I would consider checksums. Many systems have built-in checksum
functions. On Unix these are cksum and sum. You could probably write a
simple one that would work on your various systems. Search for checksum
and somewhere on the Internet is probably sample code for one in a
language that will work for you.
Dennis Williams
DBA, 40%OCP
Lifetouch, Inc.
[EMAIL PROTECTED]
-----Original Message-----
Sent: Thursday, October 24, 2002 12:35 PM
To: Multiple recipients of list ORACLE-L
I have to create packages that will generate several flat files of data
from tables that will be sent to other systems to be processed.
I am looking for ideas on how to ensure data integrity in the flat
files.
For example, the expected record count is stored on the first line of
the file to ensure that the correct amount of records was received.
The systems group is chartered to ensure the flat files are correctly
FTPed between systems, so that's covered.
I just worry that if "somehow" a flat file is scrambled then the
scrambled data is loaded into the database, therefore corrupting it.
At this phase, XML is not an option
I keep thinking that some sort of CRC should be stored with each line in
the flat file. And then before the line is loaded into the database,
the CRC is compared against the generated CRC of the just read line.
Has anyone done anything like this? Any examples out there?
Many TIA!!
--
Please see the official ORACLE-L FAQ: http://www.orafaq.com
--
Author: DENNIS WILLIAMS
INET: [EMAIL PROTECTED]
Fat City Network Services -- 858-538-5051 http://www.fatcity.com
San Diego, California -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the
message BODY, include a line containing: UNSUB ORACLE-L (or the name of
mailing list you want to be removed from). You may also send the HELP
command for other information (like subscribing).
--
Please see the official ORACLE-L FAQ: http://www.orafaq.com
--
Author: Grabowy, Chris
INET: [EMAIL PROTECTED]
Fat City Network Services -- 858-538-5051 http://www.fatcity.com
San Diego, California -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<---->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Privileged/Confidential information may be contained in this message.
If you are not the addressee indicated in this message
(or responsible for delivery of the message to such person),
you may not copy or deliver this message to anyone.
In such case, you should destroy this message and kindly notify the sender
by reply e-mail or by telephone on (61 3) 9612-6999.
Please advise immediately if you or your employer does not consent to
Internet e-mail for messages of this kind.
Opinions, conclusions and other information in this message
that do not relate to the official business of
Transurban City Link Ltd
shall be understood as neither given nor endorsed by it.
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<---->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
--
Please see the official ORACLE-L FAQ: http://www.orafaq.com
--
Author: Mark Richard
INET: [EMAIL PROTECTED]
Fat City Network Services -- 858-538-5051 http://www.fatcity.com
San Diego, California -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
