Dennis, Yes the data will be read later by down stream processing. I do have the option of either putting the data into sqlite at the start (when its read) or putting it into a flat file and then Later loading it into a sqlite db via a downstream job. A great deal of the data columns are simple numeric values and thats where sqlite really shines in that is portable between systems having differing endianness.
Here is a summary of the entire processing where a1/b1 are different processes and probably differnt host platforms. a1, read from producing system, generate output data (be it flat file or sqlite). a2. Compress data file. a3. Transfer compressed datafile to target system. a4. Goto a1 b1. Receive datafile and uncompress. b2. Read datafile and load into Master DB. (This might just be a simple attach). b3. Massage data in Master db ??? b4. Read and process data from MasterDb. Delete or mark as deleteable. b5. Delete processed data from MasterDb. (this could be in a seperate thread). b6. Goto step b1. The nice thing about simply attaching as a DB in step b2 is that when all data is processed from step b4 then step b5 to purge is a simple detach and operating system unlink for the underlying datafile. Which I suspect will be infinately faster than a sql delete command. Thanks, Ken Dennis Cote <[EMAIL PROTECTED]> wrote: Ken wrote: > > This is a write only app. 100% insert. > > Ken, Why bother putting the data into a database if you are never going to read it back out? Other formats, such as a flat text file are much better for logs or archives. If, in fact, you will be reading the data at some point then you need to also compare the time it takes to retrieve the data of interest from your flat file vs the sqlite database. Dennis Cote ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
