On Mon, 21 Oct 2002 11:23:28 -0700, in cf-talk you wrote:

>Hey all - 
>
>I have got a 76,000+ line data file (ZipUSA comma delim file) that I
>need to break into smaller tables in my DB. This needs to happen at
>least monthly, so I wrote a cfm template to automate the process. The
>problem I am having is speed and server load. I have tried using the
>following methods, and am wondering if anybody has any suggestions
>(apart from using SQL's DTS) Platform info: CF 5 on Win2k, IIS 5 writing
>to MSSQL 2000

Years ago I wrote a simple CFX to parse large server logs.
CFX_ReadLine (or something like that). See
http://www.intrafoundation.com/freeware.html.

Basically you pass it a position into a file (starting with 0) then it
returns everything up to the next newline as a string and the position
of the next line after that. Just do this in a for-next loop to get
all the lines in a file.

It's not the fastest thing in the world, but it only reads into memory
one line at a time -- not the entire file. I believe it should read
through files up to at 2GB or more.

You could probably do the same thing in java without much trouble if
you prefer that.

--min


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4
Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk
FAQ: http://www.thenetprofits.co.uk/coldfusion/faq
Get the mailserver that powers this list at http://www.coolfusion.com

Reply via email to