When we were using CFFILE to append error messages to the end of a log file we had
problems. It turns out that to append, CFFILE reads the *entire* file into memory,
then appends the new info, then saves it to disk.
So I would surmise that your CFFILE action is attempting to load the entire file into
memory then save it to disk.
Chris Norloff
Message-----
>From: Matt Lewis [mailto:[EMAIL PROTECTED]]
>Sent: Monday, August 20, 2001 10:07 PM
>To: CF-Talk
>Subject: Loading extremely large files via CFFILE
>
>
>Hi All.
>
>I'm trying to load an extremely load file via CFFILE. This file is the
>RDF
>content file from the Open Directory Project.
>
>I've already built the script which imports the file, loads into a set
>of
>structures, and then strips out the URLs and inserts into a database.
>This
>script has been tested with a subset of the entire RDF file.
>
>However, when I try to run this script with the entire file
>(approximately
>820Megs in size), I get a memory error. I've resorted to splitting the
>file into smaller segments, but each file is taking a huge amount of
>time
>to load (I've resorted to splitting the file into 10Meg segments).
>
>Can others give me an idea of how they have managed extremely large
>files
>that need to be loaded into a database via CFFILE? The file format is
>too
>complex to simply write a SQL import script.
>
>Thanks.
>
>
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Your ad could be here. Monies from ads go to support these lists and provide more
resources for the community. http://www.fusionauthority.com/ads.cfm
FAQ: http://www.thenetprofits.co.uk/coldfusion/faq
Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/
Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
