Hi Ben,
That sounds interesting. I've not done much work with Java, but I'll take a
look at the links.
Cheers
Will
-----Original Message-----
From: Ben Nadel [mailto:[EMAIL PROTECTED]
Sent: 15 December 2006 20:43
To: CF-Talk
Subject: RE: splitting large csv file into smaller parts
Perhaps you could use a Java buffered reader or a line reader and read in a
line at a time. This will stop the server from having to load an entire file
at one time. Then for each line you can parse out the fields and apply to
the DB or what ever.
I have some sample LineNumberReader code here:
http://www.bennadel.com/blog/274-Java-Exploration-in-ColdFusion-java-io-
LineNumberReader.htm
(OR http://www.bennadel.com/index.cfm?dax=blog:274.view)
This would create a LONG running page perhaps... Not sure how to deal with
that.
.......................
Ben Nadel
Certified Advanced ColdFusion MX7 Developer www.bennadel.com
Need ColdFusion Help?
www.bennadel.com/ask-ben/
-----Original Message-----
From: Will Swain [mailto:[EMAIL PROTECTED]
Sent: Friday, December 15, 2006 3:36 PM
To: CF-Talk
Subject: splitting large csv file into smaller parts
Hi,
I have a tool that reads a nightly uploaded CSV file, and imports the data
in it into a mySQL database. This works fine, after some messing around with
the code to allow for consecutive blank fields in the CSV and stuff like
that.
Only problem is, some of these CSV files can be quite large, upto 8MB, with
maybe 50,000 plus records, and these are giving CF server a bit of a
headache.
Is there a way I can split the CSV file after upload into, lets say, 8 1Mb
files instead, and then process each one in turn. Maybe using CFFILE to read
a limited number of rows, write them into another file and then delete them
from the original. I can't think of a way to do this.
Cheers
Will
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU
Archive:
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:264187
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe:
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
