That would be awesome Ben. I playing around with your code at the moment but I doubt I'll be able to do much with it this weekend (kids to look after and all that!!)
Plus the fact it's dinnertime!! Cheers Will -----Original Message----- From: Ben Nadel [mailto:[EMAIL PROTECTED] Sent: 15 December 2006 21:11 To: CF-Talk Subject: RE: splitting large csv file into smaller parts Will, If you are interested, I can get you some sample code on how I might parse a CSV file. I could have that for you on Monday. ....................... Ben Nadel Certified Advanced ColdFusion MX7 Developer www.bennadel.com Need ColdFusion Help? www.bennadel.com/ask-ben/ -----Original Message----- From: Will Swain [mailto:[EMAIL PROTECTED] Sent: Friday, December 15, 2006 3:45 PM To: CF-Talk Subject: RE: splitting large csv file into smaller parts Hi Ben, That sounds interesting. I've not done much work with Java, but I'll take a look at the links. Cheers Will -----Original Message----- From: Ben Nadel [mailto:[EMAIL PROTECTED] Sent: 15 December 2006 20:43 To: CF-Talk Subject: RE: splitting large csv file into smaller parts Perhaps you could use a Java buffered reader or a line reader and read in a line at a time. This will stop the server from having to load an entire file at one time. Then for each line you can parse out the fields and apply to the DB or what ever. I have some sample LineNumberReader code here: http://www.bennadel.com/blog/274-Java-Exploration-in-ColdFusion-java-io- LineNumberReader.htm (OR http://www.bennadel.com/index.cfm?dax=blog:274.view) This would create a LONG running page perhaps... Not sure how to deal with that. ......................... Ben Nadel Certified Advanced ColdFusion MX7 Developer www.bennadel.com Need ColdFusion Help? www.bennadel.com/ask-ben/ -----Original Message----- From: Will Swain [mailto:[EMAIL PROTECTED] Sent: Friday, December 15, 2006 3:36 PM To: CF-Talk Subject: splitting large csv file into smaller parts Hi, I have a tool that reads a nightly uploaded CSV file, and imports the data in it into a mySQL database. This works fine, after some messing around with the code to allow for consecutive blank fields in the CSV and stuff like that. Only problem is, some of these CSV files can be quite large, upto 8MB, with maybe 50,000 plus records, and these are giving CF server a bit of a headache. Is there a way I can split the CSV file after upload into, lets say, 8 1Mb files instead, and then process each one in turn. Maybe using CFFILE to read a limited number of rows, write them into another file and then delete them from the original. I can't think of a way to do this. Cheers Will ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Create robust enterprise, web RIAs. Upgrade & integrate Adobe Coldfusion MX7 with Flex 2 http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:264197 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4