RE: [KCFusion] help reading a text file...
There is an article in this month's CF Developers Journal about using a custom tag called 'querysim' to mimic database calls. The article discusses a number of different uses for it, including using it to parse through a web server log file (space delimited usually) as if it were a database record/table. You might want to check it out, it is Issue 7 and the cover has a big yellow @ symbol on it (should be available at Barnes & Nobles on the Plaza). I can try and scan the article in this weekend if you want and send you a copy. Ryan -Original Message- From: Johnson, Michael [mailto:[EMAIL PROTECTED]] Sent: Friday, August 02, 2002 12:56 PM To: '[EMAIL PROTECTED]' Subject: [KCFusion] help reading a text file... what's the best way to read a very large tab delimeted text file? does cf.. read the whole file into one variable... how does this get effected if you're reading a large file? Mike Johnson Science Application International Corporation (757) 870-9165 Emai: [EMAIL PROTECTED] __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED]
RE: [KCFusion] help reading a text file...
we have tried this and are using it, somewhat different in how things go, but you can set up a datasource to a folder, and then your queries are something like select * from somefile.csv and if there's a header line in the csv, you will get a nice normal query to work with after that point. works out pretty well in small files that are being uploaded, and not knowing the name of the file you're going to be parsing through beforehand. we haven't really tried bringing in a very large file this way, but just to make the query, you can see that it would take a large amount of ram to hold the query object. -Original Message- From: Ellis, Randy [mailto:[EMAIL PROTECTED]] Sent: Friday, August 02, 2002 1:50 PM To: '[EMAIL PROTECTED]' Subject: RE: [KCFusion] help reading a text file... I haven't tried this, but there is a Microsoft ODBC text driver that supports TXT (Text files) and CSV (comma delimited files). You may be able to setup an ODBC connection to this file. Randy Ellis Senior Programmer Analyst City of Overland Park -Original Message- From: Johnson, Michael [mailto:[EMAIL PROTECTED]] Sent: Friday, August 02, 2002 12:56 PM To: '[EMAIL PROTECTED]' Subject: [KCFusion] help reading a text file... what's the best way to read a very large tab delimeted text file? does cf.. read the whole file into one variable... how does this get effected if you're reading a large file? Mike Johnson Science Application International Corporation (757) 870-9165 Emai: [EMAIL PROTECTED] <> __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED] __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED]
RE: [KCFusion] help reading a text file...
I would recommend looking at either a java or COM component that is specifically developed to give you bite sized portions. VB/Com certainly has sequential file access, which doesn't require you to have 1 MB of RAM for every 1 MB of file size. It would just allow you to walk through the file one step at a time. Thanks, Robin Greenhagen President GSI http://www.gsi-kc.com/ Please note our new contact info -- GSI 10053 Lakeview Ave Lenexa, KS 66219 913-438-9800 phone 913-438-4200 fax -Original Message- From: Keith Purtell [mailto:[EMAIL PROTECTED]] Sent: Friday, August 02, 2002 2:00 PM To: [EMAIL PROTECTED] Subject: RE: [KCFusion] help reading a text file... Michael: I've used CFFILE to read a text file, but never tried anything beyond about 3 MB in size. No performance problems so far. snippet (real version dynamically adds the real file name): Keith Purtell, Web/Network Administrator VantageMed Operations (Kansas City) Email: [EMAIL PROTECTED] CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Adaryl Wakefield Sent: Friday, August 02, 2002 1:39 PM To: [EMAIL PROTECTED] Subject: Re: [KCFusion] help reading a text file... The only method i'm aware of is to use then you can execute a query off that. Looks like: http://www.yoursite.net/foo.txt (<--should be quotes around that) name="grabTextFile" delimiter=","> Then you can output it just like a cfquery like so: #grabTextFile.columnName# Thats straight from the text book. I have not had opportunity to try this in real life so I can't comment reliably on performance. A. - Original Message - From: "Johnson, Michael" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, August 02, 2002 12:55 PM Subject: [KCFusion] help reading a text file... > > what's the best way to read a very large tab delimeted text file? > > does cf.. read the whole file into one variable... how does this get > effected if you're reading a large file? > > > Mike Johnson > Science Application International Corporation > (757) 870-9165 > Emai: [EMAIL PROTECTED] > > <> > __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED] __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED]
RE: [KCFusion] help reading a text file...
Michael: I've used CFFILE to read a text file, but never tried anything beyond about 3 MB in size. No performance problems so far. snippet (real version dynamically adds the real file name): Keith Purtell, Web/Network Administrator VantageMed Operations (Kansas City) Email: [EMAIL PROTECTED] CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Adaryl Wakefield Sent: Friday, August 02, 2002 1:39 PM To: [EMAIL PROTECTED] Subject: Re: [KCFusion] help reading a text file... The only method i'm aware of is to use then you can execute a query off that. Looks like: http://www.yoursite.net/foo.txt (<--should be quotes around that) name="grabTextFile" delimiter=","> Then you can output it just like a cfquery like so: #grabTextFile.columnName# Thats straight from the text book. I have not had opportunity to try this in real life so I can't comment reliably on performance. A. - Original Message - From: "Johnson, Michael" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, August 02, 2002 12:55 PM Subject: [KCFusion] help reading a text file... > > what's the best way to read a very large tab delimeted text file? > > does cf.. read the whole file into one variable... how does this get > effected if you're reading a large file? > > > Mike Johnson > Science Application International Corporation > (757) 870-9165 > Emai: [EMAIL PROTECTED] > > <> > __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED]
RE: [KCFusion] help reading a text file...
I haven't tried this, but there is a Microsoft ODBC text driver that supports TXT (Text files) and CSV (comma delimited files). You may be able to setup an ODBC connection to this file. Randy Ellis Senior Programmer Analyst City of Overland Park -Original Message- From: Johnson, Michael [mailto:[EMAIL PROTECTED]] Sent: Friday, August 02, 2002 12:56 PM To: '[EMAIL PROTECTED]' Subject: [KCFusion] help reading a text file... what's the best way to read a very large tab delimeted text file? does cf.. read the whole file into one variable... how does this get effected if you're reading a large file? Mike Johnson Science Application International Corporation (757) 870-9165 Emai: [EMAIL PROTECTED] <> __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED]
RE: [KCFusion] help reading a text file...
It depends on what you want to do with it, but here's an idea.
insert into XXX(xxx, xxx, xxx, xxx,
xxx, xxx, xxx)
VALUES('#element_1#', '#element_2#',
'#element_3#,
'#element_4#', '#element_5#',
'#element_6#',
'#element_7#')
-Original Message-
From: Adaryl Wakefield [mailto:[EMAIL PROTECTED]]
Sent: Friday, August 02, 2002 1:39 PM
To: [EMAIL PROTECTED]
Subject: Re: [KCFusion] help reading a text file...
The only method i'm aware of is to use then you can execute a query
off that. Looks like:
http://www.yoursite.net/foo.txt (<--should be quotes around
that)
name="grabTextFile"
delimiter=",">
Then you can output it just like a cfquery like so:
#grabTextFile.columnName#
Thats straight from the text book. I have not had opportunity to try this in
real life so I can't comment reliably on performance.
A.
- Original Message -
From: "Johnson, Michael" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Friday, August 02, 2002 12:55 PM
Subject: [KCFusion] help reading a text file...
>
> what's the best way to read a very large tab delimeted text file?
>
> does cf.. read the whole file into one variable... how does this get
> effected if you're reading a large file?
>
>
> Mike Johnson
> Science Application International Corporation
> (757) 870-9165
> Emai: [EMAIL PROTECTED]
>
> <>
>
__
The KCFusion.org list and website is hosted by Humankind Systems, Inc.
List Archives http://www.mail-archive.com/cf-list@kcfusion.org
Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED]
To Subscribe mailto:[EMAIL PROTECTED]
To Unsubscribe mailto:[EMAIL PROTECTED]
__
The KCFusion.org list and website is hosted by Humankind Systems, Inc.
List Archives http://www.mail-archive.com/cf-list@kcfusion.org
Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED]
To Subscribe mailto:[EMAIL PROTECTED]
To Unsubscribe mailto:[EMAIL PROTECTED]
Re: [KCFusion] help reading a text file...
The only method i'm aware of is to use then you can execute a query off that. Looks like: http://www.yoursite.net/foo.txt (<--should be quotes around that) name="grabTextFile" delimiter=","> Then you can output it just like a cfquery like so: #grabTextFile.columnName# Thats straight from the text book. I have not had opportunity to try this in real life so I can't comment reliably on performance. A. - Original Message - From: "Johnson, Michael" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, August 02, 2002 12:55 PM Subject: [KCFusion] help reading a text file... > > what's the best way to read a very large tab delimeted text file? > > does cf.. read the whole file into one variable... how does this get > effected if you're reading a large file? > > > Mike Johnson > Science Application International Corporation > (757) 870-9165 > Emai: [EMAIL PROTECTED] > > <> > __ The KCFusion.org list and website is hosted by Humankind Systems, Inc. List Archives http://www.mail-archive.com/cf-list@kcfusion.org Questions, Comments or Glowing Praise.. mailto:[EMAIL PROTECTED] To Subscribe mailto:[EMAIL PROTECTED] To Unsubscribe mailto:[EMAIL PROTECTED]
