RE: cfdirectory = slow
If you are only pulling one file back, why are you using CFDIRECTORY at all? You already know the filename, so you can just use fileExists() to check if it's there, and CFFILE to read it in if you need the contents. -Original Message- From: Nelson Winters [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 10:24 AM To: CF-Talk Subject: cfdirectory = slow Where do I report issues with CFMX to Macromedia?I know I'm not the first to discover this, but cfdirectory is extremely inefficient when dealing with large directories.I don't know if this is a problem with J2EE platform that CFMX runs on top of or what, but it shouldn't take several seconds to do a cfdirectory where I'm pulling only one file back, even if the directory has a 100,000+ files in it.It seems like cfdirectory loads the entire directory contents into memory and then applies the filter. Below is some sample code if anyone wants to see what I'm talking about. -Nelson !--- Create a bunch of files --- cfset vDir = d:\test cfset NumberOfFilesToCreate = 10 cfif NOT DirectoryExists(vDir) cfdirectory action="" directory=#vDir# /cfif cfloop from=1 to=#NumberOfFilesToCreate# index=i cffile action="" file=#vDir#\z#i#.txt output=This is file number #i#. /cfloop !--- Get File Info --- !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.0 Transitional//EN html body cfset start = GetTickCount() cfset tmp = Randomize(right(start, 8)) cfset fileIndex = RandRange(1, 100) cfif FileExists(#vDir#\z#fileIndex#.txt) cfdirectory action="" directory=#vDir# name=dirTest filter=z#fileIndex#.txt cfdump var=#dirTest# cfoutput@#fileContents#@/cfoutput cfelse cfoutputFile not found./cfoutput /cfif hrProcessing Time: cfoutput#GetTickCount()-Start#/cfoutput /body /html [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
Re: cfdirectory = slow
Thanks for the suggestion.In my case, I actually need to get the LastModifiedDate provided by cfdirectory.I can work around the problem by calling some Delphi code a coworker created, but I'd like to see the problem get fixed.This really shouldn't be a problem.In this particular case I'm only retrieving 1 file, however, this issue would affect any filtered directory listing. -Nelson - Original Message - From: Barney Boisvert To: CF-Talk Sent: Tuesday, October 21, 2003 1:46 PM Subject: RE: cfdirectory = slow If you are only pulling one file back, why are you using CFDIRECTORY at all? You already know the filename, so you can just use fileExists() to check if it's there, and CFFILE to read it in if you need the contents. -Original Message- From: Nelson Winters [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 10:24 AM To: CF-Talk Subject: cfdirectory = slow Where do I report issues with CFMX to Macromedia?I know I'm not the first to discover this, but cfdirectory is extremely inefficient when dealing with large directories.I don't know if this is a problem with J2EE platform that CFMX runs on top of or what, but it shouldn't take several seconds to do a cfdirectory where I'm pulling only one file back, even if the directory has a 100,000+ files in it.It seems like cfdirectory loads the entire directory contents into memory and then applies the filter. Below is some sample code if anyone wants to see what I'm talking about. -Nelson !--- Create a bunch of files --- cfset vDir = d:\test cfset NumberOfFilesToCreate = 10 cfif NOT DirectoryExists(vDir) cfdirectory action="" directory=#vDir# /cfif cfloop from=1 to=#NumberOfFilesToCreate# index=i cffile action="" file=#vDir#\z#i#.txt output=This is file number #i#. /cfloop !--- Get File Info --- !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.0 Transitional//EN html body cfset start = GetTickCount() cfset tmp = Randomize(right(start, 8)) cfset fileIndex = RandRange(1, 100) cfif FileExists(#vDir#\z#fileIndex#.txt) cfdirectory action="" directory=#vDir# name=dirTest filter=z#fileIndex#.txt cfdump var=#dirTest# cfoutput@#fileContents#@/cfoutput cfelse cfoutputFile not found./cfoutput /cfif hrProcessing Time: cfoutput#GetTickCount()-Start#/cfoutput /body /html [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
why not use java ? CFOUTPUT #CreateObject(java, java.io.File).init(FileNameAndPathHere).lastModified()# /CFOUTPUT HTH -Original Message- From: Nelson Winters [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 19:02 To: CF-Talk Subject: Re: cfdirectory = slow Thanks for the suggestion.In my case, I actually need to get the LastModifiedDate provided by cfdirectory.I can work around the problem by calling some Delphi code a coworker created, but I'd like to see the problem get fixed.This really shouldn't be a problem.In this particular case I'm only retrieving 1 file, however, this issue would affect any filtered directory listing. -Nelson - Original Message - From: Barney Boisvert To: CF-Talk Sent: Tuesday, October 21, 2003 1:46 PM Subject: RE: cfdirectory = slow If you are only pulling one file back, why are you using CFDIRECTORY at all? You already know the filename, so you can just use fileExists() to check if it's there, and CFFILE to read it in if you need the contents. -Original Message- From: Nelson Winters [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 10:24 AM To: CF-Talk Subject: cfdirectory = slow Where do I report issues with CFMX to Macromedia?I know I'm not the first to discover this, but cfdirectory is extremely inefficient when dealing with large directories.I don't know if this is a problem with J2EE platform that CFMX runs on top of or what, but it shouldn't take several seconds to do a cfdirectory where I'm pulling only one file back, even if the directory has a 100,000+ files in it.It seems like cfdirectory loads the entire directory contents into memory and then applies the filter. Below is some sample code if anyone wants to see what I'm talking about. -Nelson !--- Create a bunch of files --- cfset vDir = d:\test cfset NumberOfFilesToCreate = 10 cfif NOT DirectoryExists(vDir) cfdirectory action="" directory=#vDir# /cfif cfloop from=1 to=#NumberOfFilesToCreate# index=i cffile action="" file=#vDir#\z#i#.txt output=This is file number #i#. /cfloop !--- Get File Info --- !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.0 Transitional//EN html body cfset start = GetTickCount() cfset tmp = Randomize(right(start, 8)) cfset fileIndex = RandRange(1, 100) cfif FileExists(#vDir#\z#fileIndex#.txt) cfdirectory action="" directory=#vDir# name=dirTest filter=z#fileIndex#.txt cfdump var=#dirTest# cfoutput@#fileContents#@/cfoutput cfelse cfoutputFile not found./cfoutput /cfif hrProcessing Time: cfoutput#GetTickCount()-Start#/cfoutput /body /html _ [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
If you need lastmodified you can get it this way without needing that Delphi
code:
cfscript
function getDateLastModified(path) {
fh = createObject(java, java.io.File);
fh.init(path);
utcOffset = getTimeZoneInfo().utcTotalOffset +
iif(getTimeZoneInfo().isDSTOn, 3600, 0);
return dateAdd(s, fh.lastModified() / 1000 - utcOffset, createDate(1970,
1, 1));
}
pathToFile = getCurrentTemplatePath();
writeOutput(getDateLastModified(pathToFile));
/cfscript
cheers,
barneyb
-Original Message-
From: Nelson Winters [mailto:[EMAIL PROTECTED]
Sent: Tuesday, October 21, 2003 11:02 AM
To: CF-Talk
Subject: Re: cfdirectory = slow
Thanks for the suggestion.In my case, I actually need to get the
LastModifiedDate provided by cfdirectory.I can work around the problem by
calling some Delphi code a coworker created, but I'd like to see the problem
get fixed.This really shouldn't be a problem.In this particular case I'm
only retrieving 1 file, however, this issue would affect any filtered
directory listing.
-Nelson
- Original Message -
From: Barney Boisvert
To: CF-Talk
Sent: Tuesday, October 21, 2003 1:46 PM
Subject: RE: cfdirectory = slow
If you are only pulling one file back, why are you using CFDIRECTORY at
all?
You already know the filename, so you can just use fileExists() to check
if
it's there, and CFFILE to read it in if you need the contents.
-Original Message-
From: Nelson Winters [mailto:[EMAIL PROTECTED]
Sent: Tuesday, October 21, 2003 10:24 AM
To: CF-Talk
Subject: cfdirectory = slow
Where do I report issues with CFMX to Macromedia?I know I'm not the
first
to discover this, but cfdirectory is extremely inefficient when
dealing
with
large directories.I don't know if this is a problem with J2EE
platform
that CFMX runs on top of or what, but it shouldn't take several
seconds to
do a cfdirectory where I'm pulling only one file back, even if the
directory
has a 100,000+ files in it.It seems like cfdirectory loads the
entire
directory contents into memory and then applies the filter.
Below is some sample code if anyone wants to see what I'm talking
about.
-Nelson
!--- Create a bunch of files ---
cfset vDir = d:\test
cfset NumberOfFilesToCreate = 10
cfif NOT DirectoryExists(vDir)
cfdirectory action="" directory=#vDir#
/cfif
cfloop from=1 to=#NumberOfFilesToCreate# index=i
cffile action="" file=#vDir#\z#i#.txt output=This is file
number
#i#.
/cfloop
!--- Get File Info ---
!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.0 Transitional//EN
html
body
cfset start = GetTickCount()
cfset tmp = Randomize(right(start, 8))
cfset fileIndex = RandRange(1, 100)
cfif FileExists(#vDir#\z#fileIndex#.txt)
cfdirectory action="" directory=#vDir# name=dirTest
filter=z#fileIndex#.txt
cfdump var=#dirTest#
cfoutput@#fileContents#@/cfoutput
cfelse
cfoutputFile not found./cfoutput
/cfif
hrProcessing Time: cfoutput#GetTickCount()-Start#/cfoutput
/body
/html
[Todays Threads]
[This Message]
[Subscription]
[Fast Unsubscribe]
[User Settings]
RE: cfdirectory = slow
Is there a similar way to find out the date the file was created? -Patti why not use java ? CFOUTPUT #CreateObject(java, java.io.File).init(FileNameAndPathHere).lastModified()# /CFOUTPUT HTH [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
Try caching the results of cfdirectory in application scope i.e. change: cfdirectory action="" directory=#vDir# name=dirTest filter=z#fileIndex#.txt cfdump var=#dirTest# to: cfif not isDefined(application.dirtest) cfdirectory action="" directory=#vDir# name=application.dirTest filter=z#fileIndex#.txt /cfif cfdump var=#application.dirTest# This way cf only executes the _slow_ cfdirectory tag once (per application cycle). If the directory changes often (i.e. files get added, deleted, etc.,), then I would run a job that periodically deletes the application.dirTest variable and recreates it.This way the processing overhead is still transparent to the user. I think you'll find i/o operations to be slow w/ any application server. HTH [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
>From CFLib.org (dont know which one as i use the CFStudio Control)
/**
* Returns the date/time a file was created. (Windows only)
*
* @param fileAbsolute or relative path to the specified file.
* @return Returns a date/time object.
* @author Rob Brooks-Bilson ([EMAIL PROTECTED])
* @version 1.0, July 24, 2001
*/
function FileDateCreated(path)
{
Var fso= CreateObject(COM, Scripting.FileSystemObject);
Var theFile = fso.Getfile(path);
Return theFile.DateCreated;
}
HTH
-Original Message-
From: Patricia G. L. Hall [mailto:[EMAIL PROTECTED]
Sent: Tuesday, October 21, 2003 15:30
To: CF-Talk
Subject: RE: cfdirectory = slow
Is there a similar way to find out the date the file was created?
-Patti
why not use java ?
CFOUTPUT
#CreateObject(java,
java.io.File).init(FileNameAndPathHere).lastModified()#
/CFOUTPUT
HTH
_
[Todays Threads]
[This Message]
[Subscription]
[Fast Unsubscribe]
[User Settings]
RE: cfdirectory = slow
Nope, not with the native File class.And be careful using the modified date, as it returns anumber of milliseconds since the epoch UTC, so you need to do some conversions.Check out the UDF i included in my last message. it encapsulates all the nastiness into something ready to use by CF. -Original Message- From: Patricia G. L. Hall [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 7:30 AM To: CF-Talk Subject: RE: cfdirectory = slow Is there a similar way to find out the date the file was created? -Patti why not use java ? CFOUTPUT #CreateObject(java, java.io.File).init(FileNameAndPathHere).lastModified()# /CFOUTPUT HTH [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
I was hoping to see how to find the date the file was created with a java
object if I could?I'll check cflib (should have thought of that), but in
the meantime anyone have something?
-Patti
From CFLib.org (dont know which one as i use the CFStudio Control)
/**
* Returns the date/time a file was created. (Windows only)
*
* @param fileAbsolute or relative path to the specified file.
* @return Returns a date/time object.
* @author Rob Brooks-Bilson ([EMAIL PROTECTED])
* @version 1.0, July 24, 2001
*/
function FileDateCreated(path)
{
Var fso= CreateObject(COM, Scripting.FileSystemObject);
Var theFile = fso.Getfile(path);
Return theFile.DateCreated;
}
HTH
-Original Message-
From: Patricia G. L. Hall [mailto:[EMAIL PROTECTED]
Sent: Tuesday, October 21, 2003 15:30
To: CF-Talk
Subject: RE: cfdirectory = slow
Is there a similar way to find out the date the file was created?
-Patti
why not use java ?
CFOUTPUT
#CreateObject(java,
java.io.File).init(FileNameAndPathHere).lastModified()#
/CFOUTPUT
HTH
_
[Todays Threads]
[This Message]
[Subscription]
[Fast Unsubscribe]
[User Settings]
RE: cfdirectory = slow
Ah.Thanks for the info.'Preciate it. -Patti Nope, not with the native File class.And be careful using the modified date, as it returns anumber of milliseconds since the epoch UTC, so you need to do some conversions.Check out the UDF i included in my last message. it encapsulates all the nastiness into something ready to use by CF. -Original Message- From: Patricia G. L. Hall [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 7:30 AM To: CF-Talk Subject: RE: cfdirectory = slow Is there a similar way to find out the date the file was created? -Patti why not use java ? CFOUTPUT #CreateObject(java, java.io.File).init(FileNameAndPathHere).lastModified()# /CFOUTPUT HTH [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
Re: cfdirectory = slow
Mike, Thanks for the info, I'll try it out along with the function Barney mentioned.To answer your question though, I didn't use java because I'm not very experienced using java and there, when trying to solve a problem, it's not the first option that I turn to. It sounds like this will solve the specific problem that I mentioned, but doesn't address the more common instance where a filter is used to pull files matching a pattern using a wildcard.I'm sure there's some other java code to handle this as well, but wouldn't it be nice ifColdFusion was up to handle this.I wonder why ColdFusion doesn't garner more respect than it does in the developer community at large. I would still like an answer to my original question as to how to bring this to macromedia's attention. Thanks, -Nelson - Original Message - From: Mike Townend To: CF-Talk Sent: Tuesday, October 21, 2003 2:13 PM Subject: RE: cfdirectory = slow why not use java ? CFOUTPUT #CreateObject(java, java.io.File).init(FileNameAndPathHere).lastModified()# /CFOUTPUT HTH -Original Message- From: Nelson Winters [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 19:02 To: CF-Talk Subject: Re: cfdirectory = slow Thanks for the suggestion.In my case, I actually need to get the LastModifiedDate provided by cfdirectory.I can work around the problem by calling some Delphi code a coworker created, but I'd like to see the problem get fixed.This really shouldn't be a problem.In this particular case I'm only retrieving 1 file, however, this issue would affect any filtered directory listing. -Nelson - Original Message - From: Barney Boisvert To: CF-Talk Sent: Tuesday, October 21, 2003 1:46 PM Subject: RE: cfdirectory = slow If you are only pulling one file back, why are you using CFDIRECTORY at all? You already know the filename, so you can just use fileExists() to check if it's there, and CFFILE to read it in if you need the contents. -Original Message- From: Nelson Winters [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 10:24 AM To: CF-Talk Subject: cfdirectory = slow Where do I report issues with CFMX to Macromedia?I know I'm not the first to discover this, but cfdirectory is extremely inefficient when dealing with large directories.I don't know if this is a problem with J2EE platform that CFMX runs on top of or what, but it shouldn't take several seconds to do a cfdirectory where I'm pulling only one file back, even if the directory has a 100,000+ files in it.It seems like cfdirectory loads the entire directory contents into memory and then applies the filter. Below is some sample code if anyone wants to see what I'm talking about. -Nelson !--- Create a bunch of files --- cfset vDir = d:\test cfset NumberOfFilesToCreate = 10 cfif NOT DirectoryExists(vDir) cfdirectory action="" directory=#vDir# /cfif cfloop from=1 to=#NumberOfFilesToCreate# index=i cffile action="" file=#vDir#\z#i#.txt output=This is file number #i#. /cfloop !--- Get File Info --- !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.0 Transitional//EN html body cfset start = GetTickCount() cfset tmp = Randomize(right(start, 8)) cfset fileIndex = RandRange(1, 100) cfif FileExists(#vDir#\z#fileIndex#.txt) cfdirectory action="" directory=#vDir# name=dirTest filter=z#fileIndex#.txt cfdump var=#dirTest# cfoutput@#fileContents#@/cfoutput cfelse cfoutputFile not found./cfoutput /cfif hrProcessing Time: cfoutput#GetTickCount()-Start#/cfoutput /body /html _ [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
http://www.macromedia.com/support/email/wishform/ You can give feedback, report bugs, generally complain or whatever there, about any MM product. barneyb -Original Message- From: Nelson Winters [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 12:18 PM To: CF-Talk Subject: Re: cfdirectory = slow Mike, Thanks for the info, I'll try it out along with the function Barney mentioned.To answer your question though, I didn't use java because I'm not very experienced using java and there, when trying to solve a problem, it's not the first option that I turn to. It sounds like this will solve the specific problem that I mentioned, but doesn't address the more common instance where a filter is used to pull files matching a pattern using a wildcard.I'm sure there's some other java code to handle this as well, but wouldn't it be nice ifColdFusion was up to handle this.I wonder why ColdFusion doesn't garner more respect than it does in the developer community at large. I would still like an answer to my original question as to how to bring this to macromedia's attention. Thanks, -Nelson [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
I would still like an answer to my original question as to how to bring this to macromedia's attention. Nelson, I logged bug 53607 for this.Thanks for pointing it out. Interestingly, while working with an test directory containing 31000 files, CFMX 6.1 required 83 seconds to run the cfdirectory tag as measured with getTickCount(), but CF5 on the same machine with same code and same directory required 154 seconds.I ran the test several times and found the times were mostly consistent within about 1 sec. -Steven Erat [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
One further note, the time required for the 'ls' command is only slightly faster than CFMX.The command 'ls' required 76 seconds whereas CFMX required about 84 seconds. (If you run this test yourself, be sure to logout then log back in to avoid any filename caching.)All CF5/CFMX/ls tests were run on RH 7.2. --- sh-2.04$ pwd /tmp/dirlist sh-2.04$ time ls -l /dev/null real 1m16.403s user 0m1.060s sys1m13.330s -- -Original Message- From: Steven Erat [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 6:03 PM To: CF-Talk Subject: RE: cfdirectory = slow I would still like an answer to my original question as to how to bring this to macromedia's attention. Nelson, I logged bug 53607 for this.Thanks for pointing it out. Interestingly, while working with an test directory containing 31000 files, CFMX 6.1 required 83 seconds to run the cfdirectory tag as measured with getTickCount(), but CF5 on the same machine with same code and same directory required 154 seconds.I ran the test several times and found the times were mostly consistent within about 1 sec. -Steven Erat _ [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
RE: cfdirectory = slow
what about `ls -l myfilename.ext` so you only get that one file?It should be really fast.`ls -l | grep myfilename.ext`, on the other hand, should be about the same speed as CFMX, though give the same result.If you've got the time (I don't, or I would), perhaps a quick little java snippet using a FileFilter (you have to write the implementation) to do a basic RE search is in order. -Original Message- From: Steven Erat [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 3:41 PM To: CF-Talk Subject: RE: cfdirectory = slow One further note, the time required for the 'ls' command is only slightly faster than CFMX.The command 'ls' required 76 seconds whereas CFMX required about 84 seconds. (If you run this test yourself, be sure to logout then log back in to avoid any filename caching.)All CF5/CFMX/ls tests were run on RH 7.2. --- sh-2.04$ pwd /tmp/dirlist sh-2.04$ time ls -l /dev/null real 1m16.403s user 0m1.060s sys1m13.330s -- [Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings]
