Can anyone suggest ways that might incrementally improve the performance of
this code?
I'm using the JavaCSV library to generate a CSV file. It works pretty well,
but has some difficulty outputting extremely large files (50,000+ records,
1800 columns or so)
(formatted pastebin here:
as your are appending to an open file, I would have thought that is the best
way, other than doing it all in memory.
Here are a few things to try.
How about using a database and then exporting the data to a CSV file when
you need it.
Or how about setting up a CSV file as a datasource and writing
Well, I think the JavaCSV method is pretty fast. What I'm wondering is if
there's a better way to do this line:
cfset fileOutput.write(
resultSet[fieldsArray[i]][resultSet.currentRow].toString() )
Let's say you improve the speed of that line by 0.1ms ... that improves a
50,000 row 100 column
Rick,
Wow... 1,800 columns is a lot... 50k rows not so much. But together you end
up with 90 million total loop iterations (50k outer loop times 1.8k inner
loop). That's a bit excessive :). Sine you are writing to the file with each
inner loop iteration (each column value in effect) you are
Well, I think the JavaCSV method is pretty fast. What I'm wondering
is if there's a better way to do this line:
cfset fileOutput.write(
resultSet[fieldsArray[i]][resultSet.currentRow].toString() )
And I mean in terms of evaluating the field
oops.. 180 columns, not 1800 =)
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
do you really need to perform the ToString() function ?
On Thu, Aug 18, 2011 at 3:06 PM, Rick Root rick.r...@gmail.com wrote:
Well, I think the JavaCSV method is pretty fast. What I'm wondering is if
there's a better way to do this line:
cfset fileOutput.write(
does the original data need to be a 2 dimensional array or can you generate
it as q uery, then you only have 1 loop.
On Thu, Aug 18, 2011 at 2:50 PM, Rick Root rick.r...@gmail.com wrote:
Can anyone suggest ways that might incrementally improve the performance of
this code?
I'm using the
does the original data need to be a 2 dimensional array or can you generate
it as q uery, then you only have 1 loop.
It actually is a query - however the query contains fields that aren't
necessarily going to be in the output file. There wouldn't necessarily have to
be an inside loop but the
do you really need to perform the ToString() function ?
Yes, since sometimes the data in the query is NOT a string object .. it is
sometimes a date, and coldfusion will try to pass it in as a date, and since
CsvWriter is java, and expects a string, it doesn't like getting date objects =)
Well now that just makes my whole point moot :)
Mark Kruger - CFG
CF Webtools
www.cfwebtools.com
www.coldfusionmuse.com
O: 402.408.3733 x105
E: mkru...@cfwebtools.com
Skype: markakruger
-Original Message-
From: Rick Root [mailto:rick.r...@gmail.com]
Sent: Thursday, August 18, 2011
Rick Root rick.r...@gmail.com wrote:
Can anyone suggest ways that might incrementally improve the performance of
this code?
I'm using the JavaCSV library to generate a CSV file. It works pretty well,
but has some difficulty outputting extremely large files (50,000+ records,
1800
so why not try just cfloop over the query normally and output only the
columns you want
cfloop query=myquery
cfsavecontent variable=currentrow#column1# #columns2#
etc.../cfsavecontent
cfset fileOutput.write( currentrow.toString() )
/cfloop
On Thu, Aug 18, 2011 at 3:39 PM, Rick Root
so, I got two suggestions from this thread that proved worthy..
#1 - someone asked about the need for the .toString() call - apparently, I
don't actually need this. I ran some tests without it and it worked and speed
things up.
#2 - Someone suggested writing the whole record instead of one
I have a string variable that I'm using dynamically in an XML document, so it's
necessary to use XMLFormat() or HTMLEditFormat() to change special characters
like , , and so that the XML doesn't break.
However, I need to convert those back to their literals before saving to the
DB, i.e., I
super, glad it helped :-)
On Thu, Aug 18, 2011 at 4:41 PM, Rick Root rick.r...@gmail.com wrote:
so, I got two suggestions from this thread that proved worthy..
#1 - someone asked about the need for the .toString() call - apparently, I
don't actually need this. I ran some tests without it
Rick,
Nice! ... I love a success story :)
Mark Kruger - CFG
CF Webtools
www.cfwebtools.com
www.coldfusionmuse.com
O: 402.408.3733 x105
E: mkru...@cfwebtools.com
Skype: markakruger
-Original Message-
From: Rick Root [mailto:rick.r...@gmail.com]
Sent: Thursday, August 18, 2011 10:42 AM
Am I correct? The image upload, using makeunique can't be used
in conjuction with reReplace as in:
cffile action= upload
filefield = image
destination = #expandPath('images\')##reReplace(image,
'[^a-zA-Z0-9_.]', '', 'all')#
accept= image/jpg,
I wouldn't use string concatenation, that's very slow. Rather I'd use
cfsavecontent surrounding the outermost loop and then write those results to
file once rather than with each iteration of the lop. FileIO can be the slowest
part of this whole thing.
Just with those two changes you should
Hello,
I am trying to invoke a java method using the code below but keep getting
method was not found.
cfscript
x = CreateObject(java, com.something.something.int.classname);
x.init();
ret = x.sendResponse(JavaCast(string, mail));
/cfscript
The cfdump of x I can see the method
At that stage, the server has already received the uploaded file.
Action=upload just moves and/or renames the file.
However, I still do not think it will work as you expect. #form.image# only
contains a temporary file path at that point. The original/client file name is
not available until
What is the signature of the method?
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
sendResponse(ServletContext servletContext,HttpServletResponse
response,String authnContext,String mail,MapString, ListString
attributes)
On 8/18/11 10:32 AM, Leigh cfsearch...@yahoo.com wrote:
What is the signature of the method?
sendResponse(ServletContext
servletContext,HttpServletResponse
response,String authnContext,String mail,MapString,
ListString
attributes)
Then you cannot just send a single string. You need to pass in five (5)
arguments. Otherwise, CF will not be able to find the method because it is a
Thank you very much. I am still having a little difficulty. Could you
please tell if this is how it is suppose to look?
sendResponse(JavaCast(string,Value_servletContext),JavaCast(string,V
alue_response),JavaCast(string,value_authnContext),JavaCast(string,
SESSION.mail),JavaCast(structure,
XMLUnFormat() from CFLIB should do the trick.
http://www.cflib.org/index.cfm?event=page.udfbyidudfid=800
--
/Kevin Pepperman
*Never memorize what you can look up in books*.
--Albert_Einstein
~|
Order the Adobe Coldfusion
Another method, which may be a bit more robust is the OWASP ESAPI's
(Enterprise Security API) Encoder methods: decodeForHTML(str) or
canonicalize(str):
http://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/Encoder.html#decodeForHTML%28java.lang.String%29
The last two
Rick,
You want to make sure you upload into a temporary directory that is outside
of the web root first, otherwise as Leigh pointed out the mime type could be
spoofed, and the file could be executed before you've even had a chance to
perform any other validation on it. The link got truncated in
No. The method expects several complex objects, but you are passing in mostly
strings. You need to pass in the correct type of objects.
ie
ServletContext ** complex object
, HttpServletResponse ** complex object
, String ** plain string
, String ** plain
Well I tried J2EE sessions and we are getting the occasional but unacceptable
Session is Invalid error even after changing the web.xml of each instance to
be longer than the session timeout in the application.cfc. It seems that there
may be numerous causes of Session is Invalid in CF8 from
JavaCast(string,Value_servletContext)
JavaCast(string,Value_response)
Also are Value_servletContext and Value_response actual +variables+ or just
strings like they seem. It is difficult to tell.
~|
Order the Adobe
Thank you so much for your help(hand-holding on this).
They are strings. Additionally, all of those values for now are null
except mail which is a session variable.
On 8/18/11 2:09 PM, Leigh cfsearch...@yahoo.com wrote:
JavaCast(string,Value_servletContext)
JavaCast(string,Value_response)
Richard,
One point, going back to your original problem is that in my experience in
order to remove the cookie the browser must match all attributes of the
cookie, so if the cookie path, domain, secure flag, httponly flag differ you
cant delete it unless you match all those parameters. Now I say
Thanks Peter, that's helpful. Any thoughts about the Session is Invalid
problem? Richard.
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
Thanks, Pete!
I'll check out the link!
Rick
-Original Message-
From: Pete Freitag [mailto:p...@foundeo.com]
Sent: Thursday, August 18, 2011 4:52 PM
To: cf-talk
Subject: Re: My limited testing shows this won't work...
Rick,
You want to make sure you upload into a temporary directory
They are strings. Additionally, all of those values for now
are null except mail which is a session variable.
I am not sure what you mean by null in this context ;) Are you saying you are
not sure what to pass into the method OR that the method allows to skip those
parameters and pass in
36 matches
Mail list logo
