I wouldn't use string concatenation, that's very slow. Rather I'd use
cfsavecontent surrounding the outermost loop and then write those results to
file once rather than with each iteration of the lop. FileIO can be the slowest
part of this whole thing.
Just with those two changes you should s
To: cf-talk
Subject: Re: Improving Performance
so, I got two suggestions from this thread that proved worthy..
#1 - someone asked about the need for the .toString() call - apparently, I
don't actually need this. I ran some tests without it and it worked and
speed things up.
#2 - So
super, glad it helped :-)
On Thu, Aug 18, 2011 at 4:41 PM, Rick Root wrote:
>
> so, I got two suggestions from this thread that proved worthy..
>
> #1 - someone asked about the need for the .toString() call - apparently, I
> don't actually need this. I ran some tests without it and it worked a
so, I got two suggestions from this thread that proved worthy..
#1 - someone asked about the need for the .toString() call - apparently, I
don't actually need this. I ran some tests without it and it worked and speed
things up.
#2 - Someone suggested writing the whole record instead of one fi
so why not try just cfloop over the query normally and output only the
columns you want
#column1# #columns2#
etc...
On Thu, Aug 18, 2011 at 3:39 PM, Rick Root wrote:
>
> >does the original data need to be a 2 dimensional array or can you
> generate
> >it as q uery, then you only have 1 loop
Rick Root wrote:
>
> Can anyone suggest ways that might incrementally improve the performance of
> this code?
>
> I'm using the JavaCSV library to generate a CSV file. It works pretty well,
> but has some difficulty outputting extremely large files (50,000+ records,
> 1800 columns or so)
9:11 AM
To: cf-talk
Subject: Re: Improving Performance
oops.. 180 columns, not 1800 =)
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive
>do you really need to perform the ToString() function ?
>
Yes, since sometimes the data in the query is NOT a "string object" .. it is
sometimes a date, and coldfusion will try to pass it in as a date, and since
CsvWriter is java, and expects a string, it doesn't like getting date objects =)
>does the original data need to be a 2 dimensional array or can you generate
>it as q uery, then you only have 1 loop.
It actually is a query - however the query contains fields that aren't
necessarily going to be in the output file. There wouldn't necessarily have to
be an inside loop but the
does the original data need to be a 2 dimensional array or can you generate
it as q uery, then you only have 1 loop.
On Thu, Aug 18, 2011 at 2:50 PM, Rick Root wrote:
>
> Can anyone suggest ways that might incrementally improve the performance of
> this code?
>
> I'm using the JavaCSV library t
do you really need to perform the ToString() function ?
On Thu, Aug 18, 2011 at 3:06 PM, Rick Root wrote:
>
> Well, I think the JavaCSV method is pretty fast. What I'm wondering is if
> there's a better way to do this line:
>
> resultSet[fieldsArray[i]][resultSet.currentRow].toString() )>
>
>
oops.. 180 columns, not 1800 =)
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/mes
> Well, I think the JavaCSV method is pretty fast. What I'm wondering
> is if there's a better way to do this line:
>
> resultSet[fieldsArray[i]][resultSet.currentRow].toString() )>
And I mean in terms of evaluating the field
resultSet[fieldsArray[i]][resultSet.currentRow]
Like, maybe if I
Rick,
Wow... 1,800 columns is a lot... 50k rows not so much. But together you end
up with 90 million total loop iterations (50k outer loop times 1.8k inner
loop). That's a bit excessive :). Sine you are writing to the file with each
inner loop iteration (each column value in effect) you are appen
Well, I think the JavaCSV method is pretty fast. What I'm wondering is if
there's a better way to do this line:
Let's say you improve the speed of that line by 0.1ms ... that improves a
50,000 row 100 column generation by 500 secnds.
This is part of a dynamic report generation tool, run by
as your are appending to an open file, I would have thought that is the best
way, other than doing it all in memory.
Here are a few things to try.
How about using a database and then exporting the data to a CSV file when
you need it.
Or how about setting up a CSV file as a datasource and writing t
16 matches
Mail list logo
