HISTORY:
We have this new application to generate ~1300 pdf letters using
<cfreport> functionality. To streamline the processing, <cfthread> tags
where used to asynchronously process 10 reports at a time, witting each
finished report to a file. This feature worked just fine during small
scale testing creating up to 100 reports. But when first attempting to
run all 1300 reports the ColdFusion came crashing down with JRun
out-of-memory errors.
We dived into the CF8 server monitor to see if we could get a hint on
what was actually happening. The first thing I see is that in 'REQUEST
STATISTICS > Active ColdFusion Threads" is that all the threads created
by this process never died. Repeat a run and more threads are created,
but none of them die. Then looking at "MEMORY USAGE > Memory Usage
Summary" we could see that as these perputual threads where created more
and more memory was being gobbled up by the JVM and never being released.
So this started us down a path to tune our JVM to make better use of the
servers resources and throttle the process so that these 1300 reports
could be made with these resources. We added some code that seemed to
cause the threads to be closed, but the JVM memory would still grow and
grow until failure. We then made some tweeks to jvm.config suggested in
the previous discussion and the first thing is that another process,
that previously worked fine, stopped running. I restored the original
default JVM settings and this other process started working again. I
then, piece by piece, modified the JVM configuration as suggested and
got something that does not prevent the running of current code. But
not our problem code is creating un-dieing threads again.
CURRENT STATE:
jvm.config
----------
# Arguments to VM
java.args=-server -Xmx1024m -Xms1024m -Dsun.io.useCanonCaches=false
-XX:MaxPermSize=192m
-XX:PermSize=192m -server -XX:+UseParallelGC
-Dsun.rmi.dgc.client.gcInterval=60000 -server
-Dsun.rmi.dgc.server.gcInterval=60000
-Dcoldfusion.rootDir={application.home}/
allren.cfm
---------
<cfinclude template = "scripts.cfm">
<cfset Application.DSN = LIC_DSN>
<cfset pathDate = #DatePart("yyyy", Now()).toString()#/>
<cfset pIssueDate = #PadDateSegment(DatePart("m", Now()))# & "/" &
#PadDateSegment(DatePart("d", Now()))# & "/" &
#DatePart("yyyy", Now()).toString().substring(2)#>
<!--- GET RENEWAL ID --->
<cfstoredproc procedure="REPORT_UTILS.getBatchRenewals"
dataSource = "#LIC_DSN#">
<cfprocresult name = ResultFirms>
</cfstoredproc>
<!--- REPORT --->
<cfset threadList = "">
<cfloop index = "i" from="1" to="#ResultFirms.recordcount#">
<cfset threadList = listAppend(threadList,"batchRenew_thread#i#")>
<cfthread name="batchRenew_thread#i#" threadIndex="#i#" action="run">
<!--- GET RENEWAL ID --->
<cfset vRenewalSeq = ""/>
<cfstoredproc procedure="REPORT_UTILS.getRenewalSequenceId"
dataSource = "#LIC_DSN#">
<cfprocparam type="out" variable="vRenewalSeq"
CFSQLType="CF_SQL_VARCHAR">
</cfstoredproc>
<!--- PATH --->
<cfset pathDate = #DatePart("yyyy", Now()).toString()#/>
<cfset vOutputFilePath = RENEWALS_FILE_PATH/>
<cfset vOutputFilePath &= pathDate & "\"/>
<cfset vOutputFilePath &=
Trim(#ResultFirms["firmno"][threadIndex]#) & "\"/>
<!--- CREATE OUTPUT PATH --->
<cfif DirectoryExists(#vOutputFilePath#) is False>
<cfdirectory action="create" directory=#vOutputFilePath#/>
</cfif>
<!--- FILE NAME --->
<cfset vOutputFilePath &= GetFileName()/>
<!--- BUILD REPORT --->
<cfreport template="ren.cfr" format="pdf" overwrite="yes"
filename=#vOutputFilePath#>
<cfreportparam name="pFirmNo"
value=#ResultFirms["firmno"][threadIndex]#>
<cfreportparam name="pIssueDate" value=#DateFormat(pIssueDate,
'mm/dd/yy')#>
<cfreportparam name="pRenewalId" value=#vRenewalSeq#>
</cfreport>
<!--- INSERT RENEWAL --->
<cfstoredproc procedure="REPORT_UTILS.insertRenewal"
dataSource = "#LIC_DSN#">
<cfprocparam type="in" value=#vRenewalSeq#
CFSQLType="CF_SQL_NUMERIC">
<cfprocparam type="in"
value=#ResultFirms["firmno"][threadIndex]# CFSQLType="CF_SQL_NUMERIC">
<cfprocparam type="in" value=#pathDate# CFSQLType="CF_SQL_NUMERIC">
</cfstoredproc>
<cfstoredproc procedure="REPORT_UTILS.insertRenewalProductsForFirm"
dataSource = "#LIC_DSN#">
<cfprocparam type="in" value=#vRenewalSeq#
CFSQLType="CF_SQL_NUMERIC">
<cfprocparam type="in"
value=#ResultFirms["firmno"][threadIndex]# CFSQLType="CF_SQL_NUMERIC">
</cfstoredproc>
</cfthread>
</cfloop>
<cfoutput>#replace(listSort(structKeyList(cfthread),'text'),',','<br/>','ALL')#</cfoutput>
<cfthread action="join" name="#structKeyList(cfthread)#"/>
<cfloop list="#structKeyList(cfthread)#" index="thread">
<cfthread action="terminate" name="#thread#"/>
</cfloop>
<p>BLAH <cfoutput>#timeformat(now(),'HH:MM:SS:l')#</cfoutput></p>
Anybody have any suggestions?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to
date
Get the Free Trial
http://ad.doubleclick.net/clk;192386516;25150098;k
Archive:
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:307421
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
