Re: The Woes of CFThread -- going out of my mind!
Well, Did you check what is the thread doing when you say the Server is hung? Can you take a thread dump and post that here? You can take the thread dump using Server monitor or you can refer to my post http://coldfused.blogspot.com/2005/11/thread-dumps.html however, I see few potential problems in the code - In the loop, when you fire 10th threads, you join with timeout of 2.5 seconds, and if by that time, all the threads are not finished, you kill them all. So it might happen that threads would not have finished by that time - If you see the docs and if you see my presentation, everywhere it is said that you should use 'terminate' only when absolutely necessary. Don't overuse it as it might lead the thread in some inconsistent state. You should code it in such way that you don't need to terminate the threads most of the time - I don't think you should duplicate the catch. get the message from catch and set it to the thread. For test purpose you can remove catch and see what is the error that you get. Always remember that the thread does not write any output or error to the response. It will be available in the thread scope. Hope that helps. Rupesh ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308845 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Well, Did you check what is the thread doing when you say the Server is hung? Can you take a thread dump and post that here? You can take the thread dump using Server monitor or you can refer to my post http://coldfused.blogspot.com/2005/11/thread-dumps.html however, I see few potential problems in the code - In the loop, when you fire 10th threads, you join with timeout of 2.5 seconds, and if by that time, all the threads are not finished, you kill them all. So it might happen that threads would not have finished by that time - If you see the docs and if you see my presentation, everywhere it is said that you should use 'terminate' only when absolutely necessary. Don't overuse it as it might lead the thread in some inconsistent state. You should code it in such way that you don't need to terminate the threads most of the time - I don't think you should duplicate the catch. get the message from catch and set it to the thread. For test purpose you can remove catch and see what is the error that you get. Always remember that the thread does not write any output or error to the response. It will be available in the thread scope. Hope that helps. Rupesh ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308844 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: The Woes of CFThread -- going out of my mind!
Ian, Can anybody point out what is fundamentally wrong with this code. Such that it has at least a 50% chance of complete failure resulting in a non-responsive ColdFusion server that must be restarted. What gets me is that sometimes it works completely correct and produces all the desired files. The next time, with the exact same code and data, it fails and hangs the server, but generates no exceptions or errors. At least none that I have been able to find. [... clip ...] !--- BUILD REPORT --- cfreport template=ren.cfr format=pdf name=vRenewalReport cfreportparam name=pFirmNo value=#variables.ResultFirms['firmno'][threadIndex]# cfreportparam name=pIssueDate value=#DateFormat(variables.pIssueDate, 'mm/dd/yy')# cfreportparam name=pRenewalId value=#thread.vRenewalSeq# /cfreport cfpdf action=write source=vRenewalReport destination=#thread.vOutputFilePath# overwrite=yes [... clip ...] I wonder if the problem isn't with cfthread / but actually with heavy usage of either cfreport / or cfpdf / tags. Are you creating large reports? It could be a memory issue your running in to. How much RAM is available to the JVM? I would try simplifying things. Does it always work if you comment out the cfreport / and cfpdf / tags? If it does, what happens if you comment out the cfreport / tag and just write a very simply PDF? I could see how if you're generating some large PDFs, that without enough RAM it could drive the JVM nuts. -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308847 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Excellent point Dan! ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308851 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Also, Ian, how many threads are you creating here based on the query results? 10? 10,000? ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308855 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Brian Kotek wrote: Also, Ian, how many threads are you creating here based on the query results? 10? 10,000? For this process 1,371. That is my basic question at this time -- What are the practical limits of the cfthread... Here is my simplest test case to date. I seems to show that there is a limit, but I really don't know where it might be. cfloop from=1 to=1500 index=i cfthread action=run name=th_#i# threadID=#i# cfset thread.foobar = threadID cfset thread.calc = threadID * randRange(1,10) /cfthread /cfloop cfthread action=join name=#structKeyList(cfThread)# timeout=1/ cfdump var=#cfThread# format=text expand=no This code will frequently cause hung threads that NEVER end until the server is restarted. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308856 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Dan G. Switzer, II wrote: Ian, I wonder if the problem isn't with cfthread / but actually with heavy usage of either cfreport / or cfpdf / tags. Are you creating large reports? It could be a memory issue your running in to. How much RAM is available to the JVM? I would try simplifying things. Does it always work if you comment out the cfreport / and cfpdf / tags? If it does, what happens if you comment out the cfreport / tag and just write a very simply PDF? I could see how if you're generating some large PDFs, that without enough RAM it could drive the JVM nuts. -Dan The memory usage of the PDF report generation is defiantly a problem. It is what we where trying to address with the usage of the thread tag. The idea being that by splitting the process into separate threads and throttle them down would allow the server time to clean up memory used in previous iterations. That is a topic for a seperate discussion. How does cfreport... work in large iterations. We found that no matter what we did, the memory used by the report tag would just climb and climb each iteration unless we broke up the process into completely separate HTTP requests. By creating a template that would process a small number of reports per request and then send a page to the browser with a META refresh to re-request the page in one second. This works fine. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308857 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: The Woes of CFThread -- going out of my mind!
Yes the problem is probably that you are creating over 1000 threads, each of which runs a cfreport tag to generate a PDF. You're probably killing the server. CFThread threads are just like any normal CF web thread, and you certainly wouldn't want to try and simultaneously process 1000 HTTP request threads either. You might look at breaking this up and processing them in smaller chunks of, say, 10 or something. On Thu, Jul 10, 2008 at 10:17 AM, Ian Skinner [EMAIL PROTECTED] wrote: Brian Kotek wrote: Also, Ian, how many threads are you creating here based on the query results? 10? 10,000? For this process 1,371. That is my basic question at this time -- What are the practical limits of the cfthread... Here is my simplest test case to date. I seems to show that there is a limit, but I really don't know where it might be. cfloop from=1 to=1500 index=i cfthread action=run name=th_#i# threadID=#i# cfset thread.foobar = threadID cfset thread.calc = threadID * randRange(1,10) /cfthread /cfloop cfthread action=join name=#structKeyList(cfThread)# timeout=1/ cfdump var=#cfThread# format=text expand=no This code will frequently cause hung threads that NEVER end until the server is restarted. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308860 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: The Woes of CFThread -- going out of my mind!
The memory usage of the PDF report generation is defiantly a problem. It is what we where trying to address with the usage of the thread tag. The idea being that by splitting the process into separate threads and throttle them down would allow the server time to clean up memory used in previous iterations. That is a topic for a seperate discussion. How does cfreport... work in large iterations. We found that no matter what we did, the memory used by the report tag would just climb and climb each iteration unless we broke up the process into completely separate HTTP requests. By creating a template that would process a small number of reports per request and then send a page to the browser with a META refresh to re-request the page in one second. This works fine. While certainly not ideal, what if you split things up like this: 1) You create one template that creates a report based on some kind of unique key identifier. 2) You create a second template that loops through all the reports you need to create and call the first template using CFHTTP. This should at least sort of emulate the meta refresh technique, but allows you to automate the process w/out relying on a browser for the meta refresh. -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308861 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: The Woes of CFThread -- going out of my mind!
Brian Kotek wrote: Also, Ian, how many threads are you creating here based on the query results? 10? 10,000? For this process 1,371. That is my basic question at this time -- What are the practical limits of the cfthread... Here is my simplest test case to date. I seems to show that there is a limit, but I really don't know where it might be. cfloop from=1 to=1500 index=i cfthread action=run name=th_#i# threadID=#i# cfset thread.foobar = threadID cfset thread.calc = threadID * randRange(1,10) /cfthread /cfloop cfthread action=join name=#structKeyList(cfThread)# timeout=1/ cfdump var=#cfThread# format=text expand=no This code will frequently cause hung threads that NEVER end until the server is restarted. Threads should be used with caution, because each thread is going to be taking up memory (although I'm not sure how much memory each thread uses.) Spawning off threads like this is in rapid fire succession is a good way for the JVM to run out of memory and once that happens, everything becomes really unstable. I'd try to really limit the number of threads your creating. I'd even look at limiting things to a fixed number of worker threads. Try dividing the work load between like 5 or 6 threads. Have each thread process it's share of the work load (i.e. each work load would handle 1/5 of the workload--so w/500 records, thread one would work on records 1-100, thread 2 would work on records 101-200, etc.) -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308862 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: The Woes of CFThread -- going out of my mind!
He could also still use cfthread, but loop over the query and create the threads in chunks rather than all at once. I don't know how well that will work given the overhead of creating the PDFs, but it would be worth trying in different chunks (2, 5, 10, etc.). On Thu, Jul 10, 2008 at 10:57 AM, Dan G. Switzer, II [EMAIL PROTECTED] wrote: The memory usage of the PDF report generation is defiantly a problem. It is what we where trying to address with the usage of the thread tag. The idea being that by splitting the process into separate threads and throttle them down would allow the server time to clean up memory used in previous iterations. That is a topic for a seperate discussion. How does cfreport... work in large iterations. We found that no matter what we did, the memory used by the report tag would just climb and climb each iteration unless we broke up the process into completely separate HTTP requests. By creating a template that would process a small number of reports per request and then send a page to the browser with a META refresh to re-request the page in one second. This works fine. While certainly not ideal, what if you split things up like this: 1) You create one template that creates a report based on some kind of unique key identifier. 2) You create a second template that loops through all the reports you need to create and call the first template using CFHTTP. This should at least sort of emulate the meta refresh technique, but allows you to automate the process w/out relying on a browser for the meta refresh. -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308865 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: The Woes of CFThread -- going out of my mind!
Ian, Here is my simplest test case to date. I seems to show that there is a limit, but I really don't know where it might be. cfloop from=1 to=1500 index=i cfthread action=run name=th_#i# threadID=#i# cfset thread.foobar = threadID cfset thread.calc = threadID * randRange(1,10) /cfthread /cfloop cfthread action=join name=#structKeyList(cfThread)# timeout=1/ cfdump var=#cfThread# format=text expand=no What if you change the above code to: cfloop from=1 to=300 index=i cfloop from=1 to=5 index=j cfthread action=run name=th_#i# threadID=#(((i-1)*5)+j)# cfset thread.foobar = threadID cfset thread.calc = threadID * randRange(1,10) /cfthread /cfloop cfthread action=join name=#structKeyList(cfThread)#/ cfdump var=#cfthread# format=text expand=no/ /cfloop This should do the exact thing, but it breaks it up to a maximum of 5 threads. -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308866 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: The Woes of CFThread -- going out of my mind!
Dan, I already suggested this approach to IAN nearly a month ago-- twice. http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 If he thought there was merit in that he probably would have tried it already. ~Brad -Original Message- From: Dan G. Switzer, II [mailto:[EMAIL PROTECTED] Threads should be used with caution, because each thread is going to be taking up memory (although I'm not sure how much memory each thread uses.) Spawning off threads like this is in rapid fire succession is a good way for the JVM to run out of memory and once that happens, everything becomes really unstable. I'd try to really limit the number of threads your creating. I'd even look at limiting things to a fixed number of worker threads. Try dividing the work load between like 5 or 6 threads. Have each thread process it's share of the work load (i.e. each work load would handle 1/5 of the workload--so w/500 records, thread one would work on records 1-100, thread 2 would work on records 101-200, etc.) -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308867 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Missing Messages --- was Re: The Woes of CFThread -- going out of my mind!
Brad Wood wrote: Dan, I already suggested this approach to IAN nearly a month ago-- twice. http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 If he thought there was merit in that he probably would have tried it already. ~Brad I suspected I have not been getting all of HOF messages, but now I have confirmation. I had noticed that once in a while I did not recieve a copy of a message I sent to the lists. I also noticed that there were increasing cases where there seemed to be replies to messages that I never receive the original. I never received either of these replies a month ago. I do not see in messages from HOF in the spam folder at my end, but I am not sure what may be happening in between. I'll have to look up Micheal's address when I get back from my 9:00 am meeting to see if anything is getting bounced back. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308869 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: Missing Messages --- was Re: The Woes of CFThread -- going out of my mind!
OK, then. I'll take you off my black list. :) ~Brad -Original Message- From: Ian Skinner [mailto:[EMAIL PROTECTED] Sent: Thursday, July 10, 2008 11:00 AM To: CF-Talk Subject: Missing Messages --- was Re: The Woes of CFThread -- going out of my mind! Brad Wood wrote: Dan, I already suggested this approach to IAN nearly a month ago-- twice. http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 If he thought there was merit in that he probably would have tried it already. ~Brad I suspected I have not been getting all of HOF messages, but now I have confirmation. I had noticed that once in a while I did not recieve a copy of a message I sent to the lists. I also noticed that there were increasing cases where there seemed to be replies to messages that I never receive the original. I never received either of these replies a month ago. I do not see in messages from HOF in the spam folder at my end, but I am not sure what may be happening in between. I'll have to look up Micheal's address when I get back from my 9:00 am meeting to see if anything is getting bounced back. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308876 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: Missing Messages --- was Re: The Woes of CFThread -- going out of my mind!
I'd also check if there's a spam blocker 'upstream' from you, especially if your getting your mail through a business location. In addition, the server is set to try and send mail for 48 hours before dropping it. A network problem that prevents or distorts mail server connections within that time frame could also cause mail to not show up. On Thu, Jul 10, 2008 at 12:00 PM, Ian Skinner [EMAIL PROTECTED] wrote: Brad Wood wrote: Dan, I already suggested this approach to IAN nearly a month ago-- twice. http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 If he thought there was merit in that he probably would have tried it already. ~Brad I suspected I have not been getting all of HOF messages, but now I have confirmation. I had noticed that once in a while I did not recieve a copy of a message I sent to the lists. I also noticed that there were increasing cases where there seemed to be replies to messages that I never receive the original. I never received either of these replies a month ago. I do not see in messages from HOF in the spam folder at my end, but I am not sure what may be happening in between. I'll have to look up Micheal's address when I get back from my 9:00 am meeting to see if anything is getting bounced back. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308878 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: The Woes of CFThread -- going out of my mind!
I'd try to really limit the number of threads your creating. I'd even look at limiting things to a fixed number of worker threads. Try dividing the work load between like 5 or 6 threads. Have each thread process it's share of the work load (i.e. each work load would handle 1/5 of the workload--so w/500 records, thread one would work on records 1-100, thread 2 would work on records 101-200, etc.) Dan and Brad's suggestions are the way to go here. With 1K concurrent threads, I'm surprised it's not failing even quicker. One common pattern of sorts to use here is to create a thread pool and a controller thread. The controller thread could keep track of which reports have and/or haven't been generated, and track the progress of the worker threads. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information! ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308911 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Well I fail. Can anybody point out what is fundamentally wrong with this code. Such that it has at least a 50% chance of complete failure resulting in a non-responsive ColdFusion server that must be restarted. What gets me is that sometimes it works completely correct and produces all the desired files. The next time, with the exact same code and data, it fails and hangs the server, but generates no exceptions or errors. At least none that I have been able to find. cfsetting requesttimeout=1800 cfinclude template = scripts.cfm cfset Application.DSN = LIC_DSN h1PROCESSING/h1 cfset variables.pathDate = #dateFormat(now(),'')#/ cfset variables.pIssueDate = #dateFormat(now(),'mm/dd/yy')# !--- GET RENEWAL ID --- cfstoredproc procedure=REPORT_UTILS.getBatchRenewals dataSource = #LIC_DSN# cfprocresult name = variables.ResultFirms /cfstoredproc cfset variables.threads = ul cfloop index = i from=1 to=#ResultFirms.recordcount# cfif i GT 10cfbreak/cfif!--- Development throttle ... only process first 15 rows --- cfset variables.threads = listAppend(variables.threads,'batchRenew_thread#i#') licfoutputbatchRenew_thread#i#/cfoutput/li cfthread name=batchRenew_thread#i# threadIndex=#i# threadPathDate=#pathDate# action=run cfset var vRenewalReport = cftry !--- GET RENEWAL ID --- cfstoredproc procedure=REPORT_UTILS.getRenewalSequenceId dataSource = #LIC_DSN# cfprocparam type=out variable=thread.vRenewalSeq CFSQLType=CF_SQL_VARCHAR /cfstoredproc !--- PATH --- cfset thread.vOutputFilePath = variables.RENEWALS_FILE_PATH variables.pathDate \ Trim(variables.ResultFirms[firmno][threadIndex]) \/ !--- CREATE OUTPUT PATH --- cfif DirectoryExists(thread.vOutputFilePath) is False cfdirectory action=create directory=#thread.vOutputFilePath#/ /cfif !--- FILE NAME --- cfset thread.vOutputFilePath = GetFileName()/ !--- BUILD REPORT --- cfreport template=ren.cfr format=pdf name=vRenewalReport cfreportparam name=pFirmNo value=#variables.ResultFirms['firmno'][threadIndex]# cfreportparam name=pIssueDate value=#DateFormat(variables.pIssueDate, 'mm/dd/yy')# cfreportparam name=pRenewalId value=#thread.vRenewalSeq# /cfreport cfpdf action=write source=vRenewalReport destination=#thread.vOutputFilePath# overwrite=yes cfset vRenewalReport = !--- INSERT RENEWAL --- cfstoredproc procedure=REPORT_UTILS.insertRenewal dataSource = #LIC_DSN# cfprocparam type=in value=#thread.vRenewalSeq# CFSQLType=CF_SQL_NUMERIC cfprocparam type=in value=#variables.ResultFirms['firmno'][threadIndex]# CFSQLType=CF_SQL_NUMERIC cfprocparam type=in value=#threadPathDate# CFSQLType=CF_SQL_NUMERIC /cfstoredproc cfstoredproc procedure=REPORT_UTILS.insertRenewalProductsForFirm dataSource = #LIC_DSN# cfprocparam type=in value=#thread.vRenewalSeq# CFSQLType=CF_SQL_NUMERIC cfprocparam type=in value=#ResultFirms['firmno'][threadIndex]# CFSQLType=CF_SQL_NUMERIC /cfstoredproc cfcatch type=any cflock name=treadLogLock timeout=5 type=exclusive cfset thread.cfcatch = duplicate(cfcatch) /cflock /cfcatch /cftry /cfthread !--- Pause every x threads to allow garbage collection. --- cfif i MOD 10 EQ 0 cfthread action=join name=#variables.threads# timeout=2500/ !--- Loop over threads and kill any hanging threads --- cfloop collection=#cfThread# item=th cfif cfThread[th].status NEQ 'COMPLETED' cfthread action=terminate name=#th#/ /cfif /cfloop h2SLEEPING cfoutput#timeFormat(now(),hh:mm:ss:l)#/cfoutput/h2 cfset sleep(1) cfset variables.threads = /ulul /cfif /cfloop /ul h2DONE/h2 cfthread action=join name=#structKeyList(cfThread)# timeout=1 / !--- Loop over threads and kill any hanging threads --- cfloop collection=#cfThread# item=th cfif cfThread[th].status NEQ 'COMPLETED' cfthread action=terminate name=#th#/ /cfif /cfloop cfoutput ul cfloop collection=#cfThread# item=th li style=margin-top: .5em;#th# ul cfloop collection=#cfThread[th]# item=key li#key#: nbsp; cfif isSimpleValue(cfThread[th][key])#cfThread[th][key]#cfelsecfdump var=#cfThread[th][key]#/cfif/li /cfloop /ul /li /cfloop /ul /cfoutput ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial
Re: The Woes of CFThread
Hello is this thing on? Raymond Camden wrote: On Mon, Jul 7, 2008 at 1:06 PM, Ian Skinner [EMAIL PROTECTED] wrote: Is anybody using cfthread... functionality? Is it working well for you? Yep. How are you using it? Are there praticle limits on how many threads to create, how long thread generating code can be running? Other factors to 'over using' this thread tool? I have been fighting for most of a month with cfthread... functionality and it is driving me batty. cfthread is a VERY powerful tool. It also takes a lot of care to use correctly. Any pointers on what these cares might be? Run the application one time and it does all the processing, but does not output the results. That is expected. The output is stored in the thread scope for the thread itself. If you use the JOIN action (Which basically means, wait for these threads to end), you can get the output and look at it. This should be documented. (And let me add - be sure you have read the docs COMPLETELY!) I have read the documents several times. I understood the relationship between threads and output. The output I was refering to was after the join statement and consisted of a Done string and a dump of the 'cfThread' structure. I freely admit I do not completely understand all the ins-and-outs which is why I am experimenting. But I have been very frustrated in that my attempts do not seem to yield predictable and repeatable results. Run it a second time and the system locks up and ColdFusion services must be cycled. Most likely it is something in your code. Error trapping seems to be a crap shoot. Nope it isn't. Again, if you look at the thread data (if you use JOIN) you can determine if an error was thrown. Or you can use try/catch to log any errors. It is your responsibility to monitor what happens in the threads. If anything goes wrong in any of the spawned threads and all is lost. Again ColdFusion has to be cycled. Nope. See above. :) What could I add to the following thread to prevent it from generating hung threads that often require a service restart to clear? cfthread name=batchRenew_thread#i# threadIndex=#i# threadPathDate=#pathDate# action=run cfset var vRenewalReport = cftry cfoutput#threadIndex#: #threadPathDate#/cfoutput cfset thread.foobar = threadIndex : threadPathDate cfcatch type=any cflock name=treadLogLock timeout=5 type=exclusive cfset thread.cfcatch = duplicate(cfcatch) /cflock /cfcatch /cftry /cfthread ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308744 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread
Just glancing at this code snippet... I haven't done any work with cfthread recently, but I have implemented it in the past... I wouldn't expect the var keyword at the top to work. The thread does its own encapsulation that's more similar to a custom tag, so I would expect the var keyword to produce an error. I believe that's a parser error, which happens before compiling the template, so the template with the cfthread in it shouldn't even run. But then I've never tested this either. Also I wouldn't lock that catch - it doesn't need a lock around it as far as I can tell and if it did, then I suspect the lock would need to be outside the try tags, because putting it inside the catch will mean you still have a race condition. But I don't think a race condition exists there in the first place because of the cfthread tag's encapsulation. You also shouldn't need to duplicate the cfcatch. I don't think it'll hurt anything, but it shouldn't be necessary. hth, ike cfthread name=batchRenew_thread#i# threadIndex=#i# threadPathDate=#pathDate# action=run cfset var vRenewalReport = cftry cfoutput#threadIndex#: #threadPathDate#/cfoutput cfset thread.foobar = threadIndex : threadPathDate cfcatch type=any cflock name=treadLogLock timeout=5 type=exclusive cfset thread.cfcatch = duplicate(cfcatch) /cflock /cfcatch /cftry /cfthread -- s. isaac dealey ^ new epoch isn't it time for a change? ph: 781.769.0723 http://onTap.riaforge.org/blog ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308804 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread
On Mon, Jul 7, 2008 at 1:06 PM, Ian Skinner [EMAIL PROTECTED] wrote: Is anybody using cfthread... functionality? Is it working well for you? Yep. I have been fighting for most of a month with cfthread... functionality and it is driving me batty. cfthread is a VERY powerful tool. It also takes a lot of care to use correctly. Run the application one time and it does all the processing, but does not output the results. That is expected. The output is stored in the thread scope for the thread itself. If you use the JOIN action (Which basically means, wait for these threads to end), you can get the output and look at it. This should be documented. (And let me add - be sure you have read the docs COMPLETELY!) Run it a second time and the system locks up and ColdFusion services must be cycled. Most likely it is something in your code. Error trapping seems to be a crap shoot. Nope it isn't. Again, if you look at the thread data (if you use JOIN) you can determine if an error was thrown. Or you can use try/catch to log any errors. It is your responsibility to monitor what happens in the threads. If anything goes wrong in any of the spawned threads and all is lost. Again ColdFusion has to be cycled. Nope. See above. :) -- === Raymond Camden, VP of Software Dev, Broadchoice Email : [EMAIL PROTECTED] Blog : www.coldfusionjedi.com AOL IM : cfjedimaster Keep up to date with the community: http://www.coldfusionbloggers.org ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308686 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread
Raymond Camden wrote: That is expected. The output is stored in the thread scope for the thread itself. If you use the JOIN action (Which basically means, wait for these threads to end), you can get the output and look at it. This should be documented. (And let me add - be sure you have read the docs COMPLETELY!) Just to quickly address this point. The expected output is not from the threads. It's basically a simple 'I'm Done' statement after the thread loop and join. The individual threads output is files written to the server. What is wierd is that sometimes all the threads have processed as evidanced that all the expected files have been created and there are no threads shown in the CF Monitor. But the output at the end of the file is not shown and the browser is still in waiting mode. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308697 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread
So then look in your code thats running in the thread. Is there something there that can hang? Do you have a deadlock of some sort? On Mon, Jul 7, 2008 at 3:21 PM, Ian Skinner [EMAIL PROTECTED] wrote: Raymond Camden wrote: That is expected. The output is stored in the thread scope for the thread itself. If you use the JOIN action (Which basically means, wait for these threads to end), you can get the output and look at it. This should be documented. (And let me add - be sure you have read the docs COMPLETELY!) Just to quickly address this point. The expected output is not from the threads. It's basically a simple 'I'm Done' statement after the thread loop and join. The individual threads output is files written to the server. What is wierd is that sometimes all the threads have processed as evidanced that all the expected files have been created and there are no threads shown in the CF Monitor. But the output at the end of the file is not shown and the browser is still in waiting mode. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308702 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: The Woes of CFThread
So then look in your code thats running in the thread. Is there something there that can hang? Do you have a deadlock of some sort? I was thinking about that and knowing that he said the threads were writing files I was wondering about the number of threads he's spawning, the number of files being written, the size of those files, what webserver is in use (and whether or not it performs any file-locking and whether or not that file-locking might be invoked, even though I doubt it) and -- the relationship of all that information to the speed of the write-head on the drive. I wouldn't expect an overabundance of file-writing to cause the CF server to seize up completely and need to be restarted, but then I also don't know the whole configuration, am not an expert on server monitoring or hardware resources, etc. There just seem to be a large number of variables potentially affecting it. -- s. isaac dealey ^ new epoch isn't it time for a change? ph: 617.365.5732 http://onTap.riaforge.org/blog ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308714 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
