RE: The Woes of CFThread -- going out of my mind!
> I'd try to really limit the number of threads your creating. > I'd even look at limiting things to a fixed number of worker > threads. Try dividing the work load between like 5 or 6 > threads. Have each thread process it's share of the work load > (i.e. each work load would handle 1/5 of the workload--so > w/500 records, thread one would work on records 1-100, thread > 2 would work on records 101-200, etc.) Dan and Brad's suggestions are the way to go here. With 1K concurrent threads, I'm surprised it's not failing even quicker. One common pattern of sorts to use here is to create a thread pool and a controller thread. The controller thread could keep track of which reports have and/or haven't been generated, and track the progress of the worker threads. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information! ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308911 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: Missing Messages --- was Re: The Woes of CFThread -- going out of my mind!
I'd also check if there's a spam blocker 'upstream' from you, especially if your getting your mail through a business location. In addition, the server is set to try and send mail for 48 hours before dropping it. A network problem that prevents or distorts mail server connections within that time frame could also cause mail to not show up. On Thu, Jul 10, 2008 at 12:00 PM, Ian Skinner <[EMAIL PROTECTED]> wrote: > Brad Wood wrote: > > Dan, I already suggested this approach to IAN nearly a month ago-- > > twice. > > > > http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 > > http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 > > > > If he thought there was merit in that he probably would have tried it > > already. > > > > ~Brad > > I suspected I have not been getting all of HOF messages, but now I have > confirmation. I had noticed that once in a while I did not recieve a > copy of a message I sent to the lists. I also noticed that there were > increasing cases where there seemed to be replies to messages that I > never receive the original. > > I never received either of these replies a month ago. > > I do not see in messages from HOF in the spam folder at my end, but I am > not sure what may be happening in between. I'll have to look up > Micheal's address when I get back from my 9:00 am meeting to see if > anything is getting bounced back. > > > ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308878 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: Missing Messages --- was Re: The Woes of CFThread -- going out of my mind!
OK, then. I'll take you off my black list. :) ~Brad -Original Message- From: Ian Skinner [mailto:[EMAIL PROTECTED] Sent: Thursday, July 10, 2008 11:00 AM To: CF-Talk Subject: Missing Messages --- was Re: The Woes of CFThread -- going out of my mind! Brad Wood wrote: > Dan, I already suggested this approach to IAN nearly a month ago-- > twice. > > http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 > http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 > > If he thought there was merit in that he probably would have tried it > already. > > ~Brad I suspected I have not been getting all of HOF messages, but now I have confirmation. I had noticed that once in a while I did not recieve a copy of a message I sent to the lists. I also noticed that there were increasing cases where there seemed to be replies to messages that I never receive the original. I never received either of these replies a month ago. I do not see in messages from HOF in the spam folder at my end, but I am not sure what may be happening in between. I'll have to look up Micheal's address when I get back from my 9:00 am meeting to see if anything is getting bounced back. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308876 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Missing Messages --- was Re: The Woes of CFThread -- going out of my mind!
Brad Wood wrote: > Dan, I already suggested this approach to IAN nearly a month ago-- > twice. > > http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 > http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 > > If he thought there was merit in that he probably would have tried it > already. > > ~Brad I suspected I have not been getting all of HOF messages, but now I have confirmation. I had noticed that once in a while I did not recieve a copy of a message I sent to the lists. I also noticed that there were increasing cases where there seemed to be replies to messages that I never receive the original. I never received either of these replies a month ago. I do not see in messages from HOF in the spam folder at my end, but I am not sure what may be happening in between. I'll have to look up Micheal's address when I get back from my 9:00 am meeting to see if anything is getting bounced back. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308869 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: The Woes of CFThread -- going out of my mind!
Dan, I already suggested this approach to IAN nearly a month ago-- twice. http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307447 http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:307626 If he thought there was merit in that he probably would have tried it already. ~Brad -Original Message- From: Dan G. Switzer, II [mailto:[EMAIL PROTECTED] Threads should be used with caution, because each thread is going to be taking up memory (although I'm not sure how much memory each thread uses.) Spawning off threads like this is in rapid fire succession is a good way for the JVM to run out of memory and once that happens, everything becomes really unstable. I'd try to really limit the number of threads your creating. I'd even look at limiting things to a fixed number of worker threads. Try dividing the work load between like 5 or 6 threads. Have each thread process it's share of the work load (i.e. each work load would handle 1/5 of the workload--so w/500 records, thread one would work on records 1-100, thread 2 would work on records 101-200, etc.) -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308867 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: The Woes of CFThread -- going out of my mind!
Ian, >Here is my simplest test case to date. I seems to show that there is a >limit, but I really don't know where it might be. > > > > > > > > > > > What if you change the above code to: This should do the exact thing, but it breaks it up to a maximum of 5 threads. -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308866 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: The Woes of CFThread -- going out of my mind!
He could also still use cfthread, but loop over the query and create the threads in chunks rather than all at once. I don't know how well that will work given the overhead of creating the PDFs, but it would be worth trying in different chunks (2, 5, 10, etc.). On Thu, Jul 10, 2008 at 10:57 AM, Dan G. Switzer, II < [EMAIL PROTECTED]> wrote: > >The memory usage of the PDF report generation is defiantly a problem. > >It is what we where trying to address with the usage of the thread tag. > >The idea being that by splitting the process into separate threads and > >throttle them down would allow the server time to clean up memory used > >in previous iterations. > > > >That is a topic for a seperate discussion. How does work > >in large iterations. We found that no matter what we did, the memory > >used by the report tag would just climb and climb each iteration unless > >we broke up the process into completely separate HTTP requests. > > > >By creating a template that would process a small number of reports per > >request and then send a page to the browser with a META refresh to > >re-request the page in one second. This works fine. > > While certainly not ideal, what if you split things up like this: > > 1) You create one template that creates a report based on some kind of > unique key identifier. > > 2) You create a second template that loops through all the reports you need > to create and call the first template using CFHTTP. > > This should at least sort of emulate the "meta refresh" technique, but > allows you to automate the process w/out relying on a browser for the meta > refresh. > > -Dan > > > ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308865 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: The Woes of CFThread -- going out of my mind!
>Brian Kotek wrote: >> Also, Ian, how many threads are you creating here based on the query >> results? 10? 10,000? >> >For this process 1,371. That is my basic question at this time -- What >are the practical limits of the > >Here is my simplest test case to date. I seems to show that there is a >limit, but I really don't know where it might be. > > > > > > > > > > > > >This code will frequently cause hung threads that NEVER end until the >server is restarted. Threads should be used with caution, because each thread is going to be taking up memory (although I'm not sure how much memory each thread uses.) Spawning off threads like this is in rapid fire succession is a good way for the JVM to run out of memory and once that happens, everything becomes really unstable. I'd try to really limit the number of threads your creating. I'd even look at limiting things to a fixed number of worker threads. Try dividing the work load between like 5 or 6 threads. Have each thread process it's share of the work load (i.e. each work load would handle 1/5 of the workload--so w/500 records, thread one would work on records 1-100, thread 2 would work on records 101-200, etc.) -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308862 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: The Woes of CFThread -- going out of my mind!
>The memory usage of the PDF report generation is defiantly a problem. >It is what we where trying to address with the usage of the thread tag. >The idea being that by splitting the process into separate threads and >throttle them down would allow the server time to clean up memory used >in previous iterations. > >That is a topic for a seperate discussion. How does work >in large iterations. We found that no matter what we did, the memory >used by the report tag would just climb and climb each iteration unless >we broke up the process into completely separate HTTP requests. > >By creating a template that would process a small number of reports per >request and then send a page to the browser with a META refresh to >re-request the page in one second. This works fine. While certainly not ideal, what if you split things up like this: 1) You create one template that creates a report based on some kind of unique key identifier. 2) You create a second template that loops through all the reports you need to create and call the first template using CFHTTP. This should at least sort of emulate the "meta refresh" technique, but allows you to automate the process w/out relying on a browser for the meta refresh. -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308861 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Yes the problem is probably that you are creating over 1000 threads, each of which runs a cfreport tag to generate a PDF. You're probably killing the server. CFThread threads are just like any normal CF web thread, and you certainly wouldn't want to try and simultaneously process 1000 HTTP request threads either. You might look at breaking this up and processing them in smaller chunks of, say, 10 or something. On Thu, Jul 10, 2008 at 10:17 AM, Ian Skinner <[EMAIL PROTECTED]> wrote: > Brian Kotek wrote: > > Also, Ian, how many threads are you creating here based on the query > > results? 10? 10,000? > > > For this process 1,371. That is my basic question at this time -- What > are the practical limits of the > > Here is my simplest test case to date. I seems to show that there is a > limit, but I really don't know where it might be. > > > > > > > > > > > > > > This code will frequently cause hung threads that NEVER end until the > server is restarted. > > ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308860 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Dan G. Switzer, II wrote: > Ian, > I wonder if the problem isn't with but actually with heavy > usage of either or tags. > > Are you creating large reports? It could be a memory issue your running in > to. How much RAM is available to the JVM? > > I would try simplifying things. Does it always work if you comment out the > and tags? > > If it does, what happens if you comment out the tag and just > write a very simply PDF? > > I could see how if you're generating some large PDFs, that without enough > RAM it could drive the JVM nuts. > > -Dan The memory usage of the PDF report generation is defiantly a problem. It is what we where trying to address with the usage of the thread tag. The idea being that by splitting the process into separate threads and throttle them down would allow the server time to clean up memory used in previous iterations. That is a topic for a seperate discussion. How does work in large iterations. We found that no matter what we did, the memory used by the report tag would just climb and climb each iteration unless we broke up the process into completely separate HTTP requests. By creating a template that would process a small number of reports per request and then send a page to the browser with a META refresh to re-request the page in one second. This works fine. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308857 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: The Woes of CFThread -- going out of my mind!
Brian Kotek wrote: > Also, Ian, how many threads are you creating here based on the query > results? 10? 10,000? > For this process 1,371. That is my basic question at this time -- What are the practical limits of the Here is my simplest test case to date. I seems to show that there is a limit, but I really don't know where it might be. This code will frequently cause hung threads that NEVER end until the server is restarted. ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308856 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Also, Ian, how many threads are you creating here based on the query results? 10? 10,000? ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308855 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Excellent point Dan! ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308851 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: The Woes of CFThread -- going out of my mind!
Ian, >Can anybody point out what is fundamentally wrong with this code. Such >that it has at least a 50% chance of complete failure resulting in a >non-responsive ColdFusion server that must be restarted. What gets me >is that sometimes it works completely correct and produces all the >desired files. The next time, with the exact same code and data, it >fails and hangs the server, but generates no exceptions or errors. At >least none that I have been able to find. > > [... clip ...] > > >name="vRenewalReport"> >value="#variables.ResultFirms['firmno'][threadIndex]#"> >value="#DateFormat(variables.pIssueDate, 'mm/dd/yy')#"> > > > >destination="#thread.vOutputFilePath#" overwrite="yes"> > > [... clip ...] I wonder if the problem isn't with but actually with heavy usage of either or tags. Are you creating large reports? It could be a memory issue your running in to. How much RAM is available to the JVM? I would try simplifying things. Does it always work if you comment out the and tags? If it does, what happens if you comment out the tag and just write a very simply PDF? I could see how if you're generating some large PDFs, that without enough RAM it could drive the JVM nuts. -Dan ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308847 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Well, Did you check what is the thread doing when you say the Server is hung? Can you take a thread dump and post that here? You can take the thread dump using Server monitor or you can refer to my post http://coldfused.blogspot.com/2005/11/thread-dumps.html however, I see few potential problems in the code - In the loop, when you fire 10th threads, you join with timeout of 2.5 seconds, and if by that time, all the threads are not finished, you kill them all. So it might happen that threads would not have finished by that time - If you see the docs and if you see my presentation, everywhere it is said that you should use 'terminate' only when absolutely necessary. Don't overuse it as it might lead the thread in some inconsistent state. You should code it in such way that you don't need to terminate the threads most of the time - I don't think you should duplicate the catch. get the message from catch and set it to the thread. For test purpose you can remove catch and see what is the error that you get. Always remember that the thread does not write any output or error to the response. It will be available in the thread scope. Hope that helps. Rupesh ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308845 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: The Woes of CFThread -- going out of my mind!
Well, Did you check what is the thread doing when you say the Server is hung? Can you take a thread dump and post that here? You can take the thread dump using Server monitor or you can refer to my post http://coldfused.blogspot.com/2005/11/thread-dumps.html however, I see few potential problems in the code - In the loop, when you fire 10th threads, you join with timeout of 2.5 seconds, and if by that time, all the threads are not finished, you kill them all. So it might happen that threads would not have finished by that time - If you see the docs and if you see my presentation, everywhere it is said that you should use 'terminate' only when absolutely necessary. Don't overuse it as it might lead the thread in some inconsistent state. You should code it in such way that you don't need to terminate the threads most of the time - I don't think you should duplicate the catch. get the message from catch and set it to the thread. For test purpose you can remove catch and see what is the error that you get. Always remember that the thread does not write any output or error to the response. It will be available in the thread scope. Hope that helps. Rupesh ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308844 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: The Woes of CFThread -- going out of my mind!
Well I fail. Can anybody point out what is fundamentally wrong with this code. Such that it has at least a 50% chance of complete failure resulting in a non-responsive ColdFusion server that must be restarted. What gets me is that sometimes it works completely correct and produces all the desired files. The next time, with the exact same code and data, it fails and hangs the server, but generates no exceptions or errors. At least none that I have been able to find. PROCESSING batchRenew_thread#i# SLEEPING #timeFormat(now(),"hh:mm:ss:l")# DONE #th# #key#: #cfThread[th][key]# ~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308739 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
