RE: What is the most like cause lf "Java Heap Space" error?
PKZip is running at a steady 823K. It wouldn't be PKZip, however, because the Java Heap error occurs when the first part of the processing starts, which uses CFFTP to download the files. And I've been watching the processes while today's tasks have been running during the trouble spots and jrun has held stead at about 130,000K and total Physical Memory using has been consistent at about 75%. CPU maxes out frequently, however. -Original Message- From: Rick Faircloth [mailto:r...@whitestonemedia.com] Sent: Tuesday, September 15, 2009 5:20 PM To: cf-talk Subject: RE: What is the most like cause lf "Java Heap Space" error? Well, now I mis"spoke"... I originally set up the unzip commands using batch files, but went to . (Which I assume actually executes a batch process?) Anyway, here one example of the code...perhaps there should be an "exit" command? ??? ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326343 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: What is the most like cause lf "Java Heap Space" error?
Thanks for the info and tips, Brad! Photos are downloaded via cfftp. And yes, cfdirectory and cffile are used to move thing around. RAM's coming soon... -Original Message- From: b...@bradwood.com [mailto:b...@bradwood.com] Sent: Tuesday, September 15, 2009 5:17 PM To: cf-talk Subject: RE: What is the most like cause lf "Java Heap Space" error? Woah, I wouldn't reduce it. If your server is already running out of memory, the last thing it needs is less to work with. Garbage collection is an automatic process of your JVM that reclaims used memory as free space if it is no longer being used. http://en.wikipedia.org/wiki/Garbage_collection_(computer_science) All the things your requests create when they run stay in memory after the thread is finished. GC runs on a schedule and looks for any objects in memory that have no hard references for them and removes them. There's a actually at least two types of GC depending on which part of the heap the objects are living in. It's a rather deep topic, and I recommend you read up on it. http://www.javaworld.com/javaworld/jw-08-1996/jw-08-gc.html Windows process manager will show you how much RAM has been devoted to your JRUN process. SeeFusion (Or Fusion-Reactor) will show you how much is actually being used. Here is some CFML code that will also give you that information: http://www.coldfusionmuse.com/index.cfm/2008/2/12/leaky.heap.jvm If you want it broken down any further, you have to get into some Java tools like JMeter: http://zacster.blogspot.com/2008/03/quick-howto-to-setup-jmeter.html Your next post stated that you are using pkzip25.exe to do the unzipping. Since pkzip runs as its own Windows process with its own memory space, you need to first identify where the memory on your server is going. Use the process manager to see how much each process is consuming. Then you need to break down your tasks into their parts and run each while watching the memory being used. How are the photos "downloaded"? CFHTTP? Are you using cfdirectory and cffile to move them around? You might need to look into some command line executables for doing that heavy lifting stuff outside of the CF process. And buy more RAM. Regardless of whether that really is your problem. your server will thank you. I hate to think of your swap usage with that low of a head room especially with SQL and FTP all running on the same box. ~Brad ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326342 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: What is the most like cause lf "Java Heap Space" error?
Well, now I mis"spoke"... I originally set up the unzip commands using batch files, but went to . (Which I assume actually executes a batch process?) Anyway, here one example of the code...perhaps there should be an "exit" command? ??? -Original Message- From: Claude Schneegans [mailto:schneeg...@internetique.com] Sent: Tuesday, September 15, 2009 5:09 PM To: cf-talk Subject: Re: What is the most like cause lf "Java Heap Space" error? >>I'm running batch files using pkzip25.exe. I may be wrong, (DOS is far away), but aren't batch files supposed to contain some sort of exit command, otherwise they may stay waiting for some one to enter the next command ? ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326341 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: What is the most like cause lf "Java Heap Space" error?
Woah, I wouldn't reduce it. If your server is already running out of memory, the last thing it needs is less to work with. Garbage collection is an automatic process of your JVM that reclaims used memory as free space if it is no longer being used. http://en.wikipedia.org/wiki/Garbage_collection_(computer_science) All the things your requests create when they run stay in memory after the thread is finished. GC runs on a schedule and looks for any objects in memory that have no hard references for them and removes them. There's a actually at least two types of GC depending on which part of the heap the objects are living in. It's a rather deep topic, and I recommend you read up on it. http://www.javaworld.com/javaworld/jw-08-1996/jw-08-gc.html Windows process manager will show you how much RAM has been devoted to your JRUN process. SeeFusion (Or Fusion-Reactor) will show you how much is actually being used. Here is some CFML code that will also give you that information: http://www.coldfusionmuse.com/index.cfm/2008/2/12/leaky.heap.jvm If you want it broken down any further, you have to get into some Java tools like JMeter: http://zacster.blogspot.com/2008/03/quick-howto-to-setup-jmeter.html Your next post stated that you are using pkzip25.exe to do the unzipping. Since pkzip runs as its own Windows process with its own memory space, you need to first identify where the memory on your server is going. Use the process manager to see how much each process is consuming. Then you need to break down your tasks into their parts and run each while watching the memory being used. How are the photos "downloaded"? CFHTTP? Are you using cfdirectory and cffile to move them around? You might need to look into some command line executables for doing that heavy lifting stuff outside of the CF process. And buy more RAM. Regardless of whether that really is your problem. your server will thank you. I hate to think of your swap usage with that low of a head room especially with SQL and FTP all running on the same box. ~Brad Original Message -------- Subject: RE: What is the most like cause lf "Java Heap Space" error? From: "Rick Faircloth" Date: Tue, September 15, 2009 3:30 pm To: cf-talk No zipping...only unzipping. And yes, I do need to add some more RAM...definitely couldn't hurt. CF8, MySQL, FTP, scheduled tasks using a lot of cfthreads set up to run consecutively. I did go into the JVM settings and reduced the max to 256 to see if that helps. I'm not familiar with "garbage collection." What do you mean? ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326340 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: What is the most like cause lf "Java Heap Space" error?
>>I'm running batch files using pkzip25.exe. I may be wrong, (DOS is far away), but aren't batch files supposed to contain some sort of exit command, otherwise they may stay waiting for some one to enter the next command ? ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326339 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: What is the most like cause lf "Java Heap Space" error?
I'm running batch files using pkzip25.exe. I'll have to check and see how the processes look that are running after the first set of scheduled tasks run. I have one set of tasks that download and process data and photos from one MLS company, and then a couple hours later (spread apart on purpose), a second set of tasks run to download and process data and photos from a second MLS company. The first set of tasks runs without issue. It's the second set that errors when the task for downloading the files from the second company starts. The second set has to download about 1 GB of data and files. As soon as it starts, most days, it throws the error. -Original Message- From: Alan Rother [mailto:alan.rot...@gmail.com] Sent: Tuesday, September 15, 2009 4:14 PM To: cf-talk Subject: Re: What is the most like cause lf "Java Heap Space" error? What are you using to process the unzip? Not properly closing the file you worked on can cause a slow memory leak... ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326337 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: What is the most like cause lf "Java Heap Space" error?
No zipping...only unzipping. And yes, I do need to add some more RAM...definitely couldn't hurt. CF8, MySQL, FTP, scheduled tasks using a lot of cfthreads set up to run consecutively. I did go into the JVM settings and reduced the max to 256 to see if that helps. I'm not familiar with "garbage collection." What do you mean? -Original Message- From: b...@bradwood.com [mailto:b...@bradwood.com] Sent: Tuesday, September 15, 2009 4:14 PM To: cf-talk Subject: RE: What is the most like cause lf "Java Heap Space" error? Zipping itself can be a little memory intensive if all those files are being loaded into memory. Even if there is no "memory leak" par se, it is entirely possible you are under-powering the server RAM-wise. If you only have 1 Gig on the entire server, that's really not much. What else runs on this server besides CF? SQL Server? FTP? What are your JVM args for the maximum heap size? With a Gig of system memory, Windows could easily take 1/4 of that, and if you have left CF at the default JVM settings, it is probably only getting 512 megs. Depending on how often this scheduled task runs (and about 3 millions other factors), it may just be a problem of garbage collection not running often enough. I would recommend grabbing SeeFusion, setting up some database logging of memory usage (counters) and running some load tests with a single task. How much does memory increase when it runs. Does it go back down by itself? After how long? Does memory go back down if you run garbage collection manually? That should be a starting point. ~Brad ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326336 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: What is the most like cause lf "Java Heap Space" error?
What are you using to process the unzip? Not properly closing the file you worked on can cause a slow memory leak... -- Alan Rother Adobe Certified Advanced ColdFusion MX 7 Developer Manager, Phoenix Cold Fusion User Group, AZCFUG.org ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326335 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: What is the most like cause lf "Java Heap Space" error?
Zipping itself can be a little memory intensive if all those files are being loaded into memory. Even if there is no "memory leak" par se, it is entirely possible you are under-powering the server RAM-wise. If you only have 1 Gig on the entire server, that's really not much. What else runs on this server besides CF? SQL Server? FTP? What are your JVM args for the maximum heap size? With a Gig of system memory, Windows could easily take 1/4 of that, and if you have left CF at the default JVM settings, it is probably only getting 512 megs. Depending on how often this scheduled task runs (and about 3 millions other factors), it may just be a problem of garbage collection not running often enough. I would recommend grabbing SeeFusion, setting up some database logging of memory usage (counters) and running some load tests with a single task. How much does memory increase when it runs. Does it go back down by itself? After how long? Does memory go back down if you run garbage collection manually? That should be a starting point. ~Brad Original Message Subject: RE: What is the most like cause lf "Java Heap Space" error? From: "Rick Faircloth" Date: Tue, September 15, 2009 2:51 pm To: cf-talk No use of CFImage in this set of tasks...just unzipping, copying, moving, deleting, etc. (Who typed that subject line for this mail: What is the most "like" cause "if" Java Heap Space error?) And English *is* my native language! Yikes! ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326334 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
RE: What is the most like cause lf "Java Heap Space" error?
No use of CFImage in this set of tasks...just unzipping, copying, moving, deleting, etc. (Who typed that subject line for this mail: What is the most "like" cause "if" Java Heap Space error?) And English *is* my native language! Yikes! -Original Message- From: Josh Nathanson [mailto:p...@oakcitygraphics.com] Sent: Tuesday, September 15, 2009 2:41 PM To: cf-talk Subject: RE: What is the most like cause lf "Java Heap Space" error? I seem to remember something about CFIMAGE hogging a lot of RAM...anyone else have more info? -- Josh -Original Message- From: Rick Faircloth [mailto:r...@whitestonemedia.com] Sent: Tuesday, September 15, 2009 11:35 AM To: cf-talk Subject: RE: What is the most like cause lf "Java Heap Space" error? Thanks, Claude... Rick -Original Message- From: Claude Schneegans [mailto:schneeg...@internetique.com] Sent: Tuesday, September 15, 2009 2:21 PM To: cf-talk Subject: Re: What is the most like cause lf "Java Heap Space" error? I can see mostly two reasons: 1. an infinite loop, this would be in your code, 2.. some memroy leak, this would be some in other's code. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326333 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: What is the most like cause lf "Java Heap Space" error?
I seem to remember something about CFIMAGE hogging a lot of RAM...anyone else have more info? -- Josh -Original Message- From: Rick Faircloth [mailto:r...@whitestonemedia.com] Sent: Tuesday, September 15, 2009 11:35 AM To: cf-talk Subject: RE: What is the most like cause lf "Java Heap Space" error? Thanks, Claude... Rick -Original Message- From: Claude Schneegans [mailto:schneeg...@internetique.com] Sent: Tuesday, September 15, 2009 2:21 PM To: cf-talk Subject: Re: What is the most like cause lf "Java Heap Space" error? I can see mostly two reasons: 1. an infinite loop, this would be in your code, 2.. some memroy leak, this would be some in other's code. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326330 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: What is the most like cause lf "Java Heap Space" error?
... hmmm, a "memroy leak" is not good, but a "memory leak" is even worse ;-/ ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326329 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
RE: What is the most like cause lf "Java Heap Space" error?
Thanks, Claude... Rick -Original Message- From: Claude Schneegans [mailto:schneeg...@internetique.com] Sent: Tuesday, September 15, 2009 2:21 PM To: cf-talk Subject: Re: What is the most like cause lf "Java Heap Space" error? I can see mostly two reasons: 1. an infinite loop, this would be in your code, 2.. some memroy leak, this would be some in other's code. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326328 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: What is the most like cause lf "Java Heap Space" error?
I can see mostly two reasons: 1. an infinite loop, this would be in your code, 2.. some memroy leak, this would be some in other's code. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326327 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
What is the most like cause lf "Java Heap Space" error?
I've got a computer set up that does nothing but process data for about 8,000 properties and 80,000 photos every day. The processing is handled by a series of about 30 scheduled tasks that run consecutively. The computer is a few years old and has only about 1 GB of ram and right now the Task Manager is telling me that 82% has been allocated. The CPU is currently at 33%. Is the Java Heap error typically caused by lack of available RAM, or what? When the scheduled task fails to run and throws the Java Heap error, I can turn right back around and manually run the task without problem. And there is typically nothing being processed by the computer when this particular scheduled task that throws the error, almost every day, tries to run. Suggestions? Thanks, Rick --- "Those who hammer their guns into plows will plow for those who do not." - Thomas Jefferson ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:326314 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
