Axton, Thank you for the information. I ran both the handle and process monitor. It looks like the run process is the issue. We are getting tons of
We do see the cmd.exe and the java as new processes, and we do see them disappear once the java code is done. But the number of Handles increased and the number of Process <Non-existent Process> also increases. We now have over 6K of Handles for the arserver.exe alone and this within 1 hour (it starts at 1K). And in the information for handles we see "tons" of these Process <Non-existent Process>(11804) 0x4414 0x001F0FFF 0x9CA42830 Process <Non-existent Process>(12800) 0x4418 0x001F0FFF 0x9A89FD88 Process <Non-existent Process>(14896) 0x441C 0x001F0FFF 0x9BD09020 Process <Non-existent Process>(10212) 0x4420 0x001F0FFF 0x9CB78198 Process <Non-existent Process>(2576) 0x4424 0x001F0FFF 0x9A4A6830 So I guess we are on the right track to find what and why. Thank you again. Pascale Sterrett (Boyer) Remedy Technical Lead & Developer Daimler Trucks North America LLC Montgomery Park, 9th floor Portland, OR 97210 U.S.A Phone:503-745-6569 Email:pascale.bo...@daimler.com http://www.daimler-trucksnorthamerica.com axton.gr...@gmail.com Sent by: arslist@ARSLIST.ORG 12/29/2010 02:38 PM Please respond to arslist@ARSLIST.ORG To arslist@ARSLIST.ORG cc Subject Re: arprocess Handles goes up to 78K which crashes the server ** A handle, in terms of a process, are some resource that the process has a hold on. It can be a socket, a file, a pipe to another process or pseudo-device, etc. The handles are growing too high if you are hitting > 8k handles. The question I have is, what are the handles to? If you find that out, you will know what is going wrong. Here is another program from MS that will list the handles given a process: http://technet.microsoft.com/en-us/sysinternals/bb896655 If you want to know more about what they are and what they mean, read this: http://en.wikipedia.org/wiki/File_descriptor On Wed, Dec 29, 2010 at 2:56 PM, <pascale.sterr...@daimler.com> wrote: ** As per our server team, (since I failed to find a understandable definition) handles are the resources being used by a process. The higher the handle count the more resource is been taken. Not sure if this is an accurate vulgarization of the term :o) Thank you, Pascale Sterrett (Boyer) Remedy Technical Lead & Developer Daimler Trucks North America LLC Montgomery Park, 9th floor Portland, OR 97210 U.S.A Phone:503-745-6569 Email:pascale.bo...@daimler.com http://www.daimler-trucksnorthamerica.com axton.gr...@gmail.com Sent by: arslist@ARSLIST.ORG 12/29/2010 12:13 PM Please respond to arslist@ARSLIST.ORG To arslist@ARSLIST.ORG cc Subject Re: arprocess Handles goes up to 78K which crashes the server ** What are the handles against? That'll tell you what's going on. http://technet.microsoft.com/en-us/sysinternals/bb896645 On Wed, Dec 29, 2010 at 10:54 AM, <pascale.sterr...@daimler.com> wrote: ** Hi all, Happy holidays and new years to all. We have an issue with our upgraded production server. Since we upgraded to 7.5 patch 007 (from 7.0.01 patch 008) our server crash every 30 to 40 hours. The reason is it runs out of resources. We noticed that the arserver process run over 78,000 Handles after a few days. We identify the cause as being an escalation that does a run process which launch a batch file that runs a Java program. Absolutely nothing has changed on the code side (Remedy) or on the Java side. When we disable the escalation, the number of Handles average 2,000 for the arserver process. But as soon as we enable that particular escalation, we see the Handles grow by almost 1,000 every hour or so to the point of crashing the server due to lack of resources. We have an opened ticket with BMC , but they are telling us its an OS issue. All our servers are on the same OS, same patches etc. We move the code (both Java and Remedy) on another server that runs the same OS, same OS patching but ARS 7.0.01 patch 008 and the Handles are stable at 2,400 for the arserver. We are adding monitoring tools to the server to try to get more information but all points back to the arserver. The Java code was compiled on 1.5.0_12. We recompiled it with 1.6.0_22 but still same problem. We changed the Java program to make sure that the connections were closing properly. Any one else have this kind of issue? any ideas? As a Band-Aid, we are currently rebooting the server every night. OS: Win2003 server ARS 7.5 patch 007 No OOB all homegrown MSSQL 2005 remote on Win 2003 Thank you, Pascale Sterrett (Boyer) Remedy Technical Lead & Developer Daimler Trucks North America LLC Montgomery Park, 9th floor Portland, OR 97210 U.S.A Phone:503-745-6569 Email:pascale.bo...@daimler.com http://www.daimler-trucksnorthamerica.com If you are not the intended addressee, please inform us immediately that you have received this e-mail in error, and delete it. We thank you for your cooperation. _attend WWRUG11 www.wwrug.comARSlist: "Where the Answers Are"_ _attend WWRUG11 www.wwrug.com ARSlist: "Where the Answers Are"_ If you are not the intended addressee, please inform us immediately that you have received this e-mail in error, and delete it. We thank you for your cooperation. _attend WWRUG11 www.wwrug.com ARSlist: "Where the Answers Are"_ _attend WWRUG11 www.wwrug.com ARSlist: "Where the Answers Are"_ If you are not the intended addressee, please inform us immediately that you have received this e-mail in error, and delete it. We thank you for your cooperation. _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org attend wwrug11 www.wwrug.com ARSList: "Where the Answers Are"
