We are on patch 007 and have been on it for a long time before this started happening.
It is production so I can't patch the system up to 008 without going through our change planning/release process which will take too long to try right now. Ken. From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of vaibhav wadekar Sent: Thursday, October 25, 2012 10:11 PM To: arslist@ARSLIST.ORG Subject: Re: ARERR 300 Malloc failed on server ** Hello Ken, Can you confirm what Patch level you are on for AR 7.5? The current Patch for 7.5 is I believe Patch 008, Did you try running patch 008 binary for arserver.exe and run into similar issue. Regards/Vaibhav On Thu, Oct 25, 2012 at 6:45 PM, Cecil, Ken <kce...@hubbell.com<mailto:kce...@hubbell.com>> wrote: ** I double checked and I do not have a max attach size configured. I just restarted arserver and tried to attach the large word doc again within a few minutes of starting the server and got the memory allocation error. I also check the autogrow setting on the database and it is set to 100MB. Ken. From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG>] On Behalf Of Jason Miller Sent: Thursday, October 25, 2012 5:22 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server ** Fascinating. Does it happen if you try right after an AR Server restart; before it's memory usage starts to get to its normal operating level? Just to double check you don't have Db-Max-Attach-Size set in the ar.cfg file? Jason On Thu, Oct 25, 2012 at 1:52 PM, Cecil, Ken <kce...@hubbell.com<mailto:kce...@hubbell.com>> wrote: ** Just tried with a 5.9 MB pdf. No go... 10MB text file works! Any ideas? Ken. From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG>] On Behalf Of Jason Miller Sent: Thursday, October 25, 2012 4:43 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server ** Just wondering. Do you keep trying with the same file each time? If so I wonder if there is something goofy going on in how the AR Server compresses and saves the file in the db? Is it only 4.6 MB Word docs? Would a 4.6 MB text file do the same thing? Jason On Thu, Oct 25, 2012 at 1:31 PM, Cecil, Ken <kce...@hubbell.com<mailto:kce...@hubbell.com>> wrote: ** I am consistently reproducing the error by trying to attach a word doc that is about 4.6 MB to a Work Log entry to an Incident (also tried in a task and it fails there too). The error comes back in about 1 second. Here is the API and SQL log lines from when the error occurs. <CLAT> /* Thu Oct 25 2012 15:37:30.6130 */+API call <API > <TID: 0000004088> <RPC ID: 0000399961> <Queue: Fast > <Client-RPC: 390620 > <USER: kcecil > /* Thu Oct 25 2012 14:37:32.7550 */+CE ARCreateEntry -- schema HPD:WorkLog from Remedy User (protocol 14) at IP address 172.16.57.154 <SQL > <TID: 0000004088> <RPC ID: 0000399961> <Queue: Fast > <Client-RPC: 390620 > <USER: kcecil > /* Thu Oct 25 2012 14:37:32.7550 */BEGIN TRANSACTION <SQL > <TID: 0000004088> <RPC ID: 0000399961> <Queue: Fast > <Client-RPC: 390620 > <USER: kcecil > /* Thu Oct 25 2012 14:37:32.7550 */OK <SQL > <TID: 0000004088> <RPC ID: 0000399961> <Queue: Fast > <Client-RPC: 390620 > <USER: kcecil > /* Thu Oct 25 2012 14:37:32.7550 */COMMIT TRANSACTION <SQL > <TID: 0000004088> <RPC ID: 0000399961> <Queue: Fast > <Client-RPC: 390620 > <USER: kcecil > /* Thu Oct 25 2012 14:37:32.7550 */OK <API > <TID: 0000004088> <RPC ID: 0000399961> <Queue: Fast > <Client-RPC: 390620 > <USER: kcecil > /* Thu Oct 25 2012 14:37:32.7550 */-CE FAIL <CLAT> /* Thu Oct 25 2012 15:37:31.3950 */-API call Here is the Performance Graph.... No increase. Everything else seems to be working fine for everyone. I can even attach smaller files ok. I triple checked the file space on all the drives on both the AR server and the DB server and there are at least 15GB free on all of them. I just don't know where to look next. I still haven't received a call back from supt just email asking for the logs Someone suggested using the /3GB boot.ini switch. My question to that would be does that work on a 64bit version of Windows like we have? Ken. From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG>] On Behalf Of Longwing, LJ CTR MDA/IC Sent: Thursday, October 25, 2012 2:34 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server ** How does the performance Graph tab look at the time of trying to add the attachment? And you have verified that you aren't out of filespace? From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of Cecil, Ken Sent: Thursday, October 25, 2012 12:18 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server ** Here are screenshots of what I see for the arserver.exe process before and after trying to attach a file (and getting the error) Try to attaché 1.2 Mb attachment (works) Try to attach 4.7 MB attachment (ARERR 300 Malloc failed) Anybody see anything? Ken. -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of Longwing, LJ CTR MDA/IC Sent: Thursday, October 25, 2012 8:57 AM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server Ken, I would agree that 1.2 on a 64Bit OS is a bit hard to explain the malloc error. How did the overall system resources look at the time? I have found at times that the memory size is a bit hard to determine. I use a SysInternals tool 'Process Explorer' to look at things. One thing that I have found is that the memory on my server (32 Bit) tends to crap out when the 'Virtual Size' hits 2 GB, not necessarily when the 'Private Bytes' hits 2 GB...and I have sometimes seen a great disparity between the two values. I additionally try to look at the 'Max Memory Usage' column (that you can add) to see if at any point it actually spiked higher. -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of Cecil, Ken Sent: Wednesday, October 24, 2012 3:04 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server Simon, The servers are Windows Server 2003 Enterprise 64 bit with 10 GB RAM. David, New changes transported to production over since the beginning of September. LJ, That is what I thought. When everything hit the fan yesterday the arserver.exe process was only using ~1.2 GB of RAM. It is a 64 bit OS so I think that seems pretty low to cause an out of memory issue? Lisa, We get the error in both the Mid Tier and WUT. Thanks guys, Ken. -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of Simon Ellis Sent: Wednesday, October 24, 2012 4:52 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server What O/S are you running on? -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of David M. Clark Sent: Thursday, 25 October 2012 9:43 a.m. To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server "We are getting this error intermittently over the past couple of weeks" Have you recently customized anything in that area? I've seen this happen when access rights to a newly-created field are not properly defined. -D -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of Longwing, LJ CTR MDA/IC Sent: Wednesday, October 24, 2012 3:31 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: Re: ARERR 300 Malloc failed on server Ken, It sounds like your arserver was bouncing up against a upper memory limit. This can happen sometimes with memory leaks that don't properly release memory once used. A restart of the service brings the system back down to initial minimums of startup. You may want to make sure that you are on the most recent patch of 7.5 to ensure that any known issues are taken care of properly. -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of Cecil, Ken Sent: Wednesday, October 24, 2012 1:42 PM To: arslist@ARSLIST.ORG<mailto:arslist@ARSLIST.ORG> Subject: ARERR 300 Malloc failed on server ** We are getting this error intermittently over the past couple of weeks. The most consistent time it is occurring is when trying to attach a largish attachment (around 4MB). Some have also received it when trying to open an attachment. Yesterday for awhile we were getting the error anytime the SDA's were trying to attach any attachment to an Incident. I ended up restarting the ar server service. Has anybody run across something similar? (Arerr 300 when working with attachments) I have a support request in with BMC and waiting for a reply. The KB articles I found did not seem to apply. Thanks Ken Cecil 574-283-4248<tel:574-283-4248> kce...@hubbell.com<mailto:kce...@hubbell.com> AR 7.5 ITSM 7.6 Windows 2003 Enterprise 64bit SQL 2008 ________________________________ This email and any files transmitted with it are confidential and intended solely for the addressee. If you have received this email in error please notify the system manager. Subject to local law, communications (including traffic data) with Hubbell may be monitored by our systems [or a third party's systems on our behalf] for the purposes of security and the assessment of internal compliance with Hubbell policies. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.hubbell.com<http://www.hubbell.com> <http://www.hubbell.com/> - Hubbell Incorporated _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org<http://www.arslist.org> attend wwrug12 www.wwrug12.com<http://www.wwrug12.com> ARSList: "Where the Answers Are" _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org<http://www.arslist.org> attend wwrug12 www.wwrug12.com<http://www.wwrug12.com> ARSList: "Where the Answers Are" _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org<http://www.arslist.org> attend wwrug12 www.wwrug12.com<http://www.wwrug12.com> ARSList: "Where the Answers Are" ****************************************************************************** This email and any files transmitted with it are confidential and intended solely for the addressee. If you have received this email in error please notify the system manager. Subject to local law, communications (including traffic data) with Hubbell may be monitored by our systems [or a third party's systems on our behalf] for the purposes of security and the assessment of internal compliance with Hubbell policies. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com<http://www.Hubbell.com> - Hubbell Incorporated _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org<http://www.arslist.org> attend wwrug12 www.wwrug12.com<http://www.wwrug12.com> ARSList: "Where the Answers Are" _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org<http://www.arslist.org> attend wwrug12 www.wwrug12.com<http://www.wwrug12.com> ARSList: "Where the Answers Are" _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers Are"_ _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org attend wwrug12 www.wwrug12.com ARSList: "Where the Answers Are"