There is also a hotfix for 7.1 involving some embedded spaces in Filter Run If quals that was causing similar problems. Wouldn't put it past the devs to reintroduce a problem that was fixed in a previous version. In fact, I just fixed one of those yesterday.
Rick On Jun 28, 2011 7:46 AM, "dcharters" <dchart...@www.charterssoftware.com> wrote: > Do you have AIE and/or Recon jobs running at all? Are those engines even > installed anywhere? > I have also seen this with special characters being passed in Diary fields. > That was a few versions back but something to look at. > > > On Tue, 28 Jun 2011 08:30:24 -0400, Reiser, John J wrote >> Mark, >> I've been working with BMC Support since last week. They have had >> me send many log files trying to capture an event but nothing has >> shown up yet. I'll check into Spotlight. >> >> Theo, >> Already disabled escalations, alerts etc. I even disabled every >> filter with a notify action because that was the first bit of >> workflow that would send arserver.exe running away. The system still >> overloads. >> >> Joe, >> I'll ask the VM admins to check the NIC settings. Since the SQL >> Server is remote and on a huge SAN that side should be ok. The DBA >> said he saw no unusual traffic to the ARSystem db. >> >> Rick, >> There ARE FOUR Lights. Sorry can you tell I'm losing it. >> The VM has 2 CPUs configured. >> VMware Tools says version 8.3.7 build 341836. >> >> LJ, >> I've been capturing Filter, thread, API, SQL logs and sent them to >> BMC Support. They see no long queries or transactions that could be >> causing the high cpu usage. I did combine sql and api. I'll add the >> filter logs and see what kind of timing I get between the three. >> >> Thanks for the feedback. >> If BMC can't solve it today I think I'm going to use Misi's rrrChive >> and export and reload the user data after we rollback the meta(?) >> data from before the problem started. >> >> --- >> John J. Reiser >> Remedy Developer/Administrator >> Senior Software Development Analyst >> Lockheed Martin - MS2 >> The star that burns twice as bright burns half as long. >> Pay close attention and be illuminated by its brilliance. - >> paraphrased by me >> >> -----Original Message----- >> From: Action Request System discussion list(ARSList) >> [mailto:arslist@ARSLIST.ORG] On Behalf Of Walters, Mark Sent: >> Tuesday, June 28, 2011 4:59 AM To: arslist@ARSLIST.ORG Subject: >> EXTERNAL: Re: arserver.exe is consuming 100% cpu - possible DB >> corruption? (Long Post) >> >> Grab a copy of Spotlight on Windows from www.quest.com and you can >> use it to view the various threads within the arserverd.exe and work >> out which one is causing the high CPU load. Once you have this you >> can reference the thread/sql/api/filter logs to see what activity it >> is. >> >> Mark >> >> I work for BMC, I don't speak for them. >> >> -----Original Message----- >> From: Action Request System discussion list(ARSList) >> [mailto:arslist@ARSLIST.ORG] On Behalf Of Reiser, John J Sent: 27 >> June 2011 22:27 To: arslist@ARSLIST.ORG Subject: arserver.exe is >> consuming 100% cpu - possible DB corruption? (Long Post) >> >> Hello Listers, >> ARS 7.6.03 >> MS 2003 Enterprise >> MS SQL 2005 (remote) >> Total home grown system. No OOTB modules. >> >> I have a real stumper here. It even has BMC scratching their heads. >> I have a production system that is experiencing cpu overload that >> runs up to 99 in the processes and sits there. The ARSystem server >> is virtual machine. We thought maybe it was a MS "Patch Tuesday" >> issue and we removed the 10 recent MS patches one at a time and >> restarted the machine each time. The problem still exists after the >> arserver service starts. Sometime immediately and sometimes it will >> sit for 1- 20 minutes before it starts to hog the CPUs. To eliminate >> any other OS and file system issues we grabbed a two week old backup >> image of the server and restored it. The system came back ok for a >> short while and then started to lock up the CPU again. Working with >> BMC I set the logs on and restarted. We saw the system jump to 100% >> within a minute and captured a 10MB arsql.log file. It can force the >> overload at anytime by firing filter workflow with a notification >> action in it. I disabled this one filter but the system still loaded >> up. I added a Filter that ran a 0 and the only action was Goto 1000 >> to jump all Filter actions that fired on the change of the Status >> field in question. Still no joy. I've disabled every piece of Notify >> workflow. That worked the best and kept the system alive for longer >> stretches but we can't run a system that way. >> >> I've come to the realization that there may be corrupted information >> in the DB object tables and I wanted to get some feedback. Using >> rrrChive I can pull a copy of every form's data since, say, two >> weeks ago. Then have the DBA restore the entire system from that >> date. After the restore I would use rrrChive to reload the two >> weeks' data (Modified date' > "06/11/2011") and hope for the best. >> >> Any workflow that was changed in the last two weeks is negligible >> and could be recreated/updated as needed. >> >> Do you think this is a viable solution? >> When I asked the BMC tech if I could dump the T,H & B tables ; >> restore the db and reload the T, H & B tables he reminded me that >> the arschema and other meta tables would probably be out of synch. >> That's when I thought of using rrrChive. >> >> Sorry to be so long winded but I need to get this back online, BMC >> can't find anything in the logs and I don't want to lose the tickets >> we've taken in the last week. >> >> --- >> John J. Reiser >> Remedy Developer/Administrator >> Senior Software Development Analyst >> Lockheed Martin - MS2 >> The star that burns twice as bright burns half as long. >> Pay close attention and be illuminated by its brilliance. - >> paraphrased by me >> >> > _____________________________________________________________________________ > __ >> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org attend >> wwrug11 www.wwrug.com ARSList: "Where the Answers Are" >> >> > _____________________________________________________________________________ > __ >> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org >> attend wwrug11 www.wwrug.com ARSList: "Where the Answers Are" >> >> > _____________________________________________________________________________ > __ >> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org >> attend wwrug11 www.wwrug.com ARSList: "Where the Answers Are" > > > -- > Open WebMail Project (http://openwebmail.org) > > _______________________________________________________________________________ > UNSUBSCRIBE or access ARSlist Archives at www.arslist.org > attend wwrug11 www.wwrug.com ARSList: "Where the Answers Are" _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org attend wwrug11 www.wwrug.com ARSList: "Where the Answers Are"
