The following will show any counts that are wrong: -- List Schema's that have different number of fields than defined in Meta Tables. Select * From ( select a.NAME, a.SCHEMAID, a.NUMFIELDS Schema_Field_Count, (Select count(f.FIELDID) From Field f Where f.SCHEMAID=a.SCHEMAID) db_Field_Count from ARSchema a ) Where Schema_Field_Count <> db_Field_Count;
If you run it in a timed fashion you might find some sort of pattern to the occurrences. Fred From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of Chowdhury, Tauf Sent: Friday, October 14, 2011 4:04 PM To: arslist@ARSLIST.ORG Subject: Re: Oracle-RAC and ARS Issues? ** Yeah, from my understanding, the way RAC works is similar to load balancing. You connect to a virtual SID like "Remedy-RAC" Then, based on some magical parameters, your individual DB connections get spread out amongst the nodes like Node1, Node2, Node3 etc... Now, could there be an issue with the Nodes themselves having replication issues? Maybe... I guess a good test would be to do some SQL queries directly against the nodes after determining which one of your servers are connecting to which node. That is the troubleshooting route we may take but it's late on a Friday and there's a slim chance I'll find any DBA help now. ________________________________ From: Action Request System discussion list(ARSList) on behalf of LJ LongWing Sent: Fri 10/14/2011 3:34 PM To: arslist@ARSLIST.ORG Subject: Re: Oracle-RAC and ARS Issues? ** JD, While I have never seen these symptoms, I agree with you that it appears, somehow to be related with the communication method between your remedy server and your DB. As I understand the RAC, it's a Clustering software, and if not performing properly, I could totally see problems with restarting and the node you were on is not the one you re-connected to. One troubleshooting step you could take would be to shut down all of the nodes of the cluster except one....work with it for awhile...then shutdown and restart to see if you have problems....I know it's a bit extreme...and may not be feasible to your situation, but it's something I would put on my list of 'to try' :) From: Action Request System discussion list(ARSList) [mailto:arslist@ARSLIST.ORG] On Behalf Of JD Hood Sent: Friday, October 14, 2011 1:13 PM To: arslist@ARSLIST.ORG Subject: Oracle-RAC and ARS Issues? ** Having a really weird issue at a client where, when you restart the AR Server service (not every time, but at random nth restarts)... - Random forms fail to load in the server (won't list in client & won't list in the Admin tool, but T,H,B tables are still there with data) - Random fields vanish on forms - All guides (filter & active link) fail to load - could be some other probs, but we didn't look as we have our hands full dealing with the first three The first time this occurred, we found that the numfields value in ARSCHEMA table did not agree with the rowcount in the FIELD table for some CMDB forms (AST: forms). The problem has occurred three times over as many weeks and restoring the DB from backup each time it happens is not feasible as along term resolution -- we need to figure out the root cause. The site is secure and sharing logs, screenshots or webexing is not allowed, so troubleshooting is problematic. I mentioned the symptoms to a colleague and it prompted a vague memory of some issue between an earlier version of ARS and Oracle-RAC, even though the versions were listed as compatible. BMC support can't find any mention and so far (considering the site security restrictions) have only been treating the symptoms. BMC support seems to think the problem is running multiple class manager jobs at the same time, but we aren't doing that. The last time it happened, the system came back up with a *bunch* of corrupted forms (bad field counts between arschema and field tables for a variety of forms) and a "change pending" class manager job showed up out of nowhere -- before system restart, we weren't doing anything in class manager / class manager was idle. In trying to make sense of this, I can only imagine that ARS starts up and connects to the Oracle-RAC cluster in such a screwed-up fashion that it either retrieves (or causes RAC to return) pieces of the database from different points in time -OR- skips parts of the DB -OR- the Oracle-RAC cluster is not keeping itself in sync like we think -OR- <heck-if-i-know-feel-free-to-guess>. All I can say is I've never encountered the behavior before and it's beyond weird. Two questions: 1. Disregarding the issues I've mentioned, does anyone recall ANY (and I mean *any*) issues between Oracle-RAC and any version of ARS? It's the only clue we have at this point and I'm grasping at straws. 2. Has anybody experienced the described weirdness, regardless of ARS version? Thanks, JDHood _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org attend wwrug11 www.wwrug.com ARSList: "Where the Answers Are"