Hi Chris; I've had 3 successive support calls into BMC regarding this particular error since my AR Server 7.1 upgrade in November. I don't have any answers for you mostly because BMC has no answers for me as yet. The error itself happens sporadically sometimes daily other times weekly and leaving the logs running on a production server hoping for another occurrence doesn't sound like a solid plan as I do have limited space available. I'll keep apprised of our efforts to resolve this issue. Wayne
-----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arsl...@arslist.org] On Behalf Of strauss Sent: Monday, February 23, 2009 11:52 AM To: arslist@ARSLIST.ORG Subject: Server Queues to support ITSM, CMDB, and various patches After watching a data management support group rename job (patch 9005) crash threads on our 7.1.00.002 server several nights in a row during the search phase, strangely enough the AREA thread 390695, I have gone back today to see exactly what threads we have configured, and where we got the information to configure them. Sat Feb 21 00:14:54 2009 390695 : AR System server terminated when a signal/exception was received by the server (ARNOTE 20) Sat Feb 21 00:14:54 2009 0xc0000005 Sat Feb 21 00:14:54 2009 390695 : AR System server terminated -- fatal error encountered (ARNOTE 21) It is a documentation nightmare, in that contradictory information is scattered throughout the guides for ARS, CMDB, ITSM, and various patch docs. Things like the Plugin Loopback RPC Program Number on the Ports and Queues tab of Server Information are virtually undocumented... in some cases it does not even appear in the screen shots to look at (ITSM 7.0.03 Installation Guide), and the entry in the ar.cfg table in the ARS 7.1.00 Confuring Guide leaves you uncertain about how to use the setting. Some of these settings have been clarified in the ITSM 7.0.03 Installation Guide (and nowhere else), such as the settings for the Reconciliation Engine [390698 or 699: 10 10] pg.20. The same guide shows setting the CAI Plugin Registry form to use a custom private queue - we originally used [390630 2 4] from the 7.0 guide pg.70, then changed it to [2 2] when we disabled the Requester Console); the 7.0.03 guide says to use [2 6] on pg.59. All of the screen shots fail to show the Plugin Loopback RPC Program Number field or its value at all (helpful). CMDB 2.1.00 Apparently sets the Plugin Loopback RPC Program Number to 390626 and creates a queue [390626 2 2] Install/Cfg pg.28, and every subsequent patch to the CMDB recreates this line for the queue even if another already exists with larger values. The confusion really begins with the Data Management Tools 9003 and 9005, which instruct you to use the existing Private Queue 390626, which is already selected as the Plugin Loopback RPC Program Number, and modify it to [16 32] and then specify it in the CAI Plugin Registry form (9003 pg.26, 9005 pg.34). This then puts the CAI Plugin used my the data management tool, the Requester Console (or SRM, I guess), and the CMDB on the same queue. Every time you update the CMDB with a patch this queue will get an additional entry in the ar.cfg for [2 2] instead of the [16 32] values, and the extra entry has to be deleted. At this point our queues and threads look like this (production and development): Type RPC # Min Max Ar.cfg Definition Admin 390600 Alert 390601 1 1 Private-RPC-Socket: 390601 1 1 Full Text Index 390602 Escalation 390603 1 1 Private-RPC-Socket: 390603 1 1 Flashboards 390619 Fast 390620 16 32 Private-RPC-Socket: 390620 16 32 List 390635 24 24 Private-RPC-Socket: 390635 24 24 Plugin Loopback 390626 16 32 Private-RPC-Socket: 390626 16 32 Plugin-Loopback-RPC-Socket: 390626 CAI Plug-in Registry = 390626 16 Plugin-Filter-API-Threads: 8 40 Per Patch 9003 pg 26-28 and 9005 pg 34-36 (CMDB Install) 390626 2 2 RESETS VALUES TO ITS OWN DEFAULTS 2 2 CMDB Installation and Configuration Guide pg.28 - delete after install (CAI Plug-in) 390630 2 2 Private-RPC-Socket: 390630 2 2 ITSM 7.0.03 Install Guide pg 59 (7.0 pg 70) But CAI Plug-in Registry no longer uses this queue due to 9003/5 (Approval) 390624 1 1 Approval-RPC-Socket: 390624 Approval Server Loopback - Approval Server Guide pg 40-43 Private 390694 1 1 Private-RPC-Socket: 390694 1 1 AIE Data Exchanges use this thread - Inferred from AIE Docs Ext Private 390698 10 10 Private-RPC-Socket: 390698 10 10 RE-RPC-Socket: 390698 ITSM 7.0.03 Install Guide pg 20 (Minimal details in CMDB Install Cfg pg 172) External Authentication 390695 External-Authentication-RPC-Socket: 390695 QUESTIONS (Finally): 1. How is the Plugin Loopback RPC Program Number _supposed_ to be used, and by which service or services? 2. Which queue is supposed to be reflected in the CAI Plug-in Registry form - the same one specified in the Plugin Loopback RPC Program Number? 3. If the data management tool and CMDB are _both_ set to use the same queue and CAI Plugin, is this why we were seeing problems when trying to run a data management job after hours? BTW, the job ran successfully this morning after 8 AM, without finding something to contend with. Christopher Strauss, Ph.D. Call Tracking Administration Manager University of North Texas Computing & IT Center http://itsm.unt.edu/ [Also posted to BMCDN] ________________________________________________________________________ _______ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: RMI Solutions ARSlist: "Where the Answers Are" _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: RMI Solutions ARSlist: "Where the Answers Are"
