Well, Microsoft SQL is known as very robust DB.  The issue is the query
being passed to it.  Unfortunately, because of the WUT's "open" querying
system, the end user can define very poor queries and pass it to the
underlying DB.

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Darrell Reading
Sent: Wednesday, December 03, 2008 4:10 PM
To: arslist@ARSLIST.ORG
Subject: Re: UPDATE: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

I noticed that you can turn off the escalation and set the tables to
lock row.  The problem appears that it might be an overall performance
hit, according to Microsoft.  You could run your ARServer on a Windows
server with the DB on an alternative OS with a more robust DB....  Just
a thought.  BTW, I have little to no experience with Microsoft's SQL
server, so I will have to defer to someone with more experience in that
realm...  At least you know what is going on. 


Darrell Reading Systems Engineer
Phone 479.204.5739
[EMAIL PROTECTED]

Wal-Mart Stores, Inc.
805 Moberly Lane, MS-0560-68
Bentonville, AR 72716
Save Money. Live Better

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Kaiser, Norm E CIV USAF AFMC
96 CS/SCCE
Sent: Wednesday, December 03, 2008 15:14
To: arslist@ARSLIST.ORG
Subject: Re: UPDATE: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Well, it looks like I found it.  Microsoft SQL handles lock escalation
dynamically.

The escalation can, well, escalate to a table lock.  Here's Microsoft
article on the issue:

http://support.microsoft.com/kb/323630

I ran the Profiler, and sure enough, that's exactly what's happening!

The fix? Write better, smaller queries.

Damn.

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Darrell Reading
Sent: Wednesday, December 03, 2008 2:30 PM
To: arslist@ARSLIST.ORG
Subject: Re: UPDATE: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Is there something going on with that table that locks it when someone
searches?  Maybe it is as simple as changing the lockmode from table to
row? 


Darrell Reading Systems Engineer
Phone 479.204.5739
[EMAIL PROTECTED]

Wal-Mart Stores, Inc.
805 Moberly Lane, MS-0560-68
Bentonville, AR 72716
Save Money. Live Better

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Kaiser, Norm E CIV USAF AFMC
96 CS/SCCE
Sent: Wednesday, December 03, 2008 14:18
To: arslist@ARSLIST.ORG
Subject: UPDATE: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

*** UPDATE ***

Well, it appears I don't have to toilet paper my own house after all.

As I reported below, I attempted to fix my problem with spotty,
intermittent performance by increasing my fast and list threads from 5
to 30.  That did nothing.  I still have the same problem.

The issue is most definitely related to diary searches, but I would
*expect* that a diary search would not so drastically impact ALL users
as it does.  I can understand the person who invoked the diary search
having a problem, but considering there are so many other threads
available and CPU utilization remains under 10%, I would think other
users would not be impacted so dramatically.

But it never fails...I can reproduce the problem 100% of the time.  I
kick off a diary search, and everyone's client stops responding.

It's almost like (I know this is dreaded and sometimes over-reported),
but it's almost like a memory leak in the server app.  Restarting the
service, naturally, rectifies the situation and Remedy just hums along
until someone else does a diary search.

Now, I understand I can block diary searches.  But my issue is wondering
why diary searches impact EVERYONE.

Ideas?

Norm



-----Original Message-----
From: Kaiser, Norm E CIV USAF AFMC 96 CS/SCCE 
Sent: Tuesday, November 25, 2008 5:39 PM
To: 'arslist@ARSLIST.ORG'
Subject: RE: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Well, a little bird (ahem...cough...Doug...cough) suggested I double up
my list and fast threads, which I've done, and that seems -- at least on
the surface -- to have corrected the problem.

I did have multiple threads, but I guess just not enough.

If that was the problem, I'm going to toilet paper my own house.

Norm

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Mayfield, Andy L.
Sent: Tuesday, November 25, 2008 5:02 PM
To: arslist@ARSLIST.ORG
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Never mind......  I guess I should read the entire thread before
responding. 

Andy L. Mayfield 
Sr. System Operation Specialist 
Alabama Power Company 
Office: 205-226-1805 
Cell: 205-288-9140 
SoLinc: 10*19140 


-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Mayfield, Andy L.
Sent: Tuesday, November 25, 2008 4:17 PM
To: arslist@ARSLIST.ORG
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

We had a similar problem recently. We found that an Active Link was
causing our problem. It was a newly created Active Link that was somehow
corrupted and caused the server to hang.

It might be worth a look. Check to see if any objects have been created
or modified recently.  

Good Luck. 

Andy L. Mayfield 
Sr. System Operation Specialist 
Alabama Power Company 
Office: 205-226-1805 
Cell: 205-288-9140 
SoLinc: 10*19140 


-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Michaud, Christopher W Mr CTR
USA MEDCOM USAMITC
Sent: Tuesday, November 25, 2008 9:27 AM
To: arslist@ARSLIST.ORG
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Classification:  UNCLASSIFIED 
Caveats: NONE

Norm,

You may want to investigate whether you can use BMC or SQL Full Text
search options to improve the performance. Alternatively, I've found it
helps to interview the culprits to understand how they are utilizing the
system to do their job. Often you can add an indexed field that allows
them to categorize/track what they are looking for on a repeat basis.

Christopher Michaud



-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Kaiser, Norm E CIV USAF AFMC
96 CS/SCCE
Sent: Tuesday, November 25, 2008 8:25 AM
To: arslist@ARSLIST.ORG
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Good suggestion...I'm pretty familiar with the new worklog model in
version 7 and its advantages and disadvantages.  Unfortunately, that
entails a very large coding effort, which I'm not able to do on this
system.

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Benedetto Cantatore
Sent: Tuesday, November 25, 2008 8:12 AM
To: arslist@ARSLIST.ORG
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

** 
Norm,
 
Perhaps you need to steal an idea from version 7 and make the worklogs a
parent-child relationship with the main form.  This would accomodate the
individuals that need to get to specific information in the worklog and
ease up the burden on your database.  If you can install version 7 on a
server, you'll see how it works and adopt it.  
 
Ben Cantatore
Remedy Manager
(914) 457-6209
 
Emerging Health IT
3 Odell Plaza
Yonkers, New York 10701


>>> [EMAIL PROTECTED] 11/25/08 8:56 AM >>>

Yeah, I suspected the same thing going in, but free disk space is
abundant.  Only about 20% of the disk is used.

I have concluded that the issue is the diary searches.  I suspected that
this was a problem about a month ago, so I created a form and a filter
that would capture a record every time a user did a diary search.  Sure
enough, I discovered users were doing diary searches dozens of times per
day.

There are now over 500,000 tickets in this system, and each ticket
contains diary entries of up to 30 pages (or more) in length.  Users
were repeatedly searching for things like, "The ticket was placed on
hold because the customer is unavailable."

To prove the theory, I had the administrator at the site repeatedly log
on to her User client.  That is,
TOOL...LOGIN...TOOLS...LOGIN...TOOLS...LOGIN...etc.  The User client
would faithfully log her on to Remedy in under a second.  I told her,
"Keep doing it!" while I went to my client and issued a diary search.
Bam! She could no longer log in.  She got the dreaded, "Setting server
port..." message that never went away.

So I have locked down the diary field to prevent these searches, but I'm
already hearing all sorts of dissent: "That puts us out of business! We
HAVE to be able to search the worklog!"

So now I'm considering other options.  I suppose the only thing I can do
is set up some type of archival system, but that comes with two
problems: 1) Users will hate it and 2) It doesn't really solve the
problem.  Putting a voluminous amount of free text on another form and
telling users, "Go search there," still puts a huge burden on the
database to sift through all that garbage.

Norm

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Joe DeSouza
Sent: Monday, November 24, 2008 8:09 PM
To: arslist@ARSLIST.ORG
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

** 
Another thing could be your disk space getting full on the Remedy
server. We had that issue recently when one of the operation some user
would do would eventually timeout and would create a temp file on the
servers Windows Temp directory that would grow and keep growing even if
the user quit the user tool from the client. The disk would eventually
be full and the AR Server would get extremely slow and eventually
impossible to login.

Bouoncing the Remedy Service would kill that temp file and release all
the used space..

Joe


________________________________

From: "Kaiser, Norm E CIV USAF AFMC 96 CS/SCCE"
<[EMAIL PROTECTED]>
To: arslist@ARSLIST.ORG
Sent: Monday, November 24, 2008 12:58:53 PM
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Yes, that's my suspicion.  I have a big suspicion that people are
searching the worklog diary field.

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Michaud, Christopher W Mr
CTRUSA MEDCOM USAMITC
Sent: Monday, November 24, 2008 11:20 AM
To: arslist@ARSLIST.ORG
Subject: Re: Intermittent, Spotty ARS Performance (UNCLASSIFIED)

Classification:  UNCLASSIFIED 
Caveats: NONE

Norm,

You may want to look closer at the SQL side. Look for locks. Perhaps
someone querying a diary or un-indexed field. Also, are you using SQL
replication? In particular, are snapshots turned on?


Christopher Michaud

-----Original Message-----
From: Action Request System discussion list(ARSList)
[mailto:[EMAIL PROTECTED] On Behalf Of Kaiser, Norm E CIV USAF AFMC96
CS/SCCE
Sent: Monday, November 24, 2008 11:03 AM
To: arslist@ARSLIST.ORG
Subject: Intermittent, Spotty ARS Performance

** 

Hi everyone:

This problem has me perplexed.

At a site I support, the Remedy server inexplicably stops responding to
requests.  It's very intermittent.  It runs fine for awhile, then
seemingly without warning, it just hangs.  Users attempting to log on
get stuck at the "Setting server port" dialog, which eventually times
out.

Other users who are already logged who try to pull up a ticket get stuck
at a blank screen that never comes back.

To resolve the issue, they have to bounce the Remedy server service. The
system works for awhile...until it hangs up again.

Any ideas what might be causing this?

-          I have monitored CPU utilization when this occurs, and the
CPU hums along at about 3% - 5% utilization
-          Network utilization is flat-lined whenever this occurs (i.e.,
no spike)
-          Memory utilization appears normal
-          CNET bandwidth tests resolve to better than dedicated T1
performance (for what that's worth)

Any thoughts are greatly appreciated.

The interesting thing is, we have the same exact Remedy apps running on
the same exact type of server in the same exact environment in four
other locations, and those four other locations never experience any
problems.

Norm

Remedy ARS 6.3
Microsoft SQL 2000 SP4
Microsoft Windows 2000 SP2
100% Custom Apps - No ITSM

__Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"
html___

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

__Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"
html___ 

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"
Classification:  UNCLASSIFIED 
Caveats: NONE

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

-----------------------------------------
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the individual or entity to whom they are
addressed. If you have received this email in error destroy it
immediately.
**********************************************************************
Wal-Mart Confidential
**********************************************************************
 

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

________________________________________________________________________
_______
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

_______________________________________________________________________________
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"

Reply via email to