Re: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-13 Thread Jeremy Young
Question about perfmon stats.  I've read that I/O stats should be 
compared against the number of spindles in your logical disk (LUN on a 
Clariion CX400) and that avg disk queue should not exceed the number of 
spindles.  So, for example, an avg queue of 4 in a 8 disk RAID5 would be 
acceptable.  Is this correct or should the number remain below a certain 
threshold regardless of spindle count?


- Jer

Steven Peck wrote:

Disk IO.  Check your disk IO.  If it is above 50 ms then you are toast
and you will have end users get the 'waiting on exchange server'
pop-ups.  Exchange Best Practices analyzer can help you spot this.

We have this issue currently using a Hitachi SAN.  A major in house
project went live recently and it is also on the SAN.  Performance on
Exchange went to hell.

There are two points this can happen.
1.  You are sharing spindles on the SAN with other applications... (no
dedicated storage on a given LUN - common).  The load on the other
application increases, you are doomed.
2.  You have a shared fabric to get to the SAN and your ports are over
subscribed or your Exchange server is sharing the same paths as your
other really busy IO intensive applications.

We have both issues.  We have dedicated storage now, but the path to
the dedicated storage is shared on the McData unit so no real help
yet.

Steven Peck



On Feb 11, 2008 11:23 AM, Robb Pickinpaugh
[EMAIL PROTECTED] wrote:
  

Sean,

I'll check the boot.ini switches hadn't thought of that yet.

As for using Diskpart, yes I partitioned those disks following my initial setup 
SAs instructions very carefully.

As for the dedicated spindles... we didn't have any way to determine the 
damage that was going to be done by the other app at the time.

Now that I'm seeing the trend I'm running the ExBPA Health Check, with the 2 
hour perfomance monitor, to see what comes back from that.

I'll also be getting in touch with EMC support to find out how to get info out 
of the array itself as far as proving the overload.

Then... and only then, can I try to get money for dedicated spindles.

The admins of the other system have shut off the particular function that was 
causing the disk smash (for the moment).

Still trying to make sure that Exchange in and of itself is configured 
correctly, and that there are no inherent performance problems besides the 
disks I have the store sitting on.

Thanks for the additional info.

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~




~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~
  


~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


Re: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-12 Thread Steven Peck
Well, I should have added that caveat I suppose.  In our case, we were
getting spikes of 4 seconds.  On the plus side, Exchange wasn't
corrupting the database, just freezing everything.

Spikes will still cause intermittent pop-ups unless you have clients
in cached mode to mitigate this.

Steven Peck

On Feb 11, 2008 3:03 PM, Andy David [EMAIL PROTECTED] wrote:
 Only if it's sustained above 50ms is it an issue.
 Spikes are ok.


 -Original Message-
 From: Steven Peck [mailto:[EMAIL PROTECTED]
 Sent: Monday, February 11, 2008 5:25 PM
 To: MS-Exchange Admin Issues

 Subject: Re: Sluggish performance of Mail Store, Server sitting mostly idle

 Disk IO.  Check your disk IO.  If it is above 50 ms then you are toast
 and you will have end users get the 'waiting on exchange server'
 pop-ups.  Exchange Best Practices analyzer can help you spot this.

 We have this issue currently using a Hitachi SAN.  A major in house
 project went live recently and it is also on the SAN.  Performance on
 Exchange went to hell.

 There are two points this can happen.
 1.  You are sharing spindles on the SAN with other applications... (no
 dedicated storage on a given LUN - common).  The load on the other
 application increases, you are doomed.
 2.  You have a shared fabric to get to the SAN and your ports are over
 subscribed or your Exchange server is sharing the same paths as your
 other really busy IO intensive applications.

 We have both issues.  We have dedicated storage now, but the path to
 the dedicated storage is shared on the McData unit so no real help
 yet.

 Steven Peck



 On Feb 11, 2008 11:23 AM, Robb Pickinpaugh
 [EMAIL PROTECTED] wrote:
  Sean,
 
  I'll check the boot.ini switches hadn't thought of that yet.
 
  As for using Diskpart, yes I partitioned those disks following my initial 
  setup SAs instructions very carefully.
 
  As for the dedicated spindles... we didn't have any way to determine the 
  damage that was going to be done by the other app at the time.
 
  Now that I'm seeing the trend I'm running the ExBPA Health Check, with the 
  2 hour perfomance monitor, to see what comes back from that.
 
  I'll also be getting in touch with EMC support to find out how to get info 
  out of the array itself as far as proving the overload.
 
  Then... and only then, can I try to get money for dedicated spindles.
 
  The admins of the other system have shut off the particular function that 
  was causing the disk smash (for the moment).
 
  Still trying to make sure that Exchange in and of itself is configured 
  correctly, and that there are no inherent performance problems besides the 
  disks I have the store sitting on.
 
  Thanks for the additional info.
 
  ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
  ~ http://www.sunbeltsoftware.com/Ninja~
 

 ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
 ~ http://www.sunbeltsoftware.com/Ninja~

 ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
 ~ http://www.sunbeltsoftware.com/Ninja~


~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-12 Thread Robb Pickinpaugh
The switch fabric is running at 4Gbps.

I didn't check the switch utilities for usage during peak times, I was 
seriously just looking at the server.

If it starts up again I'll be looking there as well.

We have a complete redundant path to the SPs on the Clariion for each host, and 
we are not load balancing (perhaps that would help too, but we're not doing it 
right now).

Ran the BPA, got some interesting errors and warnings that I need to work 
through.

Just looked at the ms time with a Full backup running with Page/sec and Avg 
Queue Length way up, the Avg. Disk/sec Transfer was at 4ms, that never reached 
50ms.

However that was NOT with the other system trashing the disk.

I'm going to work through the BPA recommendations, and try to get with EMC and 
see what I can do about proving the need for more disk due to overload.

You guys have been a fantastic help with this, I really appreciate the extra 
effort, and the fantastic resource this list has proven to be.

Robb
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


Re: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Steven Peck
Disk IO.  Check your disk IO.  If it is above 50 ms then you are toast
and you will have end users get the 'waiting on exchange server'
pop-ups.  Exchange Best Practices analyzer can help you spot this.

We have this issue currently using a Hitachi SAN.  A major in house
project went live recently and it is also on the SAN.  Performance on
Exchange went to hell.

There are two points this can happen.
1.  You are sharing spindles on the SAN with other applications... (no
dedicated storage on a given LUN - common).  The load on the other
application increases, you are doomed.
2.  You have a shared fabric to get to the SAN and your ports are over
subscribed or your Exchange server is sharing the same paths as your
other really busy IO intensive applications.

We have both issues.  We have dedicated storage now, but the path to
the dedicated storage is shared on the McData unit so no real help
yet.

Steven Peck



On Feb 11, 2008 11:23 AM, Robb Pickinpaugh
[EMAIL PROTECTED] wrote:
 Sean,

 I'll check the boot.ini switches hadn't thought of that yet.

 As for using Diskpart, yes I partitioned those disks following my initial 
 setup SAs instructions very carefully.

 As for the dedicated spindles... we didn't have any way to determine the 
 damage that was going to be done by the other app at the time.

 Now that I'm seeing the trend I'm running the ExBPA Health Check, with the 2 
 hour perfomance monitor, to see what comes back from that.

 I'll also be getting in touch with EMC support to find out how to get info 
 out of the array itself as far as proving the overload.

 Then... and only then, can I try to get money for dedicated spindles.

 The admins of the other system have shut off the particular function that was 
 causing the disk smash (for the moment).

 Still trying to make sure that Exchange in and of itself is configured 
 correctly, and that there are no inherent performance problems besides the 
 disks I have the store sitting on.

 Thanks for the additional info.

 ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
 ~ http://www.sunbeltsoftware.com/Ninja~


~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


Re: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Robb Pickinpaugh
Sean,

I'll check the boot.ini switches hadn't thought of that yet.

As for using Diskpart, yes I partitioned those disks following my initial setup 
SAs instructions very carefully.

As for the dedicated spindles... we didn't have any way to determine the 
damage that was going to be done by the other app at the time.

Now that I'm seeing the trend I'm running the ExBPA Health Check, with the 2 
hour perfomance monitor, to see what comes back from that.

I'll also be getting in touch with EMC support to find out how to get info out 
of the array itself as far as proving the overload.  

Then... and only then, can I try to get money for dedicated spindles.

The admins of the other system have shut off the particular function that was 
causing the disk smash (for the moment).

Still trying to make sure that Exchange in and of itself is configured 
correctly, and that there are no inherent performance problems besides the 
disks I have the store sitting on.

Thanks for the additional info.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


Re: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Sean Martin
Yuckyou definitely want to dedicate spindles to your Exchange server if
at all possible. Sounds like you may have identified the performance
problem.

We're running our Exchange server off a similiar setup as yours (CX700,
switched fabric, etc.)  Did you partition your Exchange LUNs using Diskpart?
(create partition primary align=64). What switches are present in your
boot.ini?  The following have worked well for our server which matches yours
almost identically hardware wise: /3GB /USERVA=3030 /MAXMEM=4096.

- Sean


On 2/11/08, Robb Pickinpaugh [EMAIL PROTECTED] wrote:

 Ok, I've got a lot to do and sort through.

 Also the comments about the SAN volumes causing my issue... they may be
 closer than I initially thought.

 One of our other production systems is sharing the disks that my Exchange
 LUN is living on... that users on that system have been running processes
 that are pounding the system pretty hard, including the disk, and this all
 started last Friday.

 Coincidence??? probably not.

 I'll keep digging, now at least I have directions to dig that may give me
 good answers.

 Thanks a bunch.
 ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
 ~ http://www.sunbeltsoftware.com/Ninja~


~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Robb Pickinpaugh
A little deeper and the Queue length is all in the disk the store is on...

Hm...

I think I want to run a longer snapshot to look at this.

Say a couple of hours.

Any suggestions about how to correct the Disk Queue time, or is that a hardware 
/ vendor specific type question?

Haven't ever dug this deep into performance tweaking before.

The sharing of your experience is greatly appreciated.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Michael B. Smith
Like the other guys said - first thing I'd take a look at is the network.

But the second thing - start perfmon on that Exchange server. Is the disk queue 
averaging over 1? Is paging averaging over 20? Is the processor averaging over 
70%?

Any of these could be causing the issue as well...

Regards,

Michael B. Smith
MCSE/Exchange MVP
http://TheEssentialExchange.com


-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: Monday, February 11, 2008 9:03 AM
To: MS-Exchange Admin Issues
Subject: Sluggish performance of Mail Store, Server sitting mostly idle

I have a really odd behavior pattern happening here and I'm sort of at a loss 
as to how to fix it.

All of the sudden starting toward the end of last week (Thursday or Friday) the 
mail store seems to be acting funny.

Outlook connections all over the building are popping the Trying to retrieve 
information message.

To the point where just changing folders in the mailbox can take up to 30 
seconds or so.

This morning I tried to delete about 1800 messages from a folder in my mailbox 
and it's taken about 30 minutes and still not done, if the MS status bar is 
right it will have taken about 40 minutes to delete those messages, and thats 
just moving them to the deleted items bin, not a shift-delete.

Pertinent data:
Exchange 2003 SP2

Windows 2003 SP1
2 Dual Core 2.8Ghz Processors
4GB Ram

Mailbox  Public folder store is on a 4Gb Fiber Channel SAN
Priv1.mdb = 130GB
Priv1.stm = 51GB

We shutdown the system for maintenance on the SAN this weekend, and that 
apparently has NOT solved the problem.

I need some advice as to where the look for the issue.

I've toyed with the idea of setting up a second store and migrating users to 
the new store, but I'm not sure if that will help.

We have a possible archiving project in the future as well, but that's not 
going to help right now.

I have mailboxes as large as 9.5 GB and as small as POP3 that don't leave 
anything on the server.

To answer the inevitable question.. no the big mailbox users never delete 
anything (hence the archiving project).


Suggestions, ideas?

TIA
Robb
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Robb Pickinpaugh
AV = Symantec... I'll check for updates there.

The switch and server are set to auto-negotiate, and they're both capable of 
1Gbps.  I'll look at specifying the 1Gb Full on both sides of that link and see 
what happens.

Thanks for the ideas so far.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Eric Wittersheim
Trend as your AV perhaps?  This happened when we brought a new server online.  
I don't recall the exact details but an update from Trend fixed the problem.

-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: Monday, February 11, 2008 8:03 AM
To: MS-Exchange Admin Issues
Subject: Sluggish performance of Mail Store, Server sitting mostly idle

I have a really odd behavior pattern happening here and I'm sort of at a loss 
as to how to fix it.

All of the sudden starting toward the end of last week (Thursday or Friday) the 
mail store seems to be acting funny.

Outlook connections all over the building are popping the Trying to retrieve 
information message.

To the point where just changing folders in the mailbox can take up to 30 
seconds or so.

This morning I tried to delete about 1800 messages from a folder in my mailbox 
and it's taken about 30 minutes and still not done, if the MS status bar is 
right it will have taken about 40 minutes to delete those messages, and thats 
just moving them to the deleted items bin, not a shift-delete.

Pertinent data:
Exchange 2003 SP2

Windows 2003 SP1
2 Dual Core 2.8Ghz Processors
4GB Ram

Mailbox  Public folder store is on a 4Gb Fiber Channel SAN
Priv1.mdb = 130GB
Priv1.stm = 51GB

We shutdown the system for maintenance on the SAN this weekend, and that 
apparently has NOT solved the problem.

I need some advice as to where the look for the issue.

I've toyed with the idea of setting up a second store and migrating users to 
the new store, but I'm not sure if that will help.

We have a possible archiving project in the future as well, but that's not 
going to help right now.

I have mailboxes as large as 9.5 GB and as small as POP3 that don't leave 
anything on the server.

To answer the inevitable question.. no the big mailbox users never delete 
anything (hence the archiving project).


Suggestions, ideas?

TIA
Robb
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Davies,Matt
Sounds like network problems, either on the clients or the server.

Check for errors on the switch and make sure both the switch and server are not 
set to auto negotiate.

Cheers

Matt



-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: 11 February 2008 14:21
To: MS-Exchange Admin Issues
Subject: re: Sluggish performance of Mail Store, Server sitting mostly idle

Also another tibit.

The server is sitting at 2.45GB of RAM in use, and the processors never exceed 
10% usage.

Hence the confusion, I would understand the behavior if the server was 
overloaded, but it's practically doing nothing.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~



_
This e-mail (including all attachments) is confidential and may be privileged. 
It is for the exclusive use of the addressee only. If you are not the 
addressee, 
you are hereby notified that any dissemination of this communication is 
strictly 
prohibited. If you have received this communication in error, please erase all 
copies of the message and its attachments and notify us immediately at 
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]. Thank You.

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Maglinger, Paul
Are you running any kind of anti-virus on the Exchange server?  Did this
start happening after an update? 

-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: Monday, February 11, 2008 8:03 AM
To: MS-Exchange Admin Issues
Subject: Sluggish performance of Mail Store, Server sitting mostly idle

I have a really odd behavior pattern happening here and I'm sort of at a
loss as to how to fix it.

All of the sudden starting toward the end of last week (Thursday or
Friday) the mail store seems to be acting funny.

Outlook connections all over the building are popping the Trying to
retrieve information message.

To the point where just changing folders in the mailbox can take up to
30 seconds or so.

This morning I tried to delete about 1800 messages from a folder in my
mailbox and it's taken about 30 minutes and still not done, if the MS
status bar is right it will have taken about 40 minutes to delete those
messages, and thats just moving them to the deleted items bin, not a
shift-delete.

Pertinent data:
Exchange 2003 SP2

Windows 2003 SP1
2 Dual Core 2.8Ghz Processors
4GB Ram

Mailbox  Public folder store is on a 4Gb Fiber Channel SAN
Priv1.mdb = 130GB
Priv1.stm = 51GB

We shutdown the system for maintenance on the SAN this weekend, and that
apparently has NOT solved the problem.

I need some advice as to where the look for the issue.

I've toyed with the idea of setting up a second store and migrating
users to the new store, but I'm not sure if that will help.

We have a possible archiving project in the future as well, but that's
not going to help right now.

I have mailboxes as large as 9.5 GB and as small as POP3 that don't
leave anything on the server.

To answer the inevitable question.. no the big mailbox users never
delete anything (hence the archiving project).


Suggestions, ideas?

TIA
Robb
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


re: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Robb Pickinpaugh
Also another tibit.

The server is sitting at 2.45GB of RAM in use, and the processors never exceed 
10% usage.

Hence the confusion, I would understand the behavior if the server was 
overloaded, but it's practically doing nothing.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Davies,Matt
Robb,

I hope your San isn't the bottleneck, but then again it is a EMC Clariion, so 
you never know. 

The disk counters in performance would be able to give you a better picture.

Cheers

Matt


-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: 11 February 2008 15:20
To: MS-Exchange Admin Issues
Subject: RE: Sluggish performance of Mail Store, Server sitting mostly idle

Matt,

FC San.

EMC Clariion CX3-20C
4Gb switched infrastructure
4Gb 146GB 15k Disks
RAID 10

The disk should by no means be the bottleneck.

I'll check out the suggested link and tools.

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~



_
This e-mail (including all attachments) is confidential and may be privileged. 
It is for the exclusive use of the addressee only. If you are not the 
addressee, 
you are hereby notified that any dissemination of this communication is 
strictly 
prohibited. If you have received this communication in error, please erase all 
copies of the message and its attachments and notify us immediately at 
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]. Thank You.

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Robb Pickinpaugh
Mine's a 2850 and has Intel NICs... figures it wouldn't be THAT easy LOL.

I've been working with newer servers lately that's why my jump to Broadcoms was 
so quick...

Thanks for the info, and I'll watch for that when I apply SP2 to the server for 
sure.

I'm still going to look for updated drivers, but this is obviously not going to 
be that simple.

It just figure.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Davies,Matt
Robb,

Either look at the following document

http://www.microsoft.com/downloads/details.aspx?FamilyId=8679F6BD-7FF0-41F5-BDD0-C09019409FC0displaylang=en

Which goes over exchange performance troubleshooting, and what to look for.

Or

Download the Microsoft exchange performance and troubleshooting tools, which 
will give you a good idea of what is wrong.


Is the storage Local, FC San or Iscsi San ?


Cheers

Matt




-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: 11 February 2008 14:57
To: MS-Exchange Admin Issues
Subject: RE: Sluggish performance of Mail Store, Server sitting mostly idle

A little deeper and the Queue length is all in the disk the store is on...

Hm...

I think I want to run a longer snapshot to look at this.

Say a couple of hours.

Any suggestions about how to correct the Disk Queue time, or is that a hardware 
/ vendor specific type question?

Haven't ever dug this deep into performance tweaking before.

The sharing of your experience is greatly appreciated.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~



_
This e-mail (including all attachments) is confidential and may be privileged. 
It is for the exclusive use of the addressee only. If you are not the 
addressee, 
you are hereby notified that any dissemination of this communication is 
strictly 
prohibited. If you have received this communication in error, please erase all 
copies of the message and its attachments and notify us immediately at 
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]. Thank You.

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Bob Fronk
I had the same thing happen a few months ago.  Turned out to be the IP Chimney 
/ RSS issue.

http://theessentialexchange.com/blogs/michael/archive/2008/01/03/more-issues-with-the-windows-server-2003-scalable-networking-pack.aspx

(Watch for wrappage in the link)

Bob Fronk



-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: Monday, February 11, 2008 9:03 AM
To: MS-Exchange Admin Issues
Subject: Sluggish performance of Mail Store, Server sitting mostly idle

I have a really odd behavior pattern happening here and I'm sort of at a loss 
as to how to fix it.

All of the sudden starting toward the end of last week (Thursday or Friday) the 
mail store seems to be acting funny.

Outlook connections all over the building are popping the Trying to retrieve 
information message.

To the point where just changing folders in the mailbox can take up to 30 
seconds or so.

This morning I tried to delete about 1800 messages from a folder in my mailbox 
and it's taken about 30 minutes and still not done, if the MS status bar is 
right it will have taken about 40 minutes to delete those messages, and thats 
just moving them to the deleted items bin, not a shift-delete.

Pertinent data:
Exchange 2003 SP2

Windows 2003 SP1
2 Dual Core 2.8Ghz Processors
4GB Ram

Mailbox  Public folder store is on a 4Gb Fiber Channel SAN
Priv1.mdb = 130GB
Priv1.stm = 51GB

We shutdown the system for maintenance on the SAN this weekend, and that 
apparently has NOT solved the problem.

I need some advice as to where the look for the issue.

I've toyed with the idea of setting up a second store and migrating users to 
the new store, but I'm not sure if that will help.

We have a possible archiving project in the future as well, but that's not 
going to help right now.

I have mailboxes as large as 9.5 GB and as small as POP3 that don't leave 
anything on the server.

To answer the inevitable question.. no the big mailbox users never delete 
anything (hence the archiving project).


Suggestions, ideas?

TIA
Robb
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~



This email and any attached files are confidential and intended solely for the 
intended recipient(s). If you are not the named recipient you should not read, 
distribute, copy or alter this email. Any views or opinions expressed in this 
email are those of the author and do not represent those of the Davis H. Elliot 
Company company. Warning: Although precautions have been taken to make sure no 
viruses are present in this email, the company cannot accept responsibility for 
any loss or damage that arise from the use of this email or attachments.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Robb Pickinpaugh
Michael,

Quick look at the permon (running real time) and I think you've picked up on 
something.

Avg Disk Queue is about 1.1
Pages/Sec avg has run as high as 35 and as low as 11.

Looks like I may need to do some performance tweaks perhaps?
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Campbell, Rob
Sounds like network.

Any changes to the adapter or switch port settings?

-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: Monday, February 11, 2008 8:21 AM
To: MS-Exchange Admin Issues
Subject: re: Sluggish performance of Mail Store, Server sitting mostly
idle

Also another tibit.

The server is sitting at 2.45GB of RAM in use, and the processors never
exceed 10% usage.

Hence the confusion, I would understand the behavior if the server was
overloaded, but it's practically doing nothing.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

**
 
Note: 
The information contained in this message may be privileged and confidential 
and 
protected from disclosure.  If the reader of this message is not the intended  
recipient, or an employee or agent responsible for delivering this message to  
the intended recipient, you are hereby notified that any dissemination,   
distribution or copying of this communication is strictly prohibited. If you  
have received this communication in error, please notify us immediately by  
replying to the message and deleting it from your computer. 
**

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Campbell, Rob
A disk queue depth of 1.1 doesn't seem to be nearly enough to cause the
kind of symptoms you're seeing, IMHO.



-Original Message-
From: Robb Pickinpaugh [mailto:[EMAIL PROTECTED] 
Sent: Monday, February 11, 2008 8:57 AM
To: MS-Exchange Admin Issues
Subject: RE: Sluggish performance of Mail Store, Server sitting mostly
idle

A little deeper and the Queue length is all in the disk the store is
on...

Hm...

I think I want to run a longer snapshot to look at this.

Say a couple of hours.

Any suggestions about how to correct the Disk Queue time, or is that a
hardware / vendor specific type question?

Haven't ever dug this deep into performance tweaking before.

The sharing of your experience is greatly appreciated.
~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

**
 
Note: 
The information contained in this message may be privileged and confidential 
and 
protected from disclosure.  If the reader of this message is not the intended  
recipient, or an employee or agent responsible for delivering this message to  
the intended recipient, you are hereby notified that any dissemination,   
distribution or copying of this communication is strictly prohibited. If you  
have received this communication in error, please notify us immediately by  
replying to the message and deleting it from your computer. 
**

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~


RE: Sluggish performance of Mail Store, Server sitting mostly idle

2008-02-11 Thread Andy David
Only if it's sustained above 50ms is it an issue.
Spikes are ok.


-Original Message-
From: Steven Peck [mailto:[EMAIL PROTECTED]
Sent: Monday, February 11, 2008 5:25 PM
To: MS-Exchange Admin Issues
Subject: Re: Sluggish performance of Mail Store, Server sitting mostly idle

Disk IO.  Check your disk IO.  If it is above 50 ms then you are toast
and you will have end users get the 'waiting on exchange server'
pop-ups.  Exchange Best Practices analyzer can help you spot this.

We have this issue currently using a Hitachi SAN.  A major in house
project went live recently and it is also on the SAN.  Performance on
Exchange went to hell.

There are two points this can happen.
1.  You are sharing spindles on the SAN with other applications... (no
dedicated storage on a given LUN - common).  The load on the other
application increases, you are doomed.
2.  You have a shared fabric to get to the SAN and your ports are over
subscribed or your Exchange server is sharing the same paths as your
other really busy IO intensive applications.

We have both issues.  We have dedicated storage now, but the path to
the dedicated storage is shared on the McData unit so no real help
yet.

Steven Peck



On Feb 11, 2008 11:23 AM, Robb Pickinpaugh
[EMAIL PROTECTED] wrote:
 Sean,

 I'll check the boot.ini switches hadn't thought of that yet.

 As for using Diskpart, yes I partitioned those disks following my initial 
 setup SAs instructions very carefully.

 As for the dedicated spindles... we didn't have any way to determine the 
 damage that was going to be done by the other app at the time.

 Now that I'm seeing the trend I'm running the ExBPA Health Check, with the 2 
 hour perfomance monitor, to see what comes back from that.

 I'll also be getting in touch with EMC support to find out how to get info 
 out of the array itself as far as proving the overload.

 Then... and only then, can I try to get money for dedicated spindles.

 The admins of the other system have shut off the particular function that was 
 causing the disk smash (for the moment).

 Still trying to make sure that Exchange in and of itself is configured 
 correctly, and that there are no inherent performance problems besides the 
 disks I have the store sitting on.

 Thanks for the additional info.

 ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
 ~ http://www.sunbeltsoftware.com/Ninja~


~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~

~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~
~ http://www.sunbeltsoftware.com/Ninja~