Wow. Great post!!!
-----Original Message----- From: Ben Scott [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 27, 2008 9:32 AM To: MS-Exchange Admin Issues Subject: Re: memory upgrade on EX2K3 On Fri, May 23, 2008 at 12:44 PM, Thomas Gonzalez <[EMAIL PROTECTED]> wrote: > I'm upgrading our exchange 2003 memory from 2 gig to 4 gig this weekend. Here's my notes from when we upgraded to a new server, which had 4 GB RAM. At the time, we were running Windows 2000 Server Standard and Exchange 2000 Enterprise. As I recall, things weren't any different for Exchange 2003. Server 2003 supports the /4GB switch even for Standard, so that may make a difference. Summary ------- New server has: - 4 GB of RAM - Windows 2000 Server Standard - Exchange 2000 Enterprise - Active Directory Domain Controller and Global Catalog - Several other applications Should we use the /3GB switch in the BOOT.INI file on server? Short answer: No Quick Background ---------------- i386 = 32-bit 32-bit = 4 GB By default, Windows on i386 splits the virtual address space into 2 GB for userland and 2 GB for the kernel. The /3GB switch changes this to give 3 GB to userland and 1 GB to the kernel. This means processes can have a larger virtual address space, at the cost of cutting kernel virtual address space in half. Reducing the kernel address space is not without penalties. Quick Explanation ----------------- 1. On a system doing many different things, there is little to no benefit in depriving the kernel of address space it could use for caching in RAM. 1a. Exchange generally only benefits from the extra address space on a dedicated Exchange box. Less kernel memory is needed since the only thing the box is doing is running Exchange, not managing many different things. 2. The /3GB switch is not supported on Win 2000 Srvr Std. It is supported on Win 2003 Srvr and Win 2000 Srvr Enterprise, but we don't have those. 2a. It isn't worth upgrading to Win 2000 Srvr Ent because of Point 1, above. Notes ----- 4GT = 4 gigabyte tuning. The /3GB switch turns this on. Virtual address space (the memory a process can address) is not the same as the physical address space (the amount of RAM the machine can address). With /3GB enabled, a program still has to request the larger memory space, or it will still be limited to 2 GB. This is a backwards compatabiltiy hack to let programs which assume a 2 GB userland keep working. This has nothing to do with PAE (physical address extension). PAE changes the 32-bit physical address space to a 36-bit physical address space. It lets the machine address more RAM. The amount of RAM the machine can address has nothing to do with the address space of a running process. This has nothing to do with page files (swap files), which are sometimes called "Virtual Memory". That kind of virtual memory has nothing to do with virtual addres space (which is also sometimes called "virtual memory"). Detailed Analysis ----------------- REF1 states that "If you're working with Exchange Server and another application that doesn't know about large memory spaces, that other program will not be able to use any of the additional memory provided by /3GB." In particular, Exchange plus an AD DC means one should not use /3GB switch. REF2 enforces this, in particular pointing out things like antivirus software and database servers, which also tend to be memory hungry. REF2 also provides some very useful looking pointers to info on how to tune Windows and Exchange for more balanced memory operation. REF3 further enforces the kernel/userland tradeoffs, noting that "It is possible to run out of kernel memory well before running out of user memory, or vice versa." MSKB 315407 provides a registry hack that tweaks the OS memory manager in a way that helps reduced memory fragmentation with large RAMs. MSKB 266768 tells how to monitor the Exchange IS to make sure memory allocations are within safe boundries (both with and without /3GB), and how to limit the Store Database cache size to make sure it stays within safe boundries. MSKB 328882 provides similar tips. It also explains out why memory limit tuning is needed in our case: Exchange calculates certain allocation sizes based on physical RAM, not address space. In REF5, in the comments, Larry Osterman states, in regards to Exchange and memory fragmentation, "it starts showing up with several (3-5) thousand users". That gives us an idea of the scale we're talking about. MSKB 325044 and 317411 contain tons of detailed information on investigation of memory related issues, but little in the way of preventive advice. MSKB 313707 detail memory-related problems that can occur with Exchange, but do not apply to us for one reason or another. Most commonly, because we cannot use the /3GB switch on Win 2000 Srvr Std. However, these do serve to highlight that the /3GB switch is not a free lunch. MSKB 291988 notes that /3GB on Win 2000 Srvr Std doesn't really work, even if it looks like it is working. MSKB 266096 implies that the /3GB swith is required, period. Other sources, including MSKB articles, and REF4.1, contradict this. MSKB 266096 also states that it applies only to Windows versions which suppor the /3GB switch properly. The conclusion is that the /3GB switch is not required; it merely lets Exchange take full advantage of larger RAM for the Store. MSKB 306860 documents issues we may encounter. MSKB 302254, 313084, 319682, 324118 document issues which are supposedly fixed in the versions we are running. MSKB 311901, 268363, 171793 document issues we should not encounter, since we are not using PAE, nor do we have more than 4 GB of physical RAM. REF4 explains what the /3GB does in rich detail. Useful mainly as general background, since we cannot use the /3GB switch. But it at least gives us an idea of what we are missing -- and what we are *NOT* missing, too. MSKB Articles ------------- 171793 - Information on Application Use of 4GT RAM Tuning 266096 - Exchange 2000 requires /3GB switch with more than 1 gigabyte of physical RAM 266768 - How to modify the Store Database maximum cache size in Exchange 2000 Server 268363 - Intel Physical Addressing Extensions (PAE) in Windows 2000 291988 - A description of the 4 GB RAM Tuning feature and the Physical Address Extension switch 298064 - Scalability planning for Exchange 2000 Server 302254 - Computer That Is Running Exchange 2000 and Windows 2000 Server May Run Out of Virtual Memory with Event ID 12800 306860 - Incorrect Memory Status When Monitoring Status for Available Virtual Memory 311901 - The effects of 4GT tuning on system Page Table Entries 313084 - The Memory Status Is Incorrect When You Monitor the Status for Available Virtual Memory 313707 - Exchange 2000 Server with the "/3GB" Switch in the Boot.ini File May Lose Network Connectivity Under a Heavy Messaging Load 315407 - The "HeapDecommitFreeBlockThreshold" registry key 317411 - How to gather data to troubleshoot Exchange virtual memory issues 319682 - The Exchange 2000 Information Store Reports an Event ID 327 Warning Message and Virtual Memory May Be Fragmented 324118 - The Extensible Storage Engine Database Engine Contributes to Virtual Memory Fragmentation 325044 - How to troubleshoot virtual memory fragmentation in Exchange 2003 and 2000 328882 - Exchange memory use and the /3GB switch Other References ---------------- REF1 "/3GB switch 'gotchas' for Exchange Server" By Serdar Yegulalp Published by SearchExchange TechTarget Published on 20 June 2006 http://searchexchange.techtarget.com/tip/0,289483,sid43_gci1194970,00.ht ml REF2 "Exchange Memory Allocation" By Amit Zinman Published on 12 Oct 2004 http://www.msexchange.org/articles/Exchange-Memory-Allocation.html REF3 "Microsoft Windows Kernel Memory Management and Microsoft Exchange Server" Published 7 Dec 2005 http://msexchangeteam.com/archive/2005/12/07/415733.aspx REF4 "Summary of the recent spate of /3GB articles" http://blogs.msdn.com/oldnewthing/archive/2004/08/22/218527.aspx REF4.1 "Why does Exchange recommend /3GB if you have more than 1GB of physical memory?" http://blogs.msdn.com/oldnewthing/archive/2004/08/13/214117.aspx REF5 "Execution protection (NX) and PAE" http://blogs.msdn.com/carmencr/archive/2004/08/06/210093.aspx ---------------------------------------------------------------------- MSKB 266768 "How to modify the Store Database maximum cache size in Exchange 2000 Server" Stopped all Exchange services MSKB article provides instructions on how to open the config container. Select "ADSI Edit" root object >From the "Action" menu, inoke "Connect to" In the "Connection Point" section, set "Naming Context" to "Configuration Container" Navigated to: LDAP://server.inside.example.com/CN=InformationStore,CN=SERVER,CN=Server s,CN=First Administrative Group,CN=Administrative Groups,CN=Org Name,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=inside,DC=example,DC=com Brought up properties of that folder (from left pane) Selected attribute "msExchESEParamCacheSizeMax" Value was not set Set to 147456 per KB article Per the article: 147456 is 576 MB, which is the "recommended size on servers without the /3GB switch set". The value is in pages (4 KB each). -------------------- MSKB 315407 "The "HeapDecommitFreeBlockThreshold" registry key" All Exchange services are still stopped Used REGEDIT Navigated to Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager Value Name: HeapDeCommitFreeBlockThreshold Value Data was zero (0) Changed to 0x00040000 (262144) per KB article Restarted server. ---------------------------------------------------------------------- ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~ ~ http://www.sunbeltsoftware.com/Ninja ~ ~ Ninja Email Security with Cloudmark Spam Engine Gets Image Spam ~ ~ http://www.sunbeltsoftware.com/Ninja ~
