RE: [ActiveDir] Raid 1 tangent -- Vendor Domain
I would say it was probably quite low relatively. Quite low is the norm for AD logs and by that it is usually barely registering compared to what you were doing the Log drive would have been hopping. I recall when you were IM'ing about it you mentioned the Log drive IOPS and I was like wow, I don't ever really expect to see those kind of numbers... -- O'Reilly Active Directory Third Edition - http://www.joeware.net/win/ad3e.htm -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman Sent: Monday, July 24, 2006 1:34 AM To: ActiveDir@mail.activedir.org Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. Actually, log IOs were quite low, considering. I bet a single spindle pair would have been enough for most of my work. The real killer was random I/O throughout the DB. Here I was pushing 1800 read / 1800 write for most of the run. I really needed more SAN paths because I'm pretty sure that was the bottleneck (it just wasn't set up to have as many redundant paths as I didn't anticipate the bottlenecks hit). I keep meaning to write a follow-up post with a lot of data. I'll do so this week and post it so this sort of stuff is a bit more clear. ~Eric -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Saturday, July 22, 2006 9:49 AM To: ActiveDir@mail.activedir.org Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than 40-50k. The smallest I tend to deal with is about 30k. I usually get called to walk in to Exchange issues where Exchange is underperforming or outright hanging, sometimes for hours at a time. There can be all sorts of issues causing this such as O poor disk subsystem design for Exchange (someone say got fancy with a SAN layout and really didn't know what they were doing seems to be popular here) O hardware/drivers on the Exchange server just
Re: [ActiveDir] Raid 1 tangent -- Vendor Domain
Just as an FYI: I've seen 64-bit DCs run and I have one thing that I can recommend to everyone:Go 64-bits as soon as possible. There are hundreds of benefits on the server side when going 64-bits, whether it's Exchange (yay for 2007) or your DCs, the performance level is just staggering compared to a 32-bit OS. All your former large application limitations just kinda disappear, unless it's an application-based limitation. No 3GB limitation on the application memory size, no paged pool memory limitation for connections (this hits Exchange first) It's like you're crippling your hardware by staying 32-bits nowadays if you don't have to. On 7/22/06, joe [EMAIL PROTECTED] wrote: That's a command line guy for you...:o)The thing is that I type in a very odd way two, my whole right hand just oneor two fingers from my left hand. People tend to get a bit confused whenthey see me type. joe--O'Reilly Active Directory Third Edition -http://www.joeware.net/win/ad3e.htm-Original Message-From: [EMAIL PROTECTED][mailto:[EMAIL PROTECTED]] On Behalf Of Kevin GentSent: Saturday, July 22, 2006 7:29 PMTo: ActiveDir@mail.activedir.orgSubject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domainjoe,you must type really, really fast- Original Message -From: Albert Duro [EMAIL PROTECTED]To: ActiveDir@mail.activedir.orgSent: Saturday, July 22, 2006 7:06 PMSubject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domain no debate from me.I was just asking.Thank you for the lesson. - Original Message - From: joe [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 9:48 AM Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than 40-50k. The smallest I tend to deal with is about 30k. I usually get called to walk in to Exchange issues where Exchange is underperforming or outright hanging, sometimes for hours at a time. There can be all sorts of issues causing this such as O poor disk subsystem design for Exchange (someone say got fancy with a SAN layout and really didn't know what they were doing seems to be popular here) O hardware/drivers
Re: [ActiveDir] Raid 1 tangent -- Vendor Domain
That being said wait on 64-bits for the client side until you know, unequivocably, that all of the software that your clients need is supported and stable on a 64-bit OS. The performance boost isn't that big of a deal, just to be honest. On 7/23/06, Matt Hargraves [EMAIL PROTECTED] wrote: Just as an FYI: I've seen 64-bit DCs run and I have one thing that I can recommend to everyone:Go 64-bits as soon as possible. There are hundreds of benefits on the server side when going 64-bits, whether it's Exchange (yay for 2007) or your DCs, the performance level is just staggering compared to a 32-bit OS. All your former large application limitations just kinda disappear, unless it's an application-based limitation. No 3GB limitation on the application memory size, no paged pool memory limitation for connections (this hits Exchange first) It's like you're crippling your hardware by staying 32-bits nowadays if you don't have to. On 7/22/06, joe [EMAIL PROTECTED] wrote: That's a command line guy for you...:o)The thing is that I type in a very odd way two, my whole right hand just oneor two fingers from my left hand. People tend to get a bit confused whenthey see me type. joe--O'Reilly Active Directory Third Edition - http://www.joeware.net/win/ad3e.htm-Original Message-From: [EMAIL PROTECTED][mailto: [EMAIL PROTECTED]] On Behalf Of Kevin GentSent: Saturday, July 22, 2006 7:29 PMTo: ActiveDir@mail.activedir.orgSubject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domainjoe,you must type really, really fast- Original Message -From: Albert Duro [EMAIL PROTECTED]To: ActiveDir@mail.activedir.orgSent: Saturday, July 22, 2006 7:06 PMSubject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domain no debate from me.I was just asking.Thank you for the lesson. - Original Message - From: joe [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 9:48 AM Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than 40-50k. The smallest I tend to deal with is about 30k. I usually get called to walk in to Exchange issues where Exchange
Re: [ActiveDir] Raid 1 tangent -- Vendor Domain
It's not that big of a deal for client software (last message)On 7/23/06, Matt Hargraves [EMAIL PROTECTED] wrote:That being said wait on 64-bits for the client side until you know, unequivocably, that all of the software that your clients need is supported and stable on a 64-bit OS. The performance boost isn't that big of a deal, just to be honest. On 7/23/06, Matt Hargraves [EMAIL PROTECTED] wrote: Just as an FYI: I've seen 64-bit DCs run and I have one thing that I can recommend to everyone:Go 64-bits as soon as possible. There are hundreds of benefits on the server side when going 64-bits, whether it's Exchange (yay for 2007) or your DCs, the performance level is just staggering compared to a 32-bit OS. All your former large application limitations just kinda disappear, unless it's an application-based limitation. No 3GB limitation on the application memory size, no paged pool memory limitation for connections (this hits Exchange first) It's like you're crippling your hardware by staying 32-bits nowadays if you don't have to. On 7/22/06, joe [EMAIL PROTECTED] wrote: That's a command line guy for you...:o)The thing is that I type in a very odd way two, my whole right hand just oneor two fingers from my left hand. People tend to get a bit confused whenthey see me type. joe--O'Reilly Active Directory Third Edition - http://www.joeware.net/win/ad3e.htm-Original Message-From: [EMAIL PROTECTED][mailto: [EMAIL PROTECTED]] On Behalf Of Kevin GentSent: Saturday, July 22, 2006 7:29 PMTo: ActiveDir@mail.activedir.orgSubject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domainjoe,you must type really, really fast- Original Message -From: Albert Duro [EMAIL PROTECTED] To: ActiveDir@mail.activedir.orgSent: Saturday, July 22, 2006 7:06 PMSubject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domain no debate from me.I was just asking.Thank you for the lesson. - Original Message - From: joe [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 9:48 AM Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than
RE: [ActiveDir] Raid 1 tangent -- Vendor Domain
I don't have a lot of experience yet with x64 DCs but my gut says that assuming you have enough RAM to cache the entire DIT and you aren't constantly rebooting the DC or doing things that force the cache to be trimmed, the disk subsystem is really only going to be important for writes (which we have already said aren't really all that much of what AD is doing) and the initial caching of the DIT. The key as joe pointed out nicely in his short note below is the size of your DIT and if it can be cached in memory by the DC. For large AD infrastructures, this is where the benefit of 64-bit DCs (either x64 or Itanium, with Itanium usually being an overkill for most AD environments). A 32-bit OS has 4GB virtual memory available that it can directly address usually split evenly between user/application memory and kernel memory, i.e. 2GB each. Win2000 DCs can use a max of approx. 512MB for the LSASS (AD) process, which is about how much of the DIT it can cache. LSASS has already been improved quite a bit for Win2003 DC, which can cache up to 1.5GB of the normal virtual address space available to the DC. Using the /3GB switch you can force the kernel to use less memory (1GB), so that you have up to 3GB available for your applications - note that apps that leverage kernel memory (such as IIS) are hurt by using this switch. For Win2000 DCs (Advanced Server only) you can use the /3GB switch to increase the DIT cache to 1GB. For Win2003 DCs (Standard and Enterprise) it's up to 2.7GB. The 32-bit x86 systems that use more than 4 GB of physical memory cannot directly address this memory - instead they leverge a technology called Physical Addressing Extensions (PAE). This is a segmented memory model that requires the use of Address Windowing Extensions (AWE) allowing the memory beyond 4 GB to be swapped in and out of an AWE window that exists in the first 4 GB of memory. These memory management technologies do cost expensive CPU cycles and are not nearly as efficient as direct 64-bit addressing. With 64-bit addressing there is no need for a /3GB switch or other memory extension techniques, as you can (theoretically) *directly* address up to 2^64 bits of memory, which is equal to 16 exa-bites (=16 billion GB). Naturally, there isn't any HW available yet to host this much memory. But we can soon expect standard server systems that are capable of handling a few hundred GBs of memory - nothing you should need anytime soon for your AD. Microsofts's support for physical memory for it's Win2003 64bit OSs is thus limited as well: - 32GB for the x64 Standard Edition - 1TB for the x64 and Itanium Enterprise and Datacenter editions So, even with a Win2003 x64 Standard Edition DC you can directly address up to 32GB of memory in your servers - the available physical memory will be split up evenly into user and kernel memory, meaning that with 32GB you'd have 16GB available for applications (and with a pure DC this would give you roughly 14GB for caching your AD DIT). Not many will need it for their ADs, but with the Enterprise or Datacenter editions you can cache approx. up to 460GB of your DIT in memory... We've done quite extensive internal tests at HP to evaluate how 64bit DCs would do performancewise as GCs (for Exchange and authentication) and found that a single 64bit DC (with sufficient RAM to cache our whole DIT, which is almost 9GB in size), could easily replace 3-4 of our current 32bit GCs. Disc configuration hardly played a role for these performance gains. Naturally we have the greatest ratio for consolidation in our largest datacenters. I can only suggest everyone to have a closer look at using 64bit for their DCs as well - it will be very benefitial down the road. /Guido -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Saturday, July 22, 2006 6:49 PM To: ActiveDir@mail.activedir.org Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers
RE: [ActiveDir] Raid 1 tangent -- Vendor Domain
The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. Actually, log IOs were quite low, considering. I bet a single spindle pair would have been enough for most of my work. The real killer was random I/O throughout the DB. Here I was pushing 1800 read / 1800 write for most of the run. I really needed more SAN paths because I'm pretty sure that was the bottleneck (it just wasn't set up to have as many redundant paths as I didn't anticipate the bottlenecks hit). I keep meaning to write a follow-up post with a lot of data. I'll do so this week and post it so this sort of stuff is a bit more clear. ~Eric -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Saturday, July 22, 2006 9:49 AM To: ActiveDir@mail.activedir.org Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than 40-50k. The smallest I tend to deal with is about 30k. I usually get called to walk in to Exchange issues where Exchange is underperforming or outright hanging, sometimes for hours at a time. There can be all sorts of issues causing this such as O poor disk subsystem design for Exchange (someone say got fancy with a SAN layout and really didn't know what they were doing seems to be popular here) O hardware/drivers on the Exchange server just aren't working properly and the drivers are experiencing timeout issues (for some reason I want to say HBA here) O poor network configurations and odd load balancing solutions, etc that generate a whole bunch of say keep alive traffic on the segment that no one had any idea about because no one understood the solution nor took time to look at the network traces. Or maybe the infamous Full/100 on one end and half/100 on the other. Whatever. O Applications that beat the crap out of Exchange that weren't accounted for in the design well or at all... such as Blackberry or Desktop Search or various Archive solutions O Poorly written event sinks, disclaimer type
RE: [ActiveDir] Raid 1 tangent -- Vendor Domain
- stop using mirrors damnit) .[1] can you please explain that? What's wrong with mirrors? [1] joe, speaking particularly in the context of Exchange List info : http://www.activedir.org/List.aspx List FAQ: http://www.activedir.org/ListFAQ.aspx List archive: http://www.activedir.org/ml/threads.aspx
RE: [ActiveDir] Raid 1 tangent -- Vendor Domain
... OS I am fine with, it is a nice mental breakout of that aspect, but the points in separating the LOGs and the DIT aren't that great that I am aware of unless you expect to run your DIT out of space and you really shouldn't be thinking about doing that (again monitoring but also protecting your directory from letting people add things unhindered). Certainly breaking things out by volume isn't a perf gain and personally I think it adds to the design complexity needlessly. So if your DIT is under 1.5GB and you have the RAM to cache that DIT on K3 AD then a mirror will probably be fine for you. If the DIT is under, what is it about, 2.7 or so GB, and you have the RAM and /3GB on K3 AD enabled then a mirror will probably be fine for you. If you have a WAN site that has some basic users logging on and getting GPOs and accessing file shares locally, a mirror will probably be fine for you. If you are just doing NOS stuff then a mirror may be fine for you even in real large orgs. If you are outside of that criteria, think hard about whether a mirror is right for you and prove that out by watching the disk counters. If you have Exchange beating against your AD and it can't be cached, a mirror is most likely not going to be as performant as it should be for *optimal* Exchange performance. I say optimal because Exchange may appear to be fine but as I often tell people, Exchange will put up with a lot of stupid things until it hits the limit and then it will throw a fit and blow out completely on you and you have to chase through and figure out out of all the stupid things you are doing, which one is the one pushing it over the edge this time so you can fix it (reminds me of some relationships I know of with girls taking on the part of Exchange and guys taking on the part of doing lots of stupid thingseg). I don't have a lot of experience yet with x64 DCs but my gut says that assuming you have enough RAM to cache the entire DIT and you aren't constantly rebooting the DC or doing things that force the cache to be trimmed, the disk subsystem is really only going to be important for writes (which we have already said aren't really all that much of what AD is doing) and the initial caching of the DIT. Let the debates begin. :) joe [1] http://blogs.technet.com/efleis/archive/2006/06/08/434255.aspx [2] BTW, I read that 5000 as total users using AD, not users using that one DC. The more users you have, the more likely your DIT is going to hit a size that can't be cached. [3] Even in one case adfind was used to prove AD was fine and the person didn't know I wrote it... That was an interesting conversation as the person tried to explain to me how ADFIND worked and then I explained he was wrong and laid out the actual algorithm for what it was doing and he said I was wrong and I said I hope not, I wrote it. -- O'Reilly Active Directory Third Edition - http://www.joeware.net/win/ad3e.htm -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Saturday, July 22, 2006 11:06 AM To: ActiveDir@mail.activedir.org Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain - stop using mirrors damnit) .[1] can you please explain that? What's wrong with mirrors? [1] joe, speaking particularly in the context of Exchange List info : http://www.activedir.org/List.aspx List FAQ: http://www.activedir.org/ListFAQ.aspx List archive: http://www.activedir.org/ml/threads.aspx List info : http://www.activedir.org/List.aspx List FAQ: http://www.activedir.org/ListFAQ.aspx List archive: http://www.activedir.org/ml/threads.aspx
Re: [ActiveDir] Raid 1 tangent -- Vendor Domain
no debate from me. I was just asking. Thank you for the lesson. - Original Message - From: joe [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 9:48 AM Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than 40-50k. The smallest I tend to deal with is about 30k. I usually get called to walk in to Exchange issues where Exchange is underperforming or outright hanging, sometimes for hours at a time. There can be all sorts of issues causing this such as O poor disk subsystem design for Exchange (someone say got fancy with a SAN layout and really didn't know what they were doing seems to be popular here) O hardware/drivers on the Exchange server just aren't working properly and the drivers are experiencing timeout issues (for some reason I want to say HBA here) O poor network configurations and odd load balancing solutions, etc that generate a whole bunch of say keep alive traffic on the segment that no one had any idea about because no one understood the solution nor took time to look at the network traces. Or maybe the infamous Full/100 on one end and half/100 on the other. Whatever. O Applications that beat the crap out of Exchange that weren't accounted for in the design well or at all... such as Blackberry or Desktop Search or various Archive solutions O Poorly written event sinks, disclaimer type products that query AD themselves for additional info fit nicely into this category (hint do not deploy one of these unless you understand the queries it generates) O DCs being too far away say like an Exchange server in the US hosting APAC users. If you are running Exchange, you put Exchange and the DCs for the domains of any users on that Exchange server on the same physical subnet. And if you have a multidomain forest, strongly consider shortcut trusts between the domains that the Exchange servers are in with the domains the users are in. O DCs underperforming The last is almost always, heck, I will say in 98% of the cases I have had to investigate, related to DC disk configuration and it is always
Re: [ActiveDir] Raid 1 tangent -- Vendor Domain
joe, you must type really, really fast - Original Message - From: Albert Duro [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 7:06 PM Subject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domain no debate from me. I was just asking. Thank you for the lesson. - Original Message - From: joe [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 9:48 AM Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than 40-50k. The smallest I tend to deal with is about 30k. I usually get called to walk in to Exchange issues where Exchange is underperforming or outright hanging, sometimes for hours at a time. There can be all sorts of issues causing this such as O poor disk subsystem design for Exchange (someone say got fancy with a SAN layout and really didn't know what they were doing seems to be popular here) O hardware/drivers on the Exchange server just aren't working properly and the drivers are experiencing timeout issues (for some reason I want to say HBA here) O poor network configurations and odd load balancing solutions, etc that generate a whole bunch of say keep alive traffic on the segment that no one had any idea about because no one understood the solution nor took time to look at the network traces. Or maybe the infamous Full/100 on one end and half/100 on the other. Whatever. O Applications that beat the crap out of Exchange that weren't accounted for in the design well or at all... such as Blackberry or Desktop Search or various Archive solutions O Poorly written event sinks, disclaimer type products that query AD themselves for additional info fit nicely into this category (hint do not deploy one of these unless you understand the queries it generates) O DCs being too far away say like an Exchange server in the US hosting APAC users. If you are running Exchange, you put Exchange and the DCs for the domains of any users on that Exchange server on the same physical subnet. And if you have a multidomain forest, strongly consider shortcut
RE: [ActiveDir] Raid 1 tangent -- Vendor Domain
That's a command line guy for you... :o) The thing is that I type in a very odd way two, my whole right hand just one or two fingers from my left hand. People tend to get a bit confused when they see me type. joe -- O'Reilly Active Directory Third Edition - http://www.joeware.net/win/ad3e.htm -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kevin Gent Sent: Saturday, July 22, 2006 7:29 PM To: ActiveDir@mail.activedir.org Subject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domain joe, you must type really, really fast - Original Message - From: Albert Duro [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 7:06 PM Subject: Re: [ActiveDir] Raid 1 tangent -- Vendor Domain no debate from me. I was just asking. Thank you for the lesson. - Original Message - From: joe [EMAIL PROTECTED] To: ActiveDir@mail.activedir.org Sent: Saturday, July 22, 2006 9:48 AM Subject: RE: [ActiveDir] Raid 1 tangent -- Vendor Domain Mirrors don't scale. Microsoft's deployment doc mostly just talks about using mirrors (small nod to RAID 10/0+1) so everyone thinks that they should build their Corporate DCs on mirrors, usually 3 - OS, Logs, and DIT. Very few people if anyone would build a corporate Exchange Server on mirrors... Why not? The DB is the same under both of them... What is critical to Exchange? IOPS and that means spindles. If something is really beating on AD and the entire DIT can't be cached, IOPS are critical to AD as well. The main difference is that AD is mostly random read and Exchange is heavy writing and reading. The exception to this is the edge case of Eric's big DIT[1] in which he dumped 2TB of data into AD in a month at which point he did something that few people see, pushed the IOPS on the log drive through the roof. In a smaller environment (very low thousands), or for a low use DC (small WAN site), or a DC with a DIT fully cached a RAID-1 drive for DIT will probably be sufficient, you will note that the only numbers mentioned in the deployment guide are about 5000[2]... That usually means a small DIT and it is extremely likely that a K3 DC will cache the entire DIT. Plus the usage is probably such that the IO capability of two spindles will likely be ok. Let me state though that even in a small user environment if there was an intensive directory based app or a buttload of data that pushes the DIT into GB's instead of MBs I would still be watching my disk queueing pretty close as well as the Read and Write Ops. AD admins who aren't running directory intensive apps (read as Exchange 2000+) usually don't see any issues but then again most aren't looking very closely at the counters because they haven't had a reason too and even if they had some short lived issues they probably wouldn't go look at the counters. At least that has been my experience in dealing with companies. I will admit that prior to implementing Exchange when I did AD Ops with a rather large company I didn't once look at the disk counters, didn't care, everything ran perfectly well and about the only measure of perf was replication latency and does ADUC start fast enough and it always was fine there unless there were network related issues or a DC was having hardware failure. Enter Exchange... Or some other app that pounds your DCs with millions of queries a day and tiny little bits of latency that you didn't previously feel start having an impact. You won't feel 70-80ms of latency in anything you are doing with normal AD tools or NOS ops, not at all. You will feel that with Exchange (and other heavy directory use apps), often with painful results unless it isn't consistent and the directory can unwind itself again and hence allow Exchange to then unwind itself. Now let me point out, I don't deal with tiny companies for work, small to me is less than 40-50k. The smallest I tend to deal with is about 30k. I usually get called to walk in to Exchange issues where Exchange is underperforming or outright hanging, sometimes for hours at a time. There can be all sorts of issues causing this such as O poor disk subsystem design for Exchange (someone say got fancy with a SAN layout and really didn't know what they were doing seems to be popular here) O hardware/drivers on the Exchange server just aren't working properly and the drivers are experiencing timeout issues (for some reason I want to say HBA here) O poor network configurations and odd load balancing solutions, etc that generate a whole bunch of say keep alive traffic on the segment that no one had any idea about because no one understood the solution nor took time to look at the network traces. Or maybe the infamous Full/100 on one end and half/100 on the other. Whatever. O Applications that beat the crap out of Exchange that weren't accounted
