Re: [ActiveDir] NTDS.DIT Size

2006-06-30 Thread Matt Hargraves
32 and 64-bit DCs don't have a problem being in the same environment.  Is your schema up to date with the latest revisions (R2 I believe)?


RE: [ActiveDir] NTDS.DIT Size

2006-06-30 Thread Joshua Coffman


Yes, the dit is small for 250K users. It is really just used as an LDAP user store, and we only care about maybe 30 attributes.
 
Our LSASS process does indeed get to about 1.3 gb every now and then, so maybe its getting close to time to use the /3GB switch, until we can get some new x64 hardware to throw at it.
 
Can 32 and 64 bit ADs comingle?
 
Thanks!
 
Josh



> Date: Fri, 30 Jun 2006 00:44:46 -0700> From: [EMAIL PROTECTED]> To: ActiveDir@mail.activedir.org> Subject: RE: [ActiveDir] NTDS.DIT Size> > > Someone said this:> > Whatever needs to be loaded should perform better when smaller.> > I suspect this is not true, or at least not very significant.  OLD (OnLine> Defrag) cinches up the DB so the fewest pages are used for data and> whitespace is consolidated to whole pages, and so while backup time would> be longer / take up more space, from a caching perspective it shouldn't> perform any better.> > Cheers,> -BrettSh [msft]> > On Fri, 30 Jun 2006, Brian Desmond wrote:> > > Sounds like he's probably just not populating many attributes. I've got> > double that DIT size at a client with half the number of users easily.> > I've also never had a reason to defrag a dit when I can just dcpromo> > down/up if I think it will fix a database issue. > > > >  > > > > Thanks,> > > > Brian Desmond> > > > [EMAIL PROTECTED]> > > >  > > > > c - 312.731.3132> > > >  > > > > From: [EMAIL PROTECTED]> > [mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier,> > Guido> > Sent: Thursday, June 29, 2006 6:53 PM> > To: ActiveDir@mail.activedir.org> > Subject: RE: [ActiveDir] NTDS.DIT Size> > > >  > > > > 1.7GB for 250.000 users is pretty small already - I guess you don't use> > Exchange for messaging or use extremely few attributes of your objects> > in AD.  With the steps outlined by Ulf you should get a fair idea on how> > much whitespace you currently have, however, you shouldn't expect to> > have much if your AD is growing at a fairly constant rate. The database> > grows fairly linear and whitespace is being used automatically be new> > data.> > > >  > > > > As you're talking about moving to 64-bit, I guess you're already using> > Win2003.  On 32-bit Windows 2003 DCs without /3GB switch, the LSASS> > process can consume (cache) up to about 1.5GB, with /3GB it's around> > 2.6GB.  /3GB is supported on both Standard and Enterprise Edition with> > respect to DCs.> > > >  > > > > So theoretically you're well in the limits of the 32-bit OS, as long as> > you have at least 4GB in your DCs and are using the /3GB switch.> > However, the /3GB switch reduces the vitual memory for the kernel down> > to 1GB, with can be a limiting factor in other situations - usually not> > on a DC (if it's not also hosting many other services).> > > >  > > > > But the 64-bit DCs won't cost you one penny extra: almost all server HW> > for the past 12 months has been x64 capable and the 64-bit Win2003> > version has the same licensing costs as the 32-bit version. So you might> > as well go for it and have even more room for growth.  Mind you, with> > your current DIT size you should not expect much performance difference> > for your AD (unless you're replacing old server HW with new HW at the> > same time...).> > > > /Guido> > > >  > > > > > > > > From: [EMAIL PROTECTED]> > [mailto:[EMAIL PROTECTED] On Behalf Of Ulf B.> > Simon-Weidner> > Sent: Donnerstag, 29. Juni 2006 23:47> > To: ActiveDir@mail.activedir.org> > Subject: RE: [ActiveDir] NTDS.DIT Size> > > > Hello Joshua,> > > >  > > > > I'd look at the whitespace to determine when to offline defrag a DC. You> > can enable the associated event which will tell you the amount of> > whitespace by setting the registry key> > HKLM\System\CurrentControlSet\Services\NTDS\Diagnostics\6 Garbage> > Collection to 1 instead of 0 (which is the default). Regkey might be> > likely - just typed it from hard.> > > > This will give you an event every time when garbage collection runs> > (every 12 hrs) and tell you the amount of whitespace in the DB.> > > >  > > > > Whatever needs to be loaded should perform better when smaller.> > > >  > > > > I've heard that a DC on x64 will perform better than on 32-bit, since> > it's very likely you already have some of the newer servers with

RE: [ActiveDir] NTDS.DIT Size

2006-06-30 Thread Brett Shirley

Someone said this:
> Whatever needs to be loaded should perform better when smaller.

I suspect this is not true, or at least not very significant.  OLD (OnLine
Defrag) cinches up the DB so the fewest pages are used for data and
whitespace is consolidated to whole pages, and so while backup time would
be longer / take up more space, from a caching perspective it shouldn't
perform any better.

Cheers,
-BrettSh [msft]

On Fri, 30 Jun 2006, Brian Desmond wrote:

> Sounds like he's probably just not populating many attributes. I've got
> double that DIT size at a client with half the number of users easily.
> I've also never had a reason to defrag a dit when I can just dcpromo
> down/up if I think it will fix a database issue. 
> 
>  
> 
> Thanks,
> 
> Brian Desmond
> 
> [EMAIL PROTECTED]
> 
>  
> 
> c - 312.731.3132
> 
>  
> 
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier,
> Guido
> Sent: Thursday, June 29, 2006 6:53 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.DIT Size
> 
>  
> 
> 1.7GB for 250.000 users is pretty small already - I guess you don't use
> Exchange for messaging or use extremely few attributes of your objects
> in AD.  With the steps outlined by Ulf you should get a fair idea on how
> much whitespace you currently have, however, you shouldn't expect to
> have much if your AD is growing at a fairly constant rate. The database
> grows fairly linear and whitespace is being used automatically be new
> data.
> 
>  
> 
> As you're talking about moving to 64-bit, I guess you're already using
> Win2003.  On 32-bit Windows 2003 DCs without /3GB switch, the LSASS
> process can consume (cache) up to about 1.5GB, with /3GB it's around
> 2.6GB.  /3GB is supported on both Standard and Enterprise Edition with
> respect to DCs.
> 
>  
> 
> So theoretically you're well in the limits of the 32-bit OS, as long as
> you have at least 4GB in your DCs and are using the /3GB switch.
> However, the /3GB switch reduces the vitual memory for the kernel down
> to 1GB, with can be a limiting factor in other situations - usually not
> on a DC (if it's not also hosting many other services).
> 
>  
> 
> But the 64-bit DCs won't cost you one penny extra: almost all server HW
> for the past 12 months has been x64 capable and the 64-bit Win2003
> version has the same licensing costs as the 32-bit version. So you might
> as well go for it and have even more room for growth.  Mind you, with
> your current DIT size you should not expect much performance difference
> for your AD (unless you're replacing old server HW with new HW at the
> same time...).
> 
> /Guido
> 
>  
> 
> 
> 
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Ulf B.
> Simon-Weidner
> Sent: Donnerstag, 29. Juni 2006 23:47
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.DIT Size
> 
> Hello Joshua,
> 
>  
> 
> I'd look at the whitespace to determine when to offline defrag a DC. You
> can enable the associated event which will tell you the amount of
> whitespace by setting the registry key
> HKLM\System\CurrentControlSet\Services\NTDS\Diagnostics\6 Garbage
> Collection to 1 instead of 0 (which is the default). Regkey might be
> likely - just typed it from hard.
> 
> This will give you an event every time when garbage collection runs
> (every 12 hrs) and tell you the amount of whitespace in the DB.
> 
>  
> 
> Whatever needs to be loaded should perform better when smaller.
> 
>  
> 
> I've heard that a DC on x64 will perform better than on 32-bit, since
> it's very likely you already have some of the newer servers with x64 I'd
> just give it a try for one DC yourself.
> 
>  
> 
> Gruesse - Sincerely, 
> 
> Ulf B. Simon-Weidner 
> 
>   Profile & Publications:
> http://mvp.support.microsoft.com/profile=35E388DE-4885-4308-B489-F2F1214
> C811D
> http://mvp.support.microsoft.com/profile=35E388DE-4885-4308-B4
> 89-F2F1214C811D>
>   Weblog: http://msmvps.org/UlfBSimonWeidner
> http://msmvps.org/UlfBSimonWeidner> 
>   Website: http://www.windowsserverfaq.org
> http://www.windowsserverfaq.org/> 
> 
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Joshua Coffman
> Sent: Thursday, June 29, 2006 10:59 PM
> To: ActiveDir@mail.activedir.org
> Subject: [ActiveDir] NTDS.DIT Size
> 
>  
> 
> Our AD (NTDS.dit) is at 1.7GB (approx. 250,000 users).
>  
> Should an offline defrag be performed at a regular interval?
>  
> Some articles I read only say it is only worthwhile if you are running
> low on space. We have plenty of drive space and RAM.
>  
> At what point should the AD be moved to 64 bit?
>  
> Thanks,
>  
> Josh
>  
>  
> 
> 

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ml/threads.aspx


RE: [ActiveDir] NTDS.DIT Size

2006-06-29 Thread Brian Desmond








Sounds like he’s probably just not populating many attributes. I’ve
got double that DIT size at a client with half the number of users easily. I’ve
also never had a reason to defrag a dit when I can just dcpromo down/up if I
think it will fix a database issue. 

 



Thanks,

Brian Desmond

[EMAIL PROTECTED]

 

c - 312.731.3132



 







From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier,
Guido
Sent: Thursday, June 29, 2006 6:53 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.DIT Size





 

1.7GB for 250.000 users is pretty small already - I guess you don't
use Exchange for messaging or use extremely few attributes of your objects in
AD.  With the steps outlined by Ulf you should get a fair idea on how much
whitespace you currently have, however, you shouldn't expect to have much if
your AD is growing at a fairly constant rate. The database grows fairly linear
and whitespace is being used automatically be new data.

 

As you're talking about moving to 64-bit, I guess you're already
using Win2003.  On 32-bit Windows 2003 DCs without /3GB switch, the LSASS
process can consume (cache) up to about 1.5GB, with /3GB it's around
2.6GB.  /3GB is supported on both Standard and Enterprise Edition with
respect to DCs.



 





So theoretically you're well in the limits of the 32-bit OS, as
long as you have at least 4GB in your DCs and are using the /3GB switch. 
However, the /3GB switch reduces the vitual memory for the kernel down to 1GB,
with can be a limiting factor in other situations - usually not on a DC (if
it's not also hosting many other services).





 





But the 64-bit DCs won't cost you one penny extra: almost all
server HW for the past 12 months has been x64 capable and the 64-bit Win2003
version has the same licensing costs as the 32-bit version. So you might as
well go for it and have even more room for growth.  Mind you, with your
current DIT size you should not expect much performance difference
for your AD (unless you're replacing old server HW with new HW at the same
time...).





/Guido



 







From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Ulf B.
Simon-Weidner
Sent: Donnerstag, 29. Juni 2006 23:47
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.DIT Size

Hello Joshua,

 

I’d look at the whitespace to determine when to offline defrag a
DC. You can enable the associated event which will tell you the amount of
whitespace by setting the registry key
HKLM\System\CurrentControlSet\Services\NTDS\Diagnostics\6 Garbage Collection to
1 instead of 0 (which is the default). Regkey might be likely – just typed it
from hard.

This will give you an event every time when garbage collection runs
(every 12 hrs) and tell you the amount of whitespace in the DB.

 

Whatever needs to be loaded should perform better when smaller.

 

I’ve heard that a DC on x64 will perform better than on 32-bit,
since it’s very likely you already have some of the newer servers with x64 I’d
just give it a try for one DC yourself.

 



Gruesse - Sincerely, 

Ulf B. Simon-Weidner 

  Profile &
Publications:   http://mvp.support.microsoft.com/profile="">   
  Weblog: http://msmvps.org/UlfBSimonWeidner
  Website: http://www.windowsserverfaq.org







From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Joshua Coffman
Sent: Thursday, June 29, 2006 10:59 PM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] NTDS.DIT Size





 

Our
AD (NTDS.dit) is at 1.7GB (approx. 250,000 users).
 
Should an offline defrag be performed at a regular interval?
 
Some articles I read only say it is only worthwhile if you are running low on
space. We have plenty of drive space and RAM.
 
At what point should the AD be moved to 64 bit?
 
Thanks,
 
Josh
 
 










RE: [ActiveDir] NTDS.DIT Size

2006-06-29 Thread Grillenmeier, Guido



1.7GB for 250.000 users is pretty small already - I guess 
you don't use Exchange for messaging or use extremely few attributes of your 
objects in AD.  With the steps outlined by Ulf you should get a fair idea 
on how much whitespace you currently have, however, you shouldn't expect to have 
much if your AD is growing at a fairly constant rate. The database grows fairly 
linear and whitespace is being used automatically be new 
data.
 
As you're talking about moving to 64-bit, I guess you're 
already using Win2003.  On 32-bit Windows 2003 DCs without /3GB switch, the 
LSASS process can consume (cache) up to about 1.5GB, with /3GB it's around 
2.6GB.  /3GB is supported on both Standard and Enterprise Edition with 
respect to DCs.
 
So 
theoretically you're well in the limits of the 32-bit OS, as long as you have at 
least 4GB in your DCs and are using the /3GB switch.  However, the /3GB 
switch reduces the vitual memory for the kernel down to 1GB, with can be a 
limiting factor in other situations - usually not on a DC (if it's not also 
hosting many other services).
 
But 
the 64-bit DCs won't cost you one penny extra: almost all server HW for the past 
12 months has been x64 capable and the 64-bit Win2003 version has the same 
licensing costs as the 32-bit version. So you might as well go for it and have 
even more room for growth.  Mind you, with your current DIT 
size you should not expect much performance difference for your AD (unless 
you're replacing old server HW with new HW at the same 
time...).
/Guido


From: [EMAIL PROTECTED] 
[mailto:[EMAIL PROTECTED] On Behalf Of Ulf B. 
Simon-WeidnerSent: Donnerstag, 29. Juni 2006 23:47To: 
ActiveDir@mail.activedir.orgSubject: RE: [ActiveDir] NTDS.DIT 
Size


Hello 
Joshua,
 
I’d 
look at the whitespace to determine when to offline defrag a DC. You can enable 
the associated event which will tell you the amount of whitespace by setting the 
registry key HKLM\System\CurrentControlSet\Services\NTDS\Diagnostics\6 Garbage 
Collection to 1 instead of 0 (which is the default). Regkey might be likely – 
just typed it from hard.
This 
will give you an event every time when garbage collection runs (every 12 hrs) 
and tell you the amount of whitespace in the DB.
 
Whatever 
needs to be loaded should perform better when smaller.
 
I’ve 
heard that a DC on x64 will perform better than on 32-bit, since it’s very 
likely you already have some of the newer servers with x64 I’d just give it a 
try for one DC yourself.
 

Gruesse - 
Sincerely, 
Ulf B. Simon-Weidner 

  Profile & 
Publications:   http://mvp.support.microsoft.com/profile=""> Weblog: http://msmvps.org/UlfBSimonWeidner  Website: http://www.windowsserverfaq.org


From: 
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] 
On Behalf Of Joshua CoffmanSent: Thursday, June 29, 2006 10:59 
PMTo: ActiveDir@mail.activedir.orgSubject: [ActiveDir] 
NTDS.DIT Size
 
Our AD (NTDS.dit) is 
at 1.7GB (approx. 250,000 users). Should an offline defrag be 
performed at a regular interval? Some articles I read only say it 
is only worthwhile if you are running low on space. We have plenty of drive 
space and RAM. At what point should the AD be moved to 64 
bit? Thanks, Josh  


RE: [ActiveDir] NTDS.DIT Size

2006-06-29 Thread Ulf B. Simon-Weidner








Hello Joshua,

 

I’d look at the whitespace to determine when to offline
defrag a DC. You can enable the associated event which will tell you the amount
of whitespace by setting the registry key
HKLM\System\CurrentControlSet\Services\NTDS\Diagnostics\6 Garbage Collection to
1 instead of 0 (which is the default). Regkey might be likely – just typed
it from hard.

This will give you an event every time when garbage collection runs
(every 12 hrs) and tell you the amount of whitespace in the DB.

 

Whatever needs to be loaded should perform better when smaller.

 

I’ve heard that a DC on x64 will perform better than on 32-bit,
since it’s very likely you already have some of the newer servers with
x64 I’d just give it a try for one DC yourself.

 



Gruesse - Sincerely, 

Ulf B. Simon-Weidner 

  Profile &
Publications:   http://mvp.support.microsoft.com/profile="">   
  Weblog: http://msmvps.org/UlfBSimonWeidner
  Website: http://www.windowsserverfaq.org







From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Joshua Coffman
Sent: Thursday, June 29, 2006 10:59 PM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] NTDS.DIT Size





 

Our
AD (NTDS.dit) is at 1.7GB (approx. 250,000 users).
 
Should an offline defrag be performed at a regular interval?
 
Some articles I read only say it is only worthwhile if you are running low on
space. We have plenty of drive space and RAM.
 
At what point should the AD be moved to 64 bit?
 
Thanks,
 
Josh
 
 








RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Grillenmeier, Guido
yep, that's what I meant - but I was too lazy to add these details ;-)

/Guido

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: Freitag, 15. April 2005 17:56
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Better yet:
http://search.msn.com/results.aspx?q=DNS+2003+%22application+partition%2
2&FORM=QBHP

I would point out, moving to app partitions does not _shrink_ the size
of the data you have to store in the aggregate as has been eluded to.
Rather, it does two things:
1) It lets you control the scope of where it is stored so non-DNS
servers don't need to keep a copy around
2) It removes the partial NC copies from GCs in other domains in the
forest, who do nothing but house these little guys (at least a PAS-worth
of them)

I know the posters probably meant this, but they didn't really state it,
so I wanted to clarify.

~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Carlos
Magalhaes
Sent: Friday, April 15, 2005 6:23 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Well Francis,

How is your DNS servers setup are they:

1. Windows DNS servers
2. Have you sepecified that your Zones are Active Directory Intergrated
Zones

If you haven't created the default DNS app partions right click on your
DNS server ---> "Create Default DNS application Partitions"  this will
create two APP partitions:

1. ForestDNS
2. DomainDNS

HTH

Carlos Magalhaes

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Francis Ouellet
Sent: 15 April 2005 02:58 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Hi Guido,

Can you provide us with some more information on moving the DNS data
into the DNS app partition?

Thanks!
Francis 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier,
Guido
Sent: 15 avril 2005 04:00
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

It's also worth to point out, that you have to distinguish heavily
between the OS version and the DIT size to expect. Other cleanup tasks
can also strongly impact DIT size. 

At HP our Win2000 GCs had an average DIT size of 18GB - we then disabled
the "Distributed Link Tracking" service on all DCs as it feeds AD with a
ton of garbage information (actually the information would be quite
useful if any app were using it - but as even the MS apps make no use to
lookup the new location of moved files in AD, this service is useless).
After removal of a ton of link-objects which were collected over the
years in each domain's \System\FileLinks container, we decreased the DIT
size easily by 6GB (don't have the exact values of the top of my head) -
naturally this was after the tombstone lifetime and an offline defrag.
So now we were down down to something like 12GB.  Checkout Q312403 for
more details - if you're running a new Win2003 AD, this service will be
turned off by default.

Then the first Win2003 DCs were introduced (we did perform some inplace
upgrades, but eventually all of them were re-installed) => the
single-instance store of ACEs introduced in Win2003 saved us another 5GB
and thus got us down to 7GB => so now we're 11GB less than it was for a
Win2000 DC with DLT objects ;-)

We've further improved DIT size (and replication) by moving the DNS data
into the DNS app partitions (so that they're not part of the GC). But
this impact is not as dramatic (will mostly impact DIT on those DCs
which aren't DNS servers...)

/Guido

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Freitag, 15. April 2005 05:43
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20 GB
TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
>

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Eric Fleischman
Sure. There is a good chunk of the db that doesn't replicate because it
is outside of the AD object model (example: indexes) or marked to not
replicate (ex: some attributes). But in the aggregate, for most objects,
a fair statement...without clouding the issue with the nuances.


~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Friday, April 15, 2005 9:02 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Just to clarify, it is the parts that change and are tagged to replicate
that replicate. You could have shitloads of changes occuring that never
leave the DC. 

  joe 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: Friday, April 15, 2005 11:45 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Trick question? The parts of the 100gb that will replicate are the parts
that change. (not counting dcpromo of new boxes) How much is changing?
Who
knows. Different for everyone.

~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Carlos
Magalhaes
Sent: Friday, April 15, 2005 2:32 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Eric,

Granted but how much of that actual 100gb will be replicated over that
64k
line? I can see the issue if you do a DC promo on a W2k3 server on the
other
size and it's the first box and has to pull info over 64k, but once
established that traffic shouldn't even be close to 100mb.'

That said it is also environment dependant :P

Carlos Magalhaes

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: 15 April 2005 06:00 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Oops, I typo'd. First paragraph should have read:

--
It's hard to characterize how "much" connectivity you need vs. how big
your
db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that changes a _ton_. So really, it's all
about
your rate of change, with the size only being a guideline.
--

I would also add, that in the average case, you're rightlarge DBs
_tend_
to require more bandwidth than smaller ones. I can't picture a 100gb DB
on
the other side of a 64k link being good in the average case.
:)

~Eric



-Original Message-
From: Eric Fleischman
Sent: Thursday, April 14, 2005 8:56 PM
To: 'ActiveDir@mail.activedir.org'
Subject: RE: [ActiveDir] NTDS.dit size

It's hard to characterize how "much" connectivity you need vs. how big
your
db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that doesn't change very much. So really,
it's
all about your rate of change, with the size only being a guideline.

For promotion, at that scale, IFM is clearly the way to go. But there's
nothing wrong with the occasional promotion that is over the wire. It'll
finish, it will just take a while, even on a fast network.

With a 20gb db, a few things might help you:
1) Explore 64bit (ia64 or x64). Recall that on 2k3 32bit your best case
cache is ~2.6gb in size. With 64bit, the sky is the limitthrow ram
at a
DC, and it will use it to cache more of the db. DB caching cuts down on
the
I/O required for reads (which for most people are the bulk of their
load)
and help your perf a lot.
2) If you're on 32bit, I like boxes w/~4gb of physical memory, nothing
else
on them, and /3gb set. It lets you really use your cache well, and still
have some headroom for the OS and tools you might use here and there.
3) I'm a fan of profiling traffic hitting my DCs and optimizing the
queries
for AD, and possibly optimizing AD for the queries (both are on the
table).
Tools like SPA, field engineering logging (mentioned in a thread on this
dl
earlier today) and any 3rd party tools you might like all can help here.
Though this advise isn't specific to large DBs..I like making things
faster at any scale. :)
4) Standard disk logic about optimizing I/O throughput applies.
5) Some people "warm" the cache on DC boot. This is particularly
interesting
on 64bit DCs where you have tons of memory headroom. That is, after the
box
boots they run some really expensive queries that walk very expensive
indexes (ancestry, dnt, etc.) to traverse as many objects as they can,
and
get them off of the disk and in to memory. It hits the DC hard from an
I/O
standpoint on boot, but it does get a lot of the db in to memory for
actual
load that starts to hit the box after. It's done in more environments
than
one. I like the idea quite a bit, and have thought about if there is
anything we should do in the product to help facilitate this.

The list is of course endless, but these are a few things that come to
mind.

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread joe
Just to clarify, it is the parts that change and are tagged to replicate
that replicate. You could have shitloads of changes occuring that never
leave the DC. 

  joe 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: Friday, April 15, 2005 11:45 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Trick question? The parts of the 100gb that will replicate are the parts
that change. (not counting dcpromo of new boxes) How much is changing? Who
knows. Different for everyone.

~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Carlos Magalhaes
Sent: Friday, April 15, 2005 2:32 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Eric,

Granted but how much of that actual 100gb will be replicated over that 64k
line? I can see the issue if you do a DC promo on a W2k3 server on the other
size and it's the first box and has to pull info over 64k, but once
established that traffic shouldn't even be close to 100mb.'

That said it is also environment dependant :P

Carlos Magalhaes

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: 15 April 2005 06:00 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Oops, I typo'd. First paragraph should have read:

--
It's hard to characterize how "much" connectivity you need vs. how big your
db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that changes a _ton_. So really, it's all about
your rate of change, with the size only being a guideline.
--

I would also add, that in the average case, you're rightlarge DBs _tend_
to require more bandwidth than smaller ones. I can't picture a 100gb DB on
the other side of a 64k link being good in the average case.
:)

~Eric



-Original Message-
From: Eric Fleischman
Sent: Thursday, April 14, 2005 8:56 PM
To: 'ActiveDir@mail.activedir.org'
Subject: RE: [ActiveDir] NTDS.dit size

It's hard to characterize how "much" connectivity you need vs. how big your
db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that doesn't change very much. So really, it's
all about your rate of change, with the size only being a guideline.

For promotion, at that scale, IFM is clearly the way to go. But there's
nothing wrong with the occasional promotion that is over the wire. It'll
finish, it will just take a while, even on a fast network.

With a 20gb db, a few things might help you:
1) Explore 64bit (ia64 or x64). Recall that on 2k3 32bit your best case
cache is ~2.6gb in size. With 64bit, the sky is the limitthrow ram at a
DC, and it will use it to cache more of the db. DB caching cuts down on the
I/O required for reads (which for most people are the bulk of their load)
and help your perf a lot.
2) If you're on 32bit, I like boxes w/~4gb of physical memory, nothing else
on them, and /3gb set. It lets you really use your cache well, and still
have some headroom for the OS and tools you might use here and there.
3) I'm a fan of profiling traffic hitting my DCs and optimizing the queries
for AD, and possibly optimizing AD for the queries (both are on the table).
Tools like SPA, field engineering logging (mentioned in a thread on this dl
earlier today) and any 3rd party tools you might like all can help here.
Though this advise isn't specific to large DBs..I like making things
faster at any scale. :)
4) Standard disk logic about optimizing I/O throughput applies.
5) Some people "warm" the cache on DC boot. This is particularly interesting
on 64bit DCs where you have tons of memory headroom. That is, after the box
boots they run some really expensive queries that walk very expensive
indexes (ancestry, dnt, etc.) to traverse as many objects as they can, and
get them off of the disk and in to memory. It hits the DC hard from an I/O
standpoint on boot, but it does get a lot of the db in to memory for actual
load that starts to hit the box after. It's done in more environments than
one. I like the idea quite a bit, and have thought about if there is
anything we should do in the product to help facilitate this.

The list is of course endless, but these are a few things that come to mind.

My $0.02
~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Thursday, April 14, 2005 8:43 PM
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20 GB TB
SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install AD
from media in 2003 but I would think there would b

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Eric Fleischman
Better yet:
http://search.msn.com/results.aspx?q=DNS+2003+%22application+partition%2
2&FORM=QBHP

I would point out, moving to app partitions does not _shrink_ the size
of the data you have to store in the aggregate as has been eluded to.
Rather, it does two things:
1) It lets you control the scope of where it is stored so non-DNS
servers don't need to keep a copy around
2) It removes the partial NC copies from GCs in other domains in the
forest, who do nothing but house these little guys (at least a PAS-worth
of them)

I know the posters probably meant this, but they didn't really state it,
so I wanted to clarify.

~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Carlos
Magalhaes
Sent: Friday, April 15, 2005 6:23 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Well Francis,

How is your DNS servers setup are they:

1. Windows DNS servers
2. Have you sepecified that your Zones are Active Directory Intergrated
Zones

If you haven't created the default DNS app partions right click on your
DNS server ---> "Create Default DNS application Partitions"  this will
create two APP partitions:

1. ForestDNS
2. DomainDNS

HTH

Carlos Magalhaes

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Francis Ouellet
Sent: 15 April 2005 02:58 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Hi Guido,

Can you provide us with some more information on moving the DNS data
into the DNS app partition?

Thanks!
Francis 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier,
Guido
Sent: 15 avril 2005 04:00
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

It's also worth to point out, that you have to distinguish heavily
between the OS version and the DIT size to expect. Other cleanup tasks
can also strongly impact DIT size. 

At HP our Win2000 GCs had an average DIT size of 18GB - we then disabled
the "Distributed Link Tracking" service on all DCs as it feeds AD with a
ton of garbage information (actually the information would be quite
useful if any app were using it - but as even the MS apps make no use to
lookup the new location of moved files in AD, this service is useless).
After removal of a ton of link-objects which were collected over the
years in each domain's \System\FileLinks container, we decreased the DIT
size easily by 6GB (don't have the exact values of the top of my head) -
naturally this was after the tombstone lifetime and an offline defrag.
So now we were down down to something like 12GB.  Checkout Q312403 for
more details - if you're running a new Win2003 AD, this service will be
turned off by default.

Then the first Win2003 DCs were introduced (we did perform some inplace
upgrades, but eventually all of them were re-installed) => the
single-instance store of ACEs introduced in Win2003 saved us another 5GB
and thus got us down to 7GB => so now we're 11GB less than it was for a
Win2000 DC with DLT objects ;-)

We've further improved DIT size (and replication) by moving the DNS data
into the DNS app partitions (so that they're not part of the GC). But
this impact is not as dramatic (will mostly impact DIT on those DCs
which aren't DNS servers...)

/Guido

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Freitag, 15. April 2005 05:43
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20 GB
TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too
as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you

> for this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-
> F

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Eric Fleischman
Trick question? The parts of the 100gb that will replicate are the parts
that change. (not counting dcpromo of new boxes)
How much is changing? Who knows. Different for everyone.

~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Carlos
Magalhaes
Sent: Friday, April 15, 2005 2:32 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Eric,

Granted but how much of that actual 100gb will be replicated over that
64k line? I can see the issue if you do a DC promo on a W2k3 server on
the other size and it's the first box and has to pull info over 64k, but
once established that traffic shouldn't even be close to 100mb.'

That said it is also environment dependant :P

Carlos Magalhaes

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: 15 April 2005 06:00 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Oops, I typo'd. First paragraph should have read:

--
It's hard to characterize how "much" connectivity you need vs. how big
your db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that changes a _ton_. So really, it's all
about your rate of change, with the size only being a guideline.
--

I would also add, that in the average case, you're rightlarge DBs
_tend_ to require more bandwidth than smaller ones. I can't picture a
100gb DB on the other side of a 64k link being good in the average case.
:)

~Eric



-Original Message-
From: Eric Fleischman 
Sent: Thursday, April 14, 2005 8:56 PM
To: 'ActiveDir@mail.activedir.org'
Subject: RE: [ActiveDir] NTDS.dit size

It's hard to characterize how "much" connectivity you need vs. how big
your db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that doesn't change very much. So really,
it's all about your rate of change, with the size only being a
guideline.

For promotion, at that scale, IFM is clearly the way to go. But there's
nothing wrong with the occasional promotion that is over the wire. It'll
finish, it will just take a while, even on a fast network.

With a 20gb db, a few things might help you:
1) Explore 64bit (ia64 or x64). Recall that on 2k3 32bit your best case
cache is ~2.6gb in size. With 64bit, the sky is the limitthrow ram
at a DC, and it will use it to cache more of the db. DB caching cuts
down on the I/O required for reads (which for most people are the bulk
of their load) and help your perf a lot.
2) If you're on 32bit, I like boxes w/~4gb of physical memory, nothing
else on them, and /3gb set. It lets you really use your cache well, and
still have some headroom for the OS and tools you might use here and
there.
3) I'm a fan of profiling traffic hitting my DCs and optimizing the
queries for AD, and possibly optimizing AD for the queries (both are on
the table). Tools like SPA, field engineering logging (mentioned in a
thread on this dl earlier today) and any 3rd party tools you might like
all can help here. Though this advise isn't specific to large DBs..I
like making things faster at any scale. :)
4) Standard disk logic about optimizing I/O throughput applies.
5) Some people "warm" the cache on DC boot. This is particularly
interesting on 64bit DCs where you have tons of memory headroom. That
is, after the box boots they run some really expensive queries that walk
very expensive indexes (ancestry, dnt, etc.) to traverse as many objects
as they can, and get them off of the disk and in to memory. It hits the
DC hard from an I/O standpoint on boot, but it does get a lot of the db
in to memory for actual load that starts to hit the box after. It's done
in more environments than one. I like the idea quite a bit, and have
thought about if there is anything we should do in the product to help
facilitate this.

The list is of course endless, but these are a few things that come to
mind.

My $0.02
~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Thursday, April 14, 2005 8:43 PM
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20
GB TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Carlos Magalhaes
Well Francis,

How is your DNS servers setup are they:

1. Windows DNS servers
2. Have you sepecified that your Zones are Active Directory Intergrated
Zones

If you haven't created the default DNS app partions right click on your
DNS server ---> "Create Default DNS application Partitions"  this will
create two APP partitions:

1. ForestDNS
2. DomainDNS

HTH

Carlos Magalhaes

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Francis Ouellet
Sent: 15 April 2005 02:58 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Hi Guido,

Can you provide us with some more information on moving the DNS data
into the DNS app partition?

Thanks!
Francis 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier,
Guido
Sent: 15 avril 2005 04:00
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

It's also worth to point out, that you have to distinguish heavily
between the OS version and the DIT size to expect. Other cleanup tasks
can also strongly impact DIT size. 

At HP our Win2000 GCs had an average DIT size of 18GB - we then disabled
the "Distributed Link Tracking" service on all DCs as it feeds AD with a
ton of garbage information (actually the information would be quite
useful if any app were using it - but as even the MS apps make no use to
lookup the new location of moved files in AD, this service is useless).
After removal of a ton of link-objects which were collected over the
years in each domain's \System\FileLinks container, we decreased the DIT
size easily by 6GB (don't have the exact values of the top of my head) -
naturally this was after the tombstone lifetime and an offline defrag.
So now we were down down to something like 12GB.  Checkout Q312403 for
more details - if you're running a new Win2003 AD, this service will be
turned off by default.

Then the first Win2003 DCs were introduced (we did perform some inplace
upgrades, but eventually all of them were re-installed) => the
single-instance store of ACEs introduced in Win2003 saved us another 5GB
and thus got us down to 7GB => so now we're 11GB less than it was for a
Win2000 DC with DLT objects ;-)

We've further improved DIT size (and replication) by moving the DNS data
into the DNS app partitions (so that they're not part of the GC). But
this impact is not as dramatic (will mostly impact DIT on those DCs
which aren't DNS servers...)

/Guido

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Freitag, 15. April 2005 05:43
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20 GB
TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too
as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you

> for this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Eric
Fleischman
> Sent: Thursday, April 14, 2005 7:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> I've seen larger.
> I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
> 100GB+ on a few occasions.
> 
> ~Eric
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:28 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> The largest production DIT I have personally seen was on the order of 
> 8GB for the GC DIT for a Fortune 5 company running about 250k users of
which
> about 180k were Exchange enabled. Also had some 250k contacts, 200k or

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Francis Ouellet
Hi Guido,

Can you provide us with some more information on moving the DNS data
into the DNS app partition?

Thanks!
Francis 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier,
Guido
Sent: 15 avril 2005 04:00
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

It's also worth to point out, that you have to distinguish heavily
between the OS version and the DIT size to expect. Other cleanup tasks
can also strongly impact DIT size. 

At HP our Win2000 GCs had an average DIT size of 18GB - we then disabled
the "Distributed Link Tracking" service on all DCs as it feeds AD with a
ton of garbage information (actually the information would be quite
useful if any app were using it - but as even the MS apps make no use to
lookup the new location of moved files in AD, this service is useless).
After removal of a ton of link-objects which were collected over the
years in each domain's \System\FileLinks container, we decreased the DIT
size easily by 6GB (don't have the exact values of the top of my head) -
naturally this was after the tombstone lifetime and an offline defrag.
So now we were down down to something like 12GB.  Checkout Q312403 for
more details - if you're running a new Win2003 AD, this service will be
turned off by default.

Then the first Win2003 DCs were introduced (we did perform some inplace
upgrades, but eventually all of them were re-installed) => the
single-instance store of ACEs introduced in Win2003 saved us another 5GB
and thus got us down to 7GB => so now we're 11GB less than it was for a
Win2000 DC with DLT objects ;-)

We've further improved DIT size (and replication) by moving the DNS data
into the DNS app partitions (so that they're not part of the GC). But
this impact is not as dramatic (will mostly impact DIT on those DCs
which aren't DNS servers...)

/Guido

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Freitag, 15. April 2005 05:43
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20 GB
TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too
as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you

> for this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Eric
Fleischman
> Sent: Thursday, April 14, 2005 7:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> I've seen larger.
> I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
> 100GB+ on a few occasions.
> 
> ~Eric
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:28 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> The largest production DIT I have personally seen was on the order of 
> 8GB for the GC DIT for a Fortune 5 company running about 250k users of
which
> about 180k were Exchange enabled. Also had some 250k contacts, 200k or

> so computer objects, 100k or so group objects and consisted of 9 
> domains.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of mike kline
> Sent: Tuesday, April 12, 2005 2:53 PM
> To: ActiveDir@mail.activedir.org
> Subject: [ActiveDir] NTDS.dit size
> 
> I know that AD can have millions of objects, just trying to see what
the
> real world size of some your AD databases are.  Do any of you have 
> databases greater than 20GB+... or more?
> 
> Thanks
> Mike
> List info   : http://www.act

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread joe
Good points. The link tracking was indeed a bite in the ass. I caught that
one pretty early on the game so it didn't give us significant growth though.
I was busy shutting down all of the services and I made MS tell me what that
one was for and I was like... I don't want that, and killed it in the DC
policy and purged the small number of objects we had (maybe 5000). 

The switch to K3 from 2K did significantly reduce the DIT size as well, I
actually think it was on the order of 30-40% and took the GC DIT to around
5-6GB from the 8GB it was on the 2K DCs.

For the DITs up in the 50-100GB range that Eric saw I would strongly
question the data going into the directory. That sounds like a company that
took MS's early ramblings of AD as the every directory to heart and actually
did it forgetting the primary functionality of the directory and what I
think should be protected at all costs, the NOS aspects of the directory.
Remember the more garbage you have in the directory that is undergoing
change (or churn if you want) the slower you are getting NOS specific
updates replicated around. All of that stuff goes through the same
replication system and urgent replication means things are queued urgently,
not replicated urgently[1].


  joe


[1] At least that was the case the last time I watched the replication queue
for any serious length of time. 




-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Grillenmeier, Guido
Sent: Friday, April 15, 2005 4:00 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

It's also worth to point out, that you have to distinguish heavily between
the OS version and the DIT size to expect. Other cleanup tasks can also
strongly impact DIT size. 

At HP our Win2000 GCs had an average DIT size of 18GB - we then disabled the
"Distributed Link Tracking" service on all DCs as it feeds AD with a ton of
garbage information (actually the information would be quite useful if any
app were using it - but as even the MS apps make no use to lookup the new
location of moved files in AD, this service is useless).
After removal of a ton of link-objects which were collected over the years
in each domain's \System\FileLinks container, we decreased the DIT size
easily by 6GB (don't have the exact values of the top of my head) -
naturally this was after the tombstone lifetime and an offline defrag.
So now we were down down to something like 12GB.  Checkout Q312403 for more
details - if you're running a new Win2003 AD, this service will be turned
off by default.

Then the first Win2003 DCs were introduced (we did perform some inplace
upgrades, but eventually all of them were re-installed) => the
single-instance store of ACEs introduced in Win2003 saved us another 5GB and
thus got us down to 7GB => so now we're 11GB less than it was for a Win2000
DC with DLT objects ;-)

We've further improved DIT size (and replication) by moving the DNS data
into the DNS app partitions (so that they're not part of the GC). But this
impact is not as dramatic (will mostly impact DIT on those DCs which aren't
DNS servers...)

/Guido

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Freitag, 15. April 2005 05:43
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20 GB TB
SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install AD
from media in 2003 but I would think there would be problems in a 2000
domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the nice
tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too
as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you 
> for this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Eric
Fleischman
> Sent: Thursday, April 14, 2005 7:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread joe
Braggert.  ;o) 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: Thursday, April 14, 2005 11:36 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Well I've seen very very large in test on many occasions. The numbers I
cited below (with those very descriptive adjectives) are just what I've seen
in production. I didn't think test counted.

If you want to count test, I could fire up a test db that is a TB or so on a
san I have nearby. :)


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Thursday, April 14, 2005 4:58 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

See I almost cc'ed you on the response to get your input on this too as I
knew you had played with some 16GB+ DITS but didn't want to bother you for
this and didn't want to speak out of turn for you.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: Thursday, April 14, 2005 7:35 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

I've seen larger.
I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
100GB+ on a few occasions.

~Eric



-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Thursday, April 14, 2005 4:28 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

The largest production DIT I have personally seen was on the order of 8GB
for the GC DIT for a Fortune 5 company running about 250k users of which
about 180k were Exchange enabled. Also had some 250k contacts, 200k or so
computer objects, 100k or so group objects and consisted of 9 domains.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Tuesday, April 12, 2005 2:53 PM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] NTDS.dit size

I know that AD can have millions of objects, just trying to see what the
real world size of some your AD databases are.  Do any of you have databases
greater than 20GB+... or more?

Thanks
Mike
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/


RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Carlos Magalhaes
Eric,

Granted but how much of that actual 100gb will be replicated over that
64k line? I can see the issue if you do a DC promo on a W2k3 server on
the other size and it's the first box and has to pull info over 64k, but
once established that traffic shouldn't even be close to 100mb.'

That said it is also environment dependant :P

Carlos Magalhaes

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: 15 April 2005 06:00 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

Oops, I typo'd. First paragraph should have read:

--
It's hard to characterize how "much" connectivity you need vs. how big
your db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that changes a _ton_. So really, it's all
about your rate of change, with the size only being a guideline.
--

I would also add, that in the average case, you're rightlarge DBs
_tend_ to require more bandwidth than smaller ones. I can't picture a
100gb DB on the other side of a 64k link being good in the average case.
:)

~Eric



-Original Message-
From: Eric Fleischman 
Sent: Thursday, April 14, 2005 8:56 PM
To: 'ActiveDir@mail.activedir.org'
Subject: RE: [ActiveDir] NTDS.dit size

It's hard to characterize how "much" connectivity you need vs. how big
your db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that doesn't change very much. So really,
it's all about your rate of change, with the size only being a
guideline.

For promotion, at that scale, IFM is clearly the way to go. But there's
nothing wrong with the occasional promotion that is over the wire. It'll
finish, it will just take a while, even on a fast network.

With a 20gb db, a few things might help you:
1) Explore 64bit (ia64 or x64). Recall that on 2k3 32bit your best case
cache is ~2.6gb in size. With 64bit, the sky is the limitthrow ram
at a DC, and it will use it to cache more of the db. DB caching cuts
down on the I/O required for reads (which for most people are the bulk
of their load) and help your perf a lot.
2) If you're on 32bit, I like boxes w/~4gb of physical memory, nothing
else on them, and /3gb set. It lets you really use your cache well, and
still have some headroom for the OS and tools you might use here and
there.
3) I'm a fan of profiling traffic hitting my DCs and optimizing the
queries for AD, and possibly optimizing AD for the queries (both are on
the table). Tools like SPA, field engineering logging (mentioned in a
thread on this dl earlier today) and any 3rd party tools you might like
all can help here. Though this advise isn't specific to large DBs..I
like making things faster at any scale. :)
4) Standard disk logic about optimizing I/O throughput applies.
5) Some people "warm" the cache on DC boot. This is particularly
interesting on 64bit DCs where you have tons of memory headroom. That
is, after the box boots they run some really expensive queries that walk
very expensive indexes (ancestry, dnt, etc.) to traverse as many objects
as they can, and get them off of the disk and in to memory. It hits the
DC hard from an I/O standpoint on boot, but it does get a lot of the db
in to memory for actual load that starts to hit the box after. It's done
in more environments than one. I like the idea quite a bit, and have
thought about if there is anything we should do in the product to help
facilitate this.

The list is of course endless, but these are a few things that come to
mind.

My $0.02
~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Thursday, April 14, 2005 8:43 PM
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20
GB TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -----Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir

RE: [ActiveDir] NTDS.dit size

2005-04-15 Thread Grillenmeier, Guido
It's also worth to point out, that you have to distinguish heavily
between the OS version and the DIT size to expect. Other cleanup tasks
can also strongly impact DIT size. 

At HP our Win2000 GCs had an average DIT size of 18GB - we then disabled
the "Distributed Link Tracking" service on all DCs as it feeds AD with a
ton of garbage information (actually the information would be quite
useful if any app were using it - but as even the MS apps make no use to
lookup the new location of moved files in AD, this service is useless).
After removal of a ton of link-objects which were collected over the
years in each domain's \System\FileLinks container, we decreased the DIT
size easily by 6GB (don't have the exact values of the top of my head) -
naturally this was after the tombstone lifetime and an offline defrag.
So now we were down down to something like 12GB.  Checkout Q312403 for
more details - if you're running a new Win2003 AD, this service will be
turned off by default.

Then the first Win2003 DCs were introduced (we did perform some inplace
upgrades, but eventually all of them were re-installed) => the
single-instance store of ACEs introduced in Win2003 saved us another 5GB
and thus got us down to 7GB => so now we're 11GB less than it was for a
Win2000 DC with DLT objects ;-)

We've further improved DIT size (and replication) by moving the DNS data
into the DNS app partitions (so that they're not part of the GC). But
this impact is not as dramatic (will mostly impact DIT on those DCs
which aren't DNS servers...)

/Guido

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Freitag, 15. April 2005 05:43
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20
GB TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too
as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you
> for
> this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Eric
Fleischman
> Sent: Thursday, April 14, 2005 7:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> I've seen larger.
> I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
> 100GB+ on a few occasions.
> 
> ~Eric
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:28 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> The largest production DIT I have personally seen was on the order of
> 8GB
> for the GC DIT for a Fortune 5 company running about 250k users of
which
> about 180k were Exchange enabled. Also had some 250k contacts, 200k or
> so
> computer objects, 100k or so group objects and consisted of 9 domains.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of mike kline
> Sent: Tuesday, April 12, 2005 2:53 PM
> To: ActiveDir@mail.activedir.org
> Subject: [ActiveDir] NTDS.dit size
> 
> I know that AD can have millions of objects, just trying to see what
the
> real world size of some your AD databases are.  Do any of you have
> databases
> greater than 20GB+... or more?
> 
> Thanks
> Mike
> List info   : http://www.activedir.org/List.aspx
> List FAQ: http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
> 
> List info   : http://www.activedir.org/List.aspx
> List FAQ: http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/ac

RE: [ActiveDir] NTDS.dit size

2005-04-14 Thread Eric Fleischman
Oops, I typo'd. First paragraph should have read:

--
It's hard to characterize how "much" connectivity you need vs. how big
your db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that changes a _ton_. So really, it's all
about your rate of change, with the size only being a guideline.
--

I would also add, that in the average case, you're rightlarge DBs
_tend_ to require more bandwidth than smaller ones. I can't picture a
100gb DB on the other side of a 64k link being good in the average case.
:)

~Eric



-Original Message-
From: Eric Fleischman 
Sent: Thursday, April 14, 2005 8:56 PM
To: 'ActiveDir@mail.activedir.org'
Subject: RE: [ActiveDir] NTDS.dit size

It's hard to characterize how "much" connectivity you need vs. how big
your db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that doesn't change very much. So really,
it's all about your rate of change, with the size only being a
guideline.

For promotion, at that scale, IFM is clearly the way to go. But there's
nothing wrong with the occasional promotion that is over the wire. It'll
finish, it will just take a while, even on a fast network.

With a 20gb db, a few things might help you:
1) Explore 64bit (ia64 or x64). Recall that on 2k3 32bit your best case
cache is ~2.6gb in size. With 64bit, the sky is the limitthrow ram
at a DC, and it will use it to cache more of the db. DB caching cuts
down on the I/O required for reads (which for most people are the bulk
of their load) and help your perf a lot.
2) If you're on 32bit, I like boxes w/~4gb of physical memory, nothing
else on them, and /3gb set. It lets you really use your cache well, and
still have some headroom for the OS and tools you might use here and
there.
3) I'm a fan of profiling traffic hitting my DCs and optimizing the
queries for AD, and possibly optimizing AD for the queries (both are on
the table). Tools like SPA, field engineering logging (mentioned in a
thread on this dl earlier today) and any 3rd party tools you might like
all can help here. Though this advise isn't specific to large DBs..I
like making things faster at any scale. :)
4) Standard disk logic about optimizing I/O throughput applies.
5) Some people "warm" the cache on DC boot. This is particularly
interesting on 64bit DCs where you have tons of memory headroom. That
is, after the box boots they run some really expensive queries that walk
very expensive indexes (ancestry, dnt, etc.) to traverse as many objects
as they can, and get them off of the disk and in to memory. It hits the
DC hard from an I/O standpoint on boot, but it does get a lot of the db
in to memory for actual load that starts to hit the box after. It's done
in more environments than one. I like the idea quite a bit, and have
thought about if there is anything we should do in the product to help
facilitate this.

The list is of course endless, but these are a few things that come to
mind.

My $0.02
~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Thursday, April 14, 2005 8:43 PM
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20
GB TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too
as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you
> for
> this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Eric
Fleischman
> Sent: Thursday, April 14, 2005 7:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> I've seen larger.
> I&

RE: [ActiveDir] NTDS.dit size

2005-04-14 Thread Eric Fleischman
It's hard to characterize how "much" connectivity you need vs. how big
your db is.  A huge db of mostly static info doesn't need nearly as much
connectivity as a smaller db that doesn't change very much. So really,
it's all about your rate of change, with the size only being a
guideline.

For promotion, at that scale, IFM is clearly the way to go. But there's
nothing wrong with the occasional promotion that is over the wire. It'll
finish, it will just take a while, even on a fast network.

With a 20gb db, a few things might help you:
1) Explore 64bit (ia64 or x64). Recall that on 2k3 32bit your best case
cache is ~2.6gb in size. With 64bit, the sky is the limitthrow ram
at a DC, and it will use it to cache more of the db. DB caching cuts
down on the I/O required for reads (which for most people are the bulk
of their load) and help your perf a lot.
2) If you're on 32bit, I like boxes w/~4gb of physical memory, nothing
else on them, and /3gb set. It lets you really use your cache well, and
still have some headroom for the OS and tools you might use here and
there.
3) I'm a fan of profiling traffic hitting my DCs and optimizing the
queries for AD, and possibly optimizing AD for the queries (both are on
the table). Tools like SPA, field engineering logging (mentioned in a
thread on this dl earlier today) and any 3rd party tools you might like
all can help here. Though this advise isn't specific to large DBs..I
like making things faster at any scale. :)
4) Standard disk logic about optimizing I/O throughput applies.
5) Some people "warm" the cache on DC boot. This is particularly
interesting on 64bit DCs where you have tons of memory headroom. That
is, after the box boots they run some really expensive queries that walk
very expensive indexes (ancestry, dnt, etc.) to traverse as many objects
as they can, and get them off of the disk and in to memory. It hits the
DC hard from an I/O standpoint on boot, but it does get a lot of the db
in to memory for actual load that starts to hit the box after. It's done
in more environments than one. I like the idea quite a bit, and have
thought about if there is anything we should do in the product to help
facilitate this.

The list is of course endless, but these are a few things that come to
mind.

My $0.02
~Eric


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Thursday, April 14, 2005 8:43 PM
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] NTDS.dit size

Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20
GB TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers
I
> cited below (with those very descriptive adjectives) are just what
I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or
so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too
as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you
> for
> this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Eric
Fleischman
> Sent: Thursday, April 14, 2005 7:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> I've seen larger.
> I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
> 100GB+ on a few occasions.
> 
> ~Eric
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:28 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> The largest production DIT I have personally seen was on the order of
> 8GB
> for the GC DIT for a Fortune 5 company running about 250k users of
which
> about 180k were Exchange enabled. Also had some 250k contacts, 200k or
> so
> computer objects, 100k or so group objects and consisted of 9 domains.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailt

Re: [ActiveDir] NTDS.dit size

2005-04-14 Thread mike kline
Eric/Joe,

Thanks for the great input!  My test lab is VM ware running on 20
GB TB SAN that you can use as a test = very nice setup.

100 GB did those sites have really good connectivity?  You can install
AD from media in 2003 but I would think there would be problems in a
2000 domain with poorly connected offices.

Joe, do you run joeware.net... if you do great site and thanks for the
nice tools.


Thanks again

Mike

On 4/14/05, Eric Fleischman <[EMAIL PROTECTED]> wrote:
> Well I've seen very very large in test on many occasions. The numbers I
> cited below (with those very descriptive adjectives) are just what I've
> seen in production. I didn't think test counted.
> 
> If you want to count test, I could fire up a test db that is a TB or so
> on a san I have nearby. :)
> 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:58 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> See I almost cc'ed you on the response to get your input on this too as
> I
> knew you had played with some 16GB+ DITS but didn't want to bother you
> for
> this and didn't want to speak out of turn for you.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
> Sent: Thursday, April 14, 2005 7:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> I've seen larger.
> I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
> 100GB+ on a few occasions.
> 
> ~Eric
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of joe
> Sent: Thursday, April 14, 2005 4:28 PM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] NTDS.dit size
> 
> The largest production DIT I have personally seen was on the order of
> 8GB
> for the GC DIT for a Fortune 5 company running about 250k users of which
> about 180k were Exchange enabled. Also had some 250k contacts, 200k or
> so
> computer objects, 100k or so group objects and consisted of 9 domains.
> 
>  joe
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of mike kline
> Sent: Tuesday, April 12, 2005 2:53 PM
> To: ActiveDir@mail.activedir.org
> Subject: [ActiveDir] NTDS.dit size
> 
> I know that AD can have millions of objects, just trying to see what the
> real world size of some your AD databases are.  Do any of you have
> databases
> greater than 20GB+... or more?
> 
> Thanks
> Mike
> List info   : http://www.activedir.org/List.aspx
> List FAQ: http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
> 
> List info   : http://www.activedir.org/List.aspx
> List FAQ: http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info   : http://www.activedir.org/List.aspx
> List FAQ: http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
> 
> List info   : http://www.activedir.org/List.aspx
> List FAQ: http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info   : http://www.activedir.org/List.aspx
> List FAQ: http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/


RE: [ActiveDir] NTDS.dit size

2005-04-14 Thread Eric Fleischman
Well I've seen very very large in test on many occasions. The numbers I
cited below (with those very descriptive adjectives) are just what I've
seen in production. I didn't think test counted.

If you want to count test, I could fire up a test db that is a TB or so
on a san I have nearby. :)


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Thursday, April 14, 2005 4:58 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

See I almost cc'ed you on the response to get your input on this too as
I
knew you had played with some 16GB+ DITS but didn't want to bother you
for
this and didn't want to speak out of turn for you.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: Thursday, April 14, 2005 7:35 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

I've seen larger.
I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
100GB+ on a few occasions.

~Eric



-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Thursday, April 14, 2005 4:28 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

The largest production DIT I have personally seen was on the order of
8GB
for the GC DIT for a Fortune 5 company running about 250k users of which
about 180k were Exchange enabled. Also had some 250k contacts, 200k or
so
computer objects, 100k or so group objects and consisted of 9 domains.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Tuesday, April 12, 2005 2:53 PM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] NTDS.dit size

I know that AD can have millions of objects, just trying to see what the
real world size of some your AD databases are.  Do any of you have
databases
greater than 20GB+... or more?

Thanks
Mike
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/


RE: [ActiveDir] NTDS.dit size

2005-04-14 Thread joe
See I almost cc'ed you on the response to get your input on this too as I
knew you had played with some 16GB+ DITS but didn't want to bother you for
this and didn't want to speak out of turn for you.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman
Sent: Thursday, April 14, 2005 7:35 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

I've seen larger.
I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
100GB+ on a few occasions.

~Eric



-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Thursday, April 14, 2005 4:28 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

The largest production DIT I have personally seen was on the order of 8GB
for the GC DIT for a Fortune 5 company running about 250k users of which
about 180k were Exchange enabled. Also had some 250k contacts, 200k or so
computer objects, 100k or so group objects and consisted of 9 domains.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Tuesday, April 12, 2005 2:53 PM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] NTDS.dit size

I know that AD can have millions of objects, just trying to see what the
real world size of some your AD databases are.  Do any of you have databases
greater than 20GB+... or more?

Thanks
Mike
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/


RE: [ActiveDir] NTDS.dit size

2005-04-14 Thread Eric Fleischman
I've seen larger.
I've seen 15GB+ on MANY occasions, 30GB+ on quite a few occasions, and
100GB+ on a few occasions.

~Eric



-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Thursday, April 14, 2005 4:28 PM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] NTDS.dit size

The largest production DIT I have personally seen was on the order of
8GB
for the GC DIT for a Fortune 5 company running about 250k users of which
about 180k were Exchange enabled. Also had some 250k contacts, 200k or
so
computer objects, 100k or so group objects and consisted of 9 domains.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Tuesday, April 12, 2005 2:53 PM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] NTDS.dit size

I know that AD can have millions of objects, just trying to see what the
real world size of some your AD databases are.  Do any of you have
databases
greater than 20GB+... or more?

Thanks
Mike
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/


RE: [ActiveDir] NTDS.dit size

2005-04-14 Thread joe
The largest production DIT I have personally seen was on the order of 8GB
for the GC DIT for a Fortune 5 company running about 250k users of which
about 180k were Exchange enabled. Also had some 250k contacts, 200k or so
computer objects, 100k or so group objects and consisted of 9 domains.

  joe

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of mike kline
Sent: Tuesday, April 12, 2005 2:53 PM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] NTDS.dit size

I know that AD can have millions of objects, just trying to see what the
real world size of some your AD databases are.  Do any of you have databases
greater than 20GB+... or more?

Thanks
Mike
List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/List.aspx
List FAQ: http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/