Lars Roland wrote:
Maybe this needs to be configurable - i.e. in some situations it might be nice to just pull from the LDAP server periodically (i.e. pull the entire user list once every hour, once every day or whatever floats your boat) - other people may need to perform a LDAP look up for each email - so as I see it, this should preferably be configurable.
I could see using some sort of cache (perhaps memcached?) and storing users/addresses that have been queried. That would be cool and would be very clean. If performance is the ultimate goal, and in-memory cache would be ideal (but poorly suited to forkserver, and would be bogus with tcpserver-based implementations.)
Dumping your Directory periodically is a scaling nightmare, and Directories are usually (hopefully) not tuned for that. Hopefully, there are checks in place to prevent too many entries from being returned from a query. If a query is returning thousands of entries, an LDAP Directory is not the best solution for that task.
The old Netscape/iPlanet/Sun Messaging server did exactly what you say (periodic dump of the Directory), but when you get into multi-millions of accounts (or even a few thousand accounts, depending on what's going on, the sizing/tuning of the Directory, the day of the week, etc.), you run into issues of the transaction taking a VERY long time. If you weren't careful, the time that it took to dump the Directory would (at times) be longer than the interval at which it dumped it.
Stale data would be another problem associated with dumping your Directory. If you disable a mail account, or if you change account properties (upping the quota limit), the changes wouldn't take effect until the next dump. It depends on the environment and requirements, of course.
Directories are usually tuned (in my experience, of course) so that if you know what you're looking for, you can find it quickly and be done with it. Limits are placed on how long a query can take, how many entries may be returned, etc. All of this in an effort to make short, fast queries as speedy as possible.
Personally I am all for the periodic pull of valid emails (I have implemented this for exchange 2000, 2003 and lotus notes servers) since this ensures email flow even if the LDAP server is down.
Well, if you're working in a small environment with an unstable Directory, I could see this being useful, but why not just add another LDAP server? Perhaps one locally stored on your mail server? Adding code to try multiple LDAP servers (on failure, or load balanced/round robin'ed) would be a lot cleaner and simpler. Ideally you'd put a load balancer in front of your Directories (LVS, perhaps?) That's the architecture I would be familiar with (multiple directories, load balanced), rather than dumping your entire Directory.
Tying qpsmtpd in with memcached or some in-memory cache would be the best of both worlds, though. An appropriate TTL of the cache would be good to prevent stale data, though.
