> > Could you described into more detail your config and share some > comments on it? Looks like its unique.
How much detail do you want? Nothing special, there was just a chance to do something right (or perhaps more sane). Not that there is anything wrong with the legacy system, but keeping users in passwd files, rsyncing configs out to cluster servers and such setups, do not scale well after a certain point. The old system uses NFS with NetApps, with quotas. Each brand is mounted as a volume, for your usual ISP hosting, (email, pop, imap, web, cgi, ftp, dns). Sun's Jonathan sent out the blog about wanting to prove they are worth looking at, and promised a free-trial (and to their credit, usually these things are only for the US, we didn't think we'd get to trial a x4500 in Japan, but 2 weeks later we had a server). So why not try it. But to do quotas, we found we really had to use zvolumes, with UFS formatted on it. zfs and quotas, and automounter, or mirror mounts, just do not work (YET!) Not that UFS is without issues. Volumes at 999GB is fine, at 1TB and you get that insane inode problem, where everyone recommends compiling your own mkfs. No time for that right now. We made a NetApp vs Sun shootout table, and made the decision to keep trying Sun. (Since we don't need to upgrade current NetApps until Jun, we have a chance to run the Sun live until then) So, all clustered, no local configuration changes when adding accounts, or domains. Provisioning is always running, so account creations are under a second. Same with all other changes, except buying new domains. The registrars are still slow. That means the provisioning pulls out the requests from DB, creates/changes LDAP for the account data, and creates/changes NFS directories only. Email: postfix, dovecot, squirrelmail. Just LDAP provisioning, no local config changes needed. Apache: double-hash the request, if the directory exists, serve it. No httpd.conf changes needed. CGI: Slight patch to suexec to get uid/gid from users directory and execute. (with extra sanity checks of course). No httpd.conf changes needed. FTP: pure-ftp with ldap, no local conf changes needed. DNS: bind with DLZ, using BDBHPT, update is immediate, and no restarts needed. Radius: FreeRadius, LDAP, (almost) no local config changes needed. It was a mad race to go migrate the first batch of users, but it went rather well. We did have some issues for sure. UFS default maximum number of quota nodes is something low like 2000. Fix and reboot. OpenLDAP replication is randomly losing data, fixed. The biggest problem was essentially MySQL Cluster. It is just not quite ready. It runs, but only because I taped it up. The giant quota file isn't all that interesting in the end. It is sparse, as one of the developers inserted into the provisioning table: +email|[EMAIL PROTECTED]|pass=test|uid=90000001|gid=2000 quota|[EMAIL PROTECTED]|size=50 Sigh. I also want to make it easy for customers to get apps on CGI installed. Tick the box for Gallery, and it is rolled out in their home directory. (For free, since ISP model is generally disk space, and network traffic). V2, or V3 maybe we want to also offer Zones, so that provisioning should be fun. Bet that was more than you wanted to know :) Lund -- Jorgen Lundman | <[EMAIL PROTECTED]> Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell) Japan | +81 (0)3 -3375-1767 (home) _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
