On Fri, Nov 24, 2000 at 03:55:33PM -0300, Federico Edelman Anaya wrote:
Brian:
Hi!, the total of subscriber of all my list is 2.000.000, now, i am running a
list-server:
- Qmail 1.03 (with big-concurrency.patch and big-todo.patch)
- daemontools
- ezmlm-0.53 + ezmlm-idx-0.40
Hardware and SO:
- Linux Debian (Dual Petium II) RAM 256 MB
- Kernel 2.2.17 with reiserfs
- HD 8 GB
- And a roundrobin DNS (with 4 relay host)
The delivery is very slowly and the kernel not support fine a
big-concurrency.patch (and NR_TASKS modifcations)
Well, I am thinking to change de Hardware. Linux? Solaris (Sun)? how many
space i need for 100 lists and 2.000.000 subscribers (total)?
The messages's size: 50 KB
If you're using ezmlm to manage these mailing lists, then I presume
that the answer to this question, which you didn't answer:
For any one newletter, will all of it's recipents get the same
newsletter? if not, then that means each newletter will have to
have 2 million copies if itself in the mail queue.
is 'yes', in that all recipients get the same newsletter.
That means that you only have 100 copies of the 50k newsletter in
your message queue. That's only about 5M of diskspace. No problem
there.
I don't have a quick handle on how much diskspace any one instance
of a mailing list takes, though, WRT the list of subscribers. I
have one fairly tiny list I run (~100 people); based on what it
takes up, I'm going to odd-handedly guess that 2 million subscribers
(over all lists) will cost you maybe 40M of diskspace.
Are you maintaining indexes of these lists? (The ezmlm-idx patch
implies that). That will open-endedly eat diskspace forever. Each
list's mailing will cause another 50k message to be saved. How
long are you going to keep messages?
How long are you going to keep mail logs for deliveries? They can
possibly eat your diskspace if kept forever. You'll want to keep
some, so you can periodically analyze how well the machine can
deliver mail.
I'd hazard, roughly, that this 8G hard drive will have the capacity
to hold you down for many months.
But, in my mail to you, I mentioned network I/O, and disk I/O (not
capacity).
Network I/O:
You want 2 million deliveries of a 50K file a day. That's 100G a day;
that's 1.1M a _second_ of network activity, sustained 24x7.
And that's _just_ the newsletters. That does not take into account:
- DNS traffic with all of the timeouts and that that implies
- SMTP traffic. Assume 300 bytes per delivery: that's 600M _a
day_ just talking SMTP.
- managing bounces
So - what's your ISP hookup like?
Disk I/O:
Qmail is notoriously disk-heavy. I have no experience with the
ReiserFS, so I don't know how much of a gain you'll get.
A 'classic' trick with qmail is to run several instances of qmail
with separate queues, preferably on separate spindles. That
way, one sluggish mailing list won't bog down the others. That
implies multiple hard drives.
You mention 'a roundrobin DNS (with 4 relay host)'. I'm not certain
I understand how you have those set up...
Again, people have been discussing such stuff on the qmail/ezmlm
lists for some time, feel free to puruse the mailing list archives...
--
Brian 'you Bastard' Reichert[EMAIL PROTECTED]
37 Crystal Ave. #303Daytime number: (603) 434-6842
Derry NH 03038-1713 USA Intel architecture: the left-hand path
