Re[2]: High volume mail handling architecture
Hello Maykel,
Thursday, September 9, 2004, 15:59:02, you wrote:
MM> Could you send your regexes, 90% of viruses stopped by regexes sounds
MM> interesting.
In fact most of them are stopped by general regexps rejecting some
dangerous attachments, so only those with .exe and .zip (which I can't
block because my users need it) have chance to come in... It also
stops attachments with multiple extensions (for example .doc.pif). I
know it can potentially stop some legitimate mail, but the sender will
know the reason for being rejected (if they will read it) so they can
change it. I have seen some people sending .doc.lnk instead of word
document :)
I hope the attachments are allowed here...
add this to your /etc/postfix/main.cf:
body_checks = pcre:/etc/postfix/checks-extensions,
pcre:/etc/postfix/checks-virus
--
Best regards,
.---..---
/ \ __ /--Marek 'Marki' Podmaka
/ / \( )/-
// ' \/ ` --- e-mail: [EMAIL PROTECTED] (preferred)
/ // :: --- [EMAIL PROTECTED]
// / / /`'--ICQ UIN: 42698938
// //..\\ mobile phone: +421 903 259949
UUUU---
'//||\\` "Love makes the world go round." (Proverb)
''``
... Hovoril jej, ze je dievca jeho snov. Nepovedal, ze hrozostrasnych.
checks-virus.
Description: Binary data
checks-attachments.
Description: Binary data
Re: High volume mail handling architecture
On Thu, 9 Sep 2004 18:44, Marcin Owsiany <[EMAIL PROTECTED]> wrote: > On Thu, Sep 09, 2004 at 06:03:20AM +1000, Russell Coker wrote: > > You have to either be doing something very intensive or very wrong to > > need more than one server for 20K users. Last time I did this I got 250K > > users per server, and I believe that I could have easily doubled that if > > I was allowed to choose the hardware. > > We have a little over 10K users, and the disk subsystem seems to be the > bottleneck. When we reach about 600 read transactions + 150 write > transactions per second (as reported by sar -b), the load average starts > to grow expotentially instead of proportionally. There are about 20K > sectors read, and 3K written per second. (That was before I turned noatime > on. After that we had about 2K sector writes and 70 write transactions > less, and load average dropped to a more sane value - about 3, instead > of 20.) Last time I was doing this I had some Dell 2U servers (2650 from memory) with 4 * 10K U160 disks in a RAID-5 (5th disk was hot-spare) and something like 4G of RAM. The machines had almost no read access to the drives, something less than 10% of disk access was for read because the cache worked really well (the accounts that receive the most mail are the ones that have clients checking them most often - in some cases people leave their email client on 24*7 checking every 5 mins). The write bottleneck was just under 3M/s, I don't recall how many transactions that was. To give better performance you may want to look at getting more RAM. RAM is cheap and you can eliminate most read bottlenecks by caching lots of stuff. 3K sectors written per second isn't too good, but I guess that's because of the 20K sectors read. Get some more cache and things should improve a lot. Also if using a typical Unix mail server (Postfix, Sendmail, etc) then the data is written synchronously somewhere under /var before being read from there and written to the destination. If you use a NVRAM card from UMEM http://www.umem.com/ for /var/spool then you could possibly double mail delivery performance. If you use data=journal and put the journal for the mail store file system on the umem device you could probably double performance again. > Also, did you implement virus/spam scanning on that box? No! Virus/spam scanning was on the front-end machines. It was believed that the mail store machines were busy enough with doing the most basic work without virus scanning (also the number of licences for the anti-virus program didn't match the number of store machines that were planned). You want to do as much work away from the mail store as possible. Mail store machines can not be replaced without major inconvenience to everyone (customers, staff, management). Front-end anti-virus machines are disposable, if you have a traffic balancing device (such as a Cisco LocalDirector or IPVS) in front of a cluster of anti-virus machines then an anti-virus machine can go down for a few days without anyone bothering. If (hypothetically) anti-virus was to take 10% of the performance from a mail store then it could require another mail store machine (if you have 5-10 machines) and therefore that's one more machine which can break and cause massive pain to everyone. Another thing, a mail store machine should require almost no CPU power. Give it a single CPU that's not the fastest available. It sucks when you have two almost unused CPUs which are both fast and hot and then one breaks down killing the machine. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: RAID 1 HELP!!!
Dear Lucas, I tried with linear and my debian just boot! I'm sending to you my lilo.conf file (in case it seems to you useful for your how-to with LILO) after making some changes that "lilo-doc" recommended for raid booting. As I thought it couldn't be the only problem... Could you take a look to the following? Note: Not all the devices are synchronized at this time. Sorry about this (It takes 7 hours each time I have to do it again...). Perhaps I just have to wait??? Thanks for everything!!! Agustín maria:/# mdadm -D /dev/md1 /dev/md1: Version : 00.90.00 Creation Time : Wed Sep 8 22:31:30 2004 Raid Level : raid1 Array Size : 14464 (14.12 MiB 14.81 MB) Device Size : 14464 (14.12 MiB 14.81 MB) Raid Disks : 2 Total Disks : 3 Preferred Minor : 1 Persistance : Superblock is persistant Update Time : Wed Sep 8 23:03:53 2004 State : dirty, no-errors Active Drives : 2 Working Drives : 2 * Failed Drives : 1 * Spare Drives : 0 Number Major Minor RaidDisk State 0 310 active sync /dev/hda1 1 2211 active sync /dev/hdc1 UUID : 2bf99542:45c208ee:df7011a5:0e8b602f maria:/# mdadm -E /dev/md1 ** mdadm: No super block found on /dev/md1 (Expected magic a92b4efc, got ) ** maria:/# cat /proc/mdstat Personalities : [raid1] read_ahead 1024 sectors md1 : active raid1 hda1[0] hdc1[1] 14464 blocks [2/2] [UU] - Original Message - From: "Lucas Albers" <[EMAIL PROTECTED]> To: "Agustín Ciciliani" <[EMAIL PROTECTED]> Sent: Wednesday, September 08, 2004 5:57 PM Subject: Re: RAID 1 HELP!!! > Try using grub or run lilo with lba or linear mode. > My guess is that it is not seeing the sata drives correctly. > Try running a newer kernel that has better sata support. > > Agustín Ciciliani said: > > Dear Lucas, > > > > I hope not to bother with my question. I've been reading your excellent > > how-to: > > > > Version 0.97 (2004-06-03) Lucas Albers -- admin At cs DOT montana dot edu > > and Roger > > Chrisman > > Home of most recent version: http://alioth.debian.org/projects/rootraiddoc > > > > I really think that it is the best raid 1 how-to in the web. > > > > Everything works fine, exactly as you describe it (I've read EVERYTHING > > three times) until > > the step 6.3. I make the 6.4 and when I reboot, my system hangs saying > > "LIL" > > I've read something in the web as: > > LIL (The descriptor table could not be read.) > > LIL (This is typically caused by media failure or geometry mismatch) > > > > But nothing of these really helped me and that is why I'm contacting you. > > I must say that > > I have two SATA WD800 disks. I'm running debian with the kernel 2.4.19 and > > LILO 22.2. > > > > I've made the process three times, and always the same, so I think I'm not > > missing any > > step... > > > > If you have a minute I would be grateful if you could give me an advice. > > > > I loof forward to hearing from you. > > > > Sincerely, > > > > Agustín Ciciliani > > > > > -- > --Luke CS Sysadmin, Montana State University-Bozeman > > > linear boot=/dev/md1 raid-extra-boot=/dev/hda,/dev/hdc disk=/dev/hda bios=0x80 disk=/dev/hdc bios=0x81 root=/dev/md2 read-only install=/boot/boot-menu.b map=/boot/map delay=20 vga=normal default=RAID image=/boot/vmlinuz-2.4.19 label=RAID
Re: High volume mail handling architecture
On Thu, 9 Sep 2004 15:32:04 +0200, Marek wrote in message <[EMAIL PROTECTED]>: > Citát "Ruth A. Kramer" <[EMAIL PROTECTED]>: > > > Adrian 'Dagurashibanipal' von Bidder wrote: > > > On behalf of all joe-job victims: Whatever you do, *please* do it > > > in a way that allows you to know whether mail is going to be > > > delivered at the front-end incoming SMTP server. (should be > > > trivial if your user database > > is > > > in LDAP or some SQL db or whatever.) > > > > Is the point of the statement above that all mail must be delivered > > via the SMTP server, and then features built into it (disabling of > > anonymous relaying??) will prevent joe-jobs? > > I think the point is in rejecting most of these email as soon as > possible. For this to work, the front-end SMTP server has to know your > users. If it doesn't, you accept these mails for further processing - > spam & virus filtering, which are CPU consuming and just after it your > server realizes that there is no recipient for it. > For the same reason I have some regexp patterns build into postfix > body_checks for most common viruses. Postfix rejects these mails > immediately. This usually catch about 90% of viruses, so I save a lot > of CPU in virus checking of incoming mail... ..my understanding is the point is spot spam bots asap, and either deliver to /dev/null or fry them off the net. Usually, these boxes are paid for some unsuspecting Joe Sixpack, who should have his box rebooting exactly as far as to show:" Your box has been used for criminal purposes, and has been shut down to secure evidence for law enforcement. Please note that tampering with this crime scene evidence, by trying to "repair" or "reinstall the OS" etc, is a crime in itself and you will not want to try do that, as we have reported to your local law enforcemen every box we shut down, to facilitate said law enforcement. If you need your box for any lawful purpose, please feel free to contact your local law enforcement to expedite securing the crime scene evidence and make your box legally available to yourself." ..and _not_ _any_ _bit_ further. A "simple" replacement of the boot loader and possibly disabling the bios. I think it is legal too. ..disclaimer; I don't do (yet) mail servers, I got drgged into doing wifi bandwith trottling, my expertize is in thermochemical gasification. -- ..med vennlig hilsen = with Kind Regards from Arnt... ;-) ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case.
Re: High volume mail handling architecture
> For the same reason I have some regexp patterns build into postfix body_checks > for most common viruses. Postfix rejects these mails immediately. This usually > catch about 90% of viruses, so I save a lot of CPU in virus checking of > incoming mail... Could you send your regexes, 90% of viruses stopped by regexes sounds interesting. Regards mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: High volume mail handling architecture
Citát "Ruth A. Kramer" <[EMAIL PROTECTED]>: > Adrian 'Dagurashibanipal' von Bidder wrote: > > On behalf of all joe-job victims: Whatever you do, *please* do it in a way > > that allows you to know whether mail is going to be delivered at the > > front-end incoming SMTP server. (should be trivial if your user database > is > > in LDAP or some SQL db or whatever.) > > Is the point of the statement above that all mail must be delivered via > the SMTP server, and then features built into it (disabling of anonymous > relaying??) will prevent joe-jobs? I think the point is in rejecting most of these email as soon as possible. For this to work, the front-end SMTP server has to know your users. If it doesn't, you accept these mails for further processing - spam & virus filtering, which are CPU consuming and just after it your server realizes that there is no recipient for it. For the same reason I have some regexp patterns build into postfix body_checks for most common viruses. Postfix rejects these mails immediately. This usually catch about 90% of viruses, so I save a lot of CPU in virus checking of incoming mail... -- bYE, Marki This message was sent using IMP, the Internet Messaging Program. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: High volume mail handling architecture
Adrian 'Dagurashibanipal' von Bidder wrote: > On behalf of all joe-job victims: Whatever you do, *please* do it in a way > that allows you to know whether mail is going to be delivered at the > front-end incoming SMTP server. (should be trivial if your user database is > in LDAP or some SQL db or whatever.) On behalf of the lurkers here who are not experienced admins (am I the only one?), could someone elaborate a little more on the above? What I think I know is that a joe job is when somebody gets mail that looks like it came from, for example me, but it really didn't. Is the point of the statement above that all mail must be delivered via the SMTP server, and then features built into it (disabling of anonymous relaying??) will prevent joe-jobs? thanks, Randy Kramer -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: High volume mail handling architecture
On Thu, Sep 09, 2004 at 06:43:21AM -0600, Nate Duehr wrote: > > On Sep 9, 2004, at 2:44 AM, Marcin Owsiany wrote: > > > >More than 90% of the disk transactions are on the (logical) disk where > >mail is stored. The only processes which touch that disk, are qmail > >delivery processes (qmail handed mail by another SMTP-IN box: 0.8 local > >deliveries per second) and courierpop3d processes (7.2 logins per > >second). > > > > Start splitting the user directories across logical disks that are on > different platters, for goodness sake. Well, adding more disks to the setup is what I planned to do next. I just want to make sure that the performance I get from the _current_ setup is normal. Marcin -- Marcin Owsiany <[EMAIL PROTECTED]> http://marcin.owsiany.pl/ GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: High volume mail handling architecture
On Sep 9, 2004, at 2:44 AM, Marcin Owsiany wrote: More than 90% of the disk transactions are on the (logical) disk where mail is stored. The only processes which touch that disk, are qmail delivery processes (qmail handed mail by another SMTP-IN box: 0.8 local deliveries per second) and courierpop3d processes (7.2 logins per second). Start splitting the user directories across logical disks that are on different platters, for goodness sake. Mount points overlaid below the primary mount point by directory can easily do this for you. -- Nate Duehr, [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: High volume mail handling architecture
Hi Marcin, How many files do you have in a single directory? > 100 ? Which filesystem are you using? You may want to try playimg with reiserfs... Regards Andrew Marcin Owsiany wrote: On Thu, Sep 09, 2004 at 06:03:20AM +1000, Russell Coker wrote: You have to either be doing something very intensive or very wrong to need more than one server for 20K users. Last time I did this I got 250K users per server, and I believe that I could have easily doubled that if I was allowed to choose the hardware. We have a little over 10K users, and the disk subsystem seems to be the bottleneck. When we reach about 600 read transactions + 150 write transactions per second (as reported by sar -b), the load average starts to grow expotentially instead of proportionally. There are about 20K sectors read, and 3K written per second. (That was before I turned noatime on. After that we had about 2K sector writes and 70 write transactions less, and load average dropped to a more sane value - about 3, instead of 20.) More than 90% of the disk transactions are on the (logical) disk where mail is stored. The only processes which touch that disk, are qmail delivery processes (qmail handed mail by another SMTP-IN box: 0.8 local deliveries per second) and courierpop3d processes (7.2 logins per second). We are using an "Intel SRCU42X" SCSI RAID controller, and the logical disk which caries mail is made of 3 Fujitsu 36GB 15K RPM disks. Please tell me, what problem we are facing? Is the hardware so weak? Is it underperforming? Or maybe our load is exceptionally high? I can provide more statistics if they are needed. Also, did you implement virus/spam scanning on that box? kind regards, Marcin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: ..Debian rejects Microsoft's "Sender ID" on contractual terms
On Wed, Sep 08, 2004 at 09:26:53PM +0200, Arnt Karlsen wrote: > On Thu, 9 Sep 2004 02:01:28 +1000, Russell wrote in message > <[EMAIL PROTECTED]>: > > > http://www.nwfusion.com/news/2004/0907opensourc.html?net > > ..hear, hear. But you guys let the weenies get away with confusing > their "end user license agreement's" with licenses. Booo. > > ..any contract has legally binding terms, to both parties. So we should > refer to any such EULA terms, as "EULA contractual terms". > > ..note this fundamental difference between these contracts and the > GPL. Microsoft may use, study, reverse engineer etc GNU/Linux as > they damned please, because that and any other _use_ of it is allowed > under the GPL. To distribute, they will have to either abide by the GPL, > or, defeat it. > > ..these EULA's are contracts; _because_ you have to agree on what; > their terms, to use their property. That very agreement forms a legally > binding _contract_. > > ..we know that, but Joe Sixpack doesn't. Because we fail to educate > the public telling the truth that includes those 2 "extra" words. > > ..http://www.groklaw.net/article.php?story=20040905212754195 > and, to get on-topic everywhere ;-) : Is there a smtp rejection > code for "I accept no shit from Wintendo"mail" boxes!" ? 552 I accept no shit from Wintendo 'mail' boxes As simple as that. -- EARTH smog | bricks AIR -- mud -- FIRE soda water | tequila WATER -- with thanks to fortune -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: High volume mail handling architecture
On Thu, Sep 09, 2004 at 06:03:20AM +1000, Russell Coker wrote: > You have to either be doing something very intensive or very wrong to need > more than one server for 20K users. Last time I did this I got 250K users > per server, and I believe that I could have easily doubled that if I was > allowed to choose the hardware. We have a little over 10K users, and the disk subsystem seems to be the bottleneck. When we reach about 600 read transactions + 150 write transactions per second (as reported by sar -b), the load average starts to grow expotentially instead of proportionally. There are about 20K sectors read, and 3K written per second. (That was before I turned noatime on. After that we had about 2K sector writes and 70 write transactions less, and load average dropped to a more sane value - about 3, instead of 20.) More than 90% of the disk transactions are on the (logical) disk where mail is stored. The only processes which touch that disk, are qmail delivery processes (qmail handed mail by another SMTP-IN box: 0.8 local deliveries per second) and courierpop3d processes (7.2 logins per second). We are using an "Intel SRCU42X" SCSI RAID controller, and the logical disk which caries mail is made of 3 Fujitsu 36GB 15K RPM disks. Please tell me, what problem we are facing? Is the hardware so weak? Is it underperforming? Or maybe our load is exceptionally high? I can provide more statistics if they are needed. Also, did you implement virus/spam scanning on that box? kind regards, Marcin -- Marcin Owsiany <[EMAIL PROTECTED]> http://marcin.owsiany.pl/ GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
