Kiggs, Pole sana. Now here's what you must do as a matter of urgency. "reboot server, lock down postfix config file to prevent spamming. go home. Drink whiskey." Timo.
-----Original Message----- From: Kiggundu Mukasa [mailto:[EMAIL PROTECTED] Sent: 17 July 2003 14:06 To: LUG Subject: lug_: tale of a linux admin Day 1. Magambo sends cryptic note which he later decrypts to say one of the servers I am looking after is relaying spam Day 1 Me thinking how serious could it be as the server has been up less than one month ignore it. Day2 Drive to customer saying need to check on server "routine maintainance". Check mail log, see spamming evidence. Type mailq. SHOCK of LIFETIME!!! for five minutes mail scrolls non-stop. Hit Ctrl C. HOW BIG IS PROBLEM? shut down postfix. Take deep breath.... run few tests. Find out there are 987,203 mail messages in queue!!!!! Server is still handling (Dual Xeon 2.4 GHZ 512 MB of RAM) load average only 3. I wonder why (postfix sleeping .. thats why)? Some of the messages are in various folders so need to find folder and remove. write script grep -r @ * | awk -F: '{print "rm -v", $1}' | uniq > temp ; chmod u+x temp ; ./temp ; rm temp script runs for 20 mins and ends. Remove bounced mail. Thinking I am done, restart postfix. BIG mistake! only removed 200,000 or so messages!!! 45 minutes later and postfix has still not restarted, load average 50+ and rising. No commands responding, Pull plug! Hey since I got reiserfs, it will be there when I get back! Problem is starting machine will start postfix and then I am back to square 1. At boot, type "linux single" to boot into single user mode. Then once there type "telinit 2" to move up one level. So far so good, no starting postfix. Go to each and every postfix directory, and run script above (and modified version) and after 3 hours of constant running, mail is down to about 15 mails I can see in mailq but I cannot find on server. reboot server, lock down postfix config file to prevent spamming. go home. Drink whiskey. Day three ....... ssh to server to check on work of yesterday .................... 112,513 messges!!!!!!!!!!!!! AAAAAAAAAAAAAHHHHHHHHHHHHHHHHHHHHHH TO BE CONTINUED.... -- **************** ***************************** Kiggundu Mukasa # Computer Network Consultancy### KYM-NET LTD. # Intranets & Internet Solutions# House 73 # Data Communication Service #### Plot 80 kanjokya Street P.O. Box 173 Kampala, Uganda Tel: +256 77 972255 +256 71 221141 Fax: +256 31 262122 *************************************************************************
