RE: Monitoring Software Configuration for Debian server.
Everyone, Thanks for all the great ideas and info. This is part of what makes the debian community so great. :) I will be trying your suggestions shortly. I'll be sure to report on the outcome. Thanks!! -Original Message- From: Alex Malinovich [mailto:[EMAIL PROTECTED] Sent: Saturday, January 19, 2008 5:07 PM To: debian-amd64@lists.debian.org Subject: Re: Monitoring Software Configuration for Debian server. On Sat, 2008-01-19 at 12:56 -0500, [EMAIL PROTECTED] wrote: --snip-- > He tells me that when he investigated he found that one of the drives > had failed silently some time earlier, and the RAID was covering for > the failed drive by using the other. He also told me the SMART stuff > had provided no indication of any trouble. Not all RAID controllers will provide SMART info from the drives, so using SMART status to check your array health is a very bad idea. The only sure way to know when a drive in an array fails is to monitor the array itself via appropriate tools. > Should I ask further about *exactly* what he had set up abd report > back here? (by the way, it wasn't Debian, nor was it and AMD-64, so > this is technically off-topic.) My guess would be that, as I said above, he had some SMART monitoring tools going, but no RAID monitoring tools. He just had log entries from the RAID kernel driver telling him that a drive had failed, and probably wasn't checking his logs regularly enough. (This is where logcheck comes in handy, as Steve mentioned previously.) -- Alex Malinovich Support Free Software, delete your Windows partition TODAY! Encrypted mail preferred. You can get my public key from any of the pgp.net keyservers. Key ID: A6D24837 No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.516 / Virus Database: 269.19.7/1232 - Release Date: 1/18/2008 7:32 PM No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.516 / Virus Database: 269.19.7/1232 - Release Date: 1/18/2008 7:32 PM -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Monitoring Software Configuration for Debian server.
Alex On Sat, 2008-01-19 at 14:02 -0800, Alex Malinovich wrote: > On Sat, 2008-01-19 at 08:45 +, Steve Dobson wrote: > --snip-- > > I use Nagios. When it comes to monitoring remote systems it works very > > well. I wouldn't trust it to monitor itself because if the system fails > > how can it notify you? > --snip-- > > This is where having multiple Nagios servers monitoring each other comes > in handy. :) Agreed. And multiple (independent) reporting methods, Steve -- Steve Dobson There is nothing which cannot be answered by means of my doctrine," said a monk, coming into a teahouse where Nasrudin sat. "And yet just a short time ago, I was challenged by a scholar with an unanswerable question," said Nasrudin. "I could have answered it if I had been there." "Very well. He asked, 'Why are you breaking into my house in the middle of the night?'" signature.asc Description: This is a digitally signed message part
Re: Monitoring Software Configuration for Debian server.
On Sat, 2008-01-19 at 12:56 -0500, [EMAIL PROTECTED] wrote: --snip-- > He tells me that when he investigated he found that one of the drives > had failed silently some time earlier, and the RAID was covering for the > failed drive by using the other. He also told me the SMART stuff had > provided no indication of any trouble. Not all RAID controllers will provide SMART info from the drives, so using SMART status to check your array health is a very bad idea. The only sure way to know when a drive in an array fails is to monitor the array itself via appropriate tools. > Should I ask further about *exactly* what he had set up abd report back > here? (by the way, it wasn't Debian, nor was it and AMD-64, so this is > technically off-topic.) My guess would be that, as I said above, he had some SMART monitoring tools going, but no RAID monitoring tools. He just had log entries from the RAID kernel driver telling him that a drive had failed, and probably wasn't checking his logs regularly enough. (This is where logcheck comes in handy, as Steve mentioned previously.) -- Alex Malinovich Support Free Software, delete your Windows partition TODAY! Encrypted mail preferred. You can get my public key from any of the pgp.net keyservers. Key ID: A6D24837 signature.asc Description: This is a digitally signed message part
Re: Monitoring Software Configuration for Debian server.
On Sat, 2008-01-19 at 08:45 +, Steve Dobson wrote: --snip-- > I use Nagios. When it comes to monitoring remote systems it works very > well. I wouldn't trust it to monitor itself because if the system fails > how can it notify you? --snip-- This is where having multiple Nagios servers monitoring each other comes in handy. :) -- Alex Malinovich Support Free Software, delete your Windows partition TODAY! Encrypted mail preferred. You can get my public key from any of the pgp.net keyservers. Key ID: A6D24837 signature.asc Description: This is a digitally signed message part
Re: Monitoring Software Configuration for Debian server.
On Fri, Jan 18, 2008 at 11:34:43PM -0500, Andrew Syrewicze wrote: > Right, I've got Smartmontools installed and working just fine, as well mdadm > working for my array. > > Can these tools be configured to send alerts to a local mail account??? And > from there be mailed to external mail addresses via something like > sendmail??? mdadm already mails [EMAIL PROTECTED] if there is a failure, which should of course redirect to somebody that is an admin. -- Len Sorensen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Monitoring Software Configuration for Debian server.
On Fri, Jan 18, 2008 at 11:34:43PM -0500, Andrew Syrewicze wrote: > Right, I've got Smartmontools installed and working just fine, as well mdadm > working for my array. > > Can these tools be configured to send alerts to a local mail account??? And > from there be mailed to external mail addresses via something like > sendmail??? > > If so how do I go about configuring smartmontools and mdadm for alerts?? Where does root mail go now? /etc/mdadm.conf has the line MAILADDR root If your mail is set so that you get root's mail, then you will get mdadm mail. I have this little script called drivestatus that I put in /usr/local/sbin: 8>-- #! /bin/sh # PROGRAM /usr/local/sbin/drivestatus # 2007-01-06 Doug Tutty /bin/echo "Runs all drive and array status commands." /bin/echo /bin/echo "/dev/sda:" /usr/sbin/smartctl -a -d ata /dev/sda /bin/echo /bin/echo "/dev/sdb" /usr/sbin/smartctl -a -d ata /dev/sdb /bin/echo /sbin/mdadm -D /dev/md0 /sbin/mdadm -D /dev/md1 8>-- This gives me the current status. I also have this script that cron runs weekly but I can run anytime: 8>-- #! /bin/sh # PROGRAM /usr/local/sbin/drivecheck # 2007-01-06 Doug Tutty # # Runs programs to monitor drive and array status weekly. # Some packages put cron jobs to do this in the middle of the night, # but this computer is turned off overnight. Anacron takes care of this file. /bin/echo "Initiate a long S.M.A.R.T. selftest on /dev/sda," /bin/echo "waits 40 minutes and then on /dev/sdb:" /bin/echo "waits 40 minutes and then checks the md arrays." /bin/echo "Watch progress and results with:" /bin/echo "/usr/sbin/smartctl -a -d ata /dev/sda" /bin/echo "/usr/sbin/smartctl -a -d ata /dev/sdb" /bin/echo "/sbin/mdadm -D /dev/md0" /bin/echo "/sbin/mdadm -D /dev/md1" /bin/echo "Or just use /usr/local/sbin/drivestatus|less" /usr/sbin/smartctl -t long -d ata /dev/sda sleep 40m /usr/sbin/smartctl -t long -d ata /dev/sdb sleep 40m /usr/share/mdadm/checkarray -a 8>-- I hope this helps. Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Monitoring Software Configuration for Debian server.
On Fri, Jan 18, 2008 at 11:24:09PM -0500, Douglas A. Tutty wrote: > On Fri, Jan 18, 2008 at 08:41:40PM -0500, Andy Syrewicze wrote: > > > > I've got a server with no gui. Just CLI and Webmin. > > > > I would like to have some sort of monitoring and reporting service > > that notifys me via e-mail if one of my Raid arrays Degrade and to let > > me know if the SMART check on one of my Disks fail. > > > > I would also like to use a easy to setup/simple to use mail service > > for this. > > smartmontools, madam > > Have it mail to you wherever you like. Where does other root mail end > up? > > Doug. A friend of mine had two drives fail simultaneously in his RAID setup. Since those were the only two, he faces serious data loss. (everything since the last backup, and yes, he did have one of those). He tells me that when he investigated he found that one of the drives had failed silently some time earlier, and the RAID was covering for the failed drive by using the other. He also told me the SMART stuff had provided no indication of any trouble. I find myself wondering just how he found out that his drive had been nonfunctional for some time if he didn't have any diagnostic information -- but this story is enough to make me suspicious of RAID monitoring tools. Should I ask further about *exactly* what he had set up abd report back here? (by the way, it wasn't Debian, nor was it and AMD-64, so this is technically off-topic.) -- hendrik -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Monitoring Software Configuration for Debian server.
Andrew Syrewicze 23:34 Fri 18 Jan If so how do I go about configuring smartmontools and mdadm for alerts?? -Original Message- From: Douglas A. Tutty [mailto:[EMAIL PROTECTED] smartmontools, madam I want to know how you figured out that "madam" meant "mdadm" . SMART looks pretty neat, time to check it out. -- Things are more like they are today than they ever were before. -- Dwight Eisenhower -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Monitoring Software Configuration for Debian server.
Andy On Fri, 2008-01-18 at 20:41 -0500, Andy Syrewicze wrote: > I've got a server with no gui. Just CLI and Webmin. Sounds like a server to me. > I would like to have some sort of monitoring and reporting service > that notifys me via e-mail if one of my Raid arrays Degrade and to let > me know if the SMART check on one of my Disks fail. I would suggest "logcheck" it scans the log files and reports anything that it thinks out of the ordinary. With a bit of config you could probably get it to report just want you're looking for. > I would also like to use a easy to setup/simple to use mail service > for this. Exim4 maybe a heavy weight e-mail system, but the package config can make it very simple to set up your server to forward all received mail on to you main e-mail server. This is how I configure the various systems (workstation and servers) in my networks. > I've looked at Nagios for the Monitoring and it looks like hitting an > ant with a bazooka. I figured there is something not so involved out > there. I use Nagios. When it comes to monitoring remote systems it works very well. I wouldn't trust it to monitor itself because if the system fails how can it notify you? Steve -- Steve Dobson Boomer Envy: Envy of material wealth and long-range material security accrued by older members of the baby boom generation by virtue of fortunate births. -- Douglas Coupland, "Generation X: Tales for an Accelerated Culture" signature.asc Description: This is a digitally signed message part
RE: Monitoring Software Configuration for Debian server.
Right, I've got Smartmontools installed and working just fine, as well mdadm working for my array. Can these tools be configured to send alerts to a local mail account??? And from there be mailed to external mail addresses via something like sendmail??? If so how do I go about configuring smartmontools and mdadm for alerts?? -Original Message- From: Douglas A. Tutty [mailto:[EMAIL PROTECTED] Sent: Friday, January 18, 2008 11:24 PM To: debian-amd64@lists.debian.org Subject: Re: Monitoring Software Configuration for Debian server. On Fri, Jan 18, 2008 at 08:41:40PM -0500, Andy Syrewicze wrote: > > I've got a server with no gui. Just CLI and Webmin. > > I would like to have some sort of monitoring and reporting service > that notifys me via e-mail if one of my Raid arrays Degrade and to let > me know if the SMART check on one of my Disks fail. > > I would also like to use a easy to setup/simple to use mail service > for this. smartmontools, madam Have it mail to you wherever you like. Where does other root mail end up? Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.516 / Virus Database: 269.19.6/1230 - Release Date: 1/17/2008 4:59 PM No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.516 / Virus Database: 269.19.6/1230 - Release Date: 1/17/2008 4:59 PM -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Monitoring Software Configuration for Debian server.
On Fri, Jan 18, 2008 at 08:41:40PM -0500, Andy Syrewicze wrote: > > I've got a server with no gui. Just CLI and Webmin. > > I would like to have some sort of monitoring and reporting service > that notifys me via e-mail if one of my Raid arrays Degrade and to let > me know if the SMART check on one of my Disks fail. > > I would also like to use a easy to setup/simple to use mail service > for this. smartmontools, madam Have it mail to you wherever you like. Where does other root mail end up? Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]