Re: [CentOS] Strange performance issue on CentOS 6.7 server
On Feb 3, 2016, at 3:23 PM, Alfred von Campe wrote: > > On Feb 3, 2016, at 17:10, Warren Young wrote: > >> smartctl can see through several different types of RAID controller to the >> underlying physical disks via its -d option. > > This is what I have: > > # smartctl --all /dev/sda > smartctl 5.43 2012-06-30 r3573 [i686-linux-2.6.32-573.12.1.el6.i686] (local > build) > Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net > > Vendor: IBM > Product: ServeRAID M5110e A bit of Googling says that’s an LSI 2208 based card. So, try smartctl -a -d megaraid,0 If that works, you should be able to walk through each disk by incrementing that trailing number. Then, you can add -t flags to do active tests. ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange performance issue on CentOS 6.7 server
On 3 Feb 2016 22:24, "Alfred von Campe" wrote: > > On Feb 3, 2016, at 17:10, Warren Young wrote: > > > smartctl can see through several different types of RAID controller to the underlying physical disks via its -d option. > > This is what I have: > > # smartctl --all /dev/sda > smartctl 5.43 2012-06-30 r3573 [i686-linux-2.6.32-573.12.1.el6.i686] (local build) > Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net > > Vendor: IBM > Product: ServeRAID M5110e > Revision: 3.19 > User Capacity:1,494,996,746,240 bytes [1.49 TB] > Logical block size: 512 bytes > Logical Unit id: 0x60050760408e81b018be601809efd11c > Serial number:001cd1ef091860be18b0818e40600705 > Device type: disk > Local Time is:Wed Feb 3 17:13:34 2016 EST > Device does not support SMART > > Error Counter logging not supported > Device does not support Self Test logging > > I guess I am stuck since it says it doesn’t support SMART. Or is there some way to get some status from this “disk” to see if it’s really the root cause of my performance issues. I think I would have seen something in /var/log/messages if there was a critical issue. > Severely degraded hardware RAID performance can often be caused by things like a failed cache battery. There is usually some sort of tool to interrogate the device to check things like cache behaviour. ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange performance issue on CentOS 6.7 server
On Feb 3, 2016, at 17:10, Warren Young wrote: > smartctl can see through several different types of RAID controller to the > underlying physical disks via its -d option. This is what I have: # smartctl --all /dev/sda smartctl 5.43 2012-06-30 r3573 [i686-linux-2.6.32-573.12.1.el6.i686] (local build) Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net Vendor: IBM Product: ServeRAID M5110e Revision: 3.19 User Capacity:1,494,996,746,240 bytes [1.49 TB] Logical block size: 512 bytes Logical Unit id: 0x60050760408e81b018be601809efd11c Serial number:001cd1ef091860be18b0818e40600705 Device type: disk Local Time is:Wed Feb 3 17:13:34 2016 EST Device does not support SMART Error Counter logging not supported Device does not support Self Test logging I guess I am stuck since it says it doesn’t support SMART. Or is there some way to get some status from this “disk” to see if it’s really the root cause of my performance issues. I think I would have seen something in /var/log/messages if there was a critical issue. Alfred ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange performance issue on CentOS 6.7 server
On Feb 3, 2016, at 2:26 PM, Alfred von Campe wrote: > > On Feb 3, 2016, at 16:13, Warren Young wrote: > >> A dying hard disk can do it. HDDs try to silently paper over I/O errors, >> but what they can’t hide is the time it takes to do this. If your HDD is >> constantly correcting errors at the oxide layer, it will be reallly >> sow. >> >> You can try running SMART tests on it, though that’s not guaranteed to show >> the problem. > > Well, it’s not “a” disk: it’s a HW RAID of about dozen (server grade) drives smartctl can see through several different types of RAID controller to the underlying physical disks via its -d option. ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange performance issue on CentOS 6.7 server
Alfred von Campe wrote: > On Feb 3, 2016, at 16:13, Warren Young wrote: > >> A dying hard disk can do it. HDDs try to silently paper over I/O >> errors, but what they can’t hide is the time it takes to do this. If >> your HDD is constantly correcting errors at the oxide layer, it will be >> reallly sow. >> >> You can try running SMART tests on it, though that’s not guaranteed to >> show the problem. > > Well, it’s not “a” disk: it’s a HW RAID of about dozen (server grade) > drives, with a VG/LV on top of that. Are there any log files I can check > that test the underlying VG/LV health status? > You don't mention what kind of h/w RAID. LSI-based controller, and HP-based ones, both have utilities to check out the drives (MegaRAID and hpacli, respectively); AC&NC JetStors have a web interface. mark ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange performance issue on CentOS 6.7 server
On Feb 3, 2016, at 16:13, Warren Young wrote: > A dying hard disk can do it. HDDs try to silently paper over I/O errors, but > what they can’t hide is the time it takes to do this. If your HDD is > constantly correcting errors at the oxide layer, it will be reallly > sow. > > You can try running SMART tests on it, though that’s not guaranteed to show > the problem. Well, it’s not “a” disk: it’s a HW RAID of about dozen (server grade) drives, with a VG/LV on top of that. Are there any log files I can check that test the underlying VG/LV health status? Alfred ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange performance issue on CentOS 6.7 server
On Feb 3, 2016, at 1:30 PM, Alfred von Campe wrote: > > I suspected that file I/O might be slow, and sure enough, that appears to be > the case….What could cause this A dying hard disk can do it. HDDs try to silently paper over I/O errors, but what they can’t hide is the time it takes to do this. If your HDD is constantly correcting errors at the oxide layer, it will be reallly sow. You can try running SMART tests on it, though that’s not guaranteed to show the problem. Got tested backups? :) ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
