Hello,
I've provisioned an LVM RAID 6 across 4 physical disks. I'm trying to
understand the RAID behavior after injecting the failure - removing
physical disk /dev/sdc.
pvcreate /dev/sdc /dev/sdd /dev/sde /dev/sdf
vgcreate pool_vg /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg
lvcreate -l +100%FREE -n pool_lv --type raid6 pool_vg
mkfs.xfs /dev/pool_vg/pool_lv
echo "/dev/mapper/pool_vg-pool_lv /mnt xfs
defaults,x-systemd.mount-timeout=30 0 0" >> /etc/fstab
Everything appears to be working fine:
# pvs --segments -o
pv_name,pv_size,seg_size,vg_name,lv_name,lv_attr,lv_size,seg_pe_ranges
PV PSize SSize VG LV Attr LSize
PE Ranges
/dev/sda3 <49.00g <24.50g ubuntu-vg ubuntu-lv -wi-ao----
<24.50g /dev/sda3:0-6270
/dev/sda3 <49.00g 24.50g ubuntu-vg 0
/dev/sdc <100.00g 4.00m pool_vg [pool_lv_rmeta_0] ewi-aor---
4.00m /dev/sdc:0-0
/dev/sdc <100.00g 99.99g pool_vg [pool_lv_rimage_0] iwi-aor---
99.99g /dev/sdc:1-25598
/dev/sdd <100.00g 4.00m pool_vg [pool_lv_rmeta_1] ewi-aor---
4.00m /dev/sdd:0-0
/dev/sdd <100.00g 99.99g pool_vg [pool_lv_rimage_1] iwi-aor---
99.99g /dev/sdd:1-25598
/dev/sde <100.00g 4.00m pool_vg [pool_lv_rmeta_2] ewi-aor---
4.00m /dev/sde:0-0
/dev/sde <100.00g 99.99g pool_vg [pool_lv_rimage_2] iwi-aor---
99.99g /dev/sde:1-25598
/dev/sdf <100.00g 4.00m pool_vg [pool_lv_rmeta_3] ewi-aor---
4.00m /dev/sdf:0-0
/dev/sdf <100.00g 99.99g pool_vg [pool_lv_rimage_3] iwi-aor---
99.99g /dev/sdf:1-25598
/dev/sdg <100.00g 4.00m pool_vg [pool_lv_rmeta_4] ewi-aor---
4.00m /dev/sdg:0-0
/dev/sdg <100.00g 99.99g pool_vg [pool_lv_rimage_4] iwi-aor---
99.99g /dev/sdg:1-25598
# lvs -a -o name,lv_attr,copy_percent,health_status,devices pool_vg
LV Attr Cpy%Sync Health Devices
pool_lv rwi-aor--- 100.00
pool_lv_rimage_0(0),pool_lv_rimage_1(0),pool_lv_rimage_2(0),pool_lv_rimage_3(0),pool_lv_rimage_4(0)
[pool_lv_rimage_0] iwi-aor--- /dev/sdc(1)
[pool_lv_rimage_1] iwi-aor--- /dev/sdd(1)
[pool_lv_rimage_2] iwi-aor--- /dev/sde(1)
[pool_lv_rimage_3] iwi-aor--- /dev/sdf(1)
[pool_lv_rimage_4] iwi-aor--- /dev/sdg(1)
[pool_lv_rmeta_0] ewi-aor--- /dev/sdc(0)
[pool_lv_rmeta_1] ewi-aor--- /dev/sdd(0)
[pool_lv_rmeta_2] ewi-aor--- /dev/sde(0)
[pool_lv_rmeta_3] ewi-aor--- /dev/sdf(0)
[pool_lv_rmeta_4] ewi-aor--- /dev/sdg(0)
After the /dev/sdc is removed and the system is rebooted, the RAID goes
into "partial" health state and is no longer accessible.
# lvs -a -o name,lv_attr,copy_percent,health_status,devices pool_vg
WARNING: Couldn't find device with uuid
03KtEG-cJ5S-cMAD-RlL8-yBXM-jCav-EyD9I3.
WARNING: VG pool_vg is missing PV 03KtEG-cJ5S-cMAD-RlL8-yBXM-jCav-EyD9I3
(last written to /dev/sdc).
LV Attr Cpy%Sync Health Devices
pool_lv rwi---r-p- partial
pool_lv_rimage_0(0),pool_lv_rimage_1(0),pool_lv_rimage_2(0),pool_lv_rimage_3(0),pool_lv_rimage_4(0)
[pool_lv_rimage_0] Iwi---r-p- partial [unknown](1)
[pool_lv_rimage_1] Iwi---r--- /dev/sdd(1)
[pool_lv_rimage_2] Iwi---r--- /dev/sde(1)
[pool_lv_rimage_3] Iwi---r--- /dev/sdf(1)
[pool_lv_rimage_4] Iwi---r--- /dev/sdg(1)
[pool_lv_rmeta_0] ewi---r-p- partial [unknown](0)
[pool_lv_rmeta_1] ewi---r--- /dev/sdd(0)
[pool_lv_rmeta_2] ewi---r--- /dev/sde(0)
[pool_lv_rmeta_3] ewi---r--- /dev/sdf(0)
[pool_lv_rmeta_4] ewi---r--- /dev/sdg(0)
>From what I understand, the RAID should be able to continue with a physical
disk loss and be in a "degraded" state, not "partial", because the data is
fully present on the surviving disks.
>From /etc/lvm/lvm.conf:
# degraded
# Like complete, but additionally RAID LVs of segment type
raid1,
# raid4, raid5, radid6 and raid10 will be activated if there is
no
# data loss, i.e. they have sufficient redundancy to present the
# entire addressable range of the Logical Volume.
# partial
# Allows the activation of any LV even if a missing or failed PV
# could cause data loss with a portion of the LV inaccessible.
# This setting should not normally be used, but may sometimes
# assist with data recovery.
From
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_logical_volumes/assembly_configure-mange-raid-configuring-and-managing-logical-volumes
:
"RAID is not like traditional LVM mirroring. LVM mirroring required failed
devices to be removed or the mirrored logical volume would hang. RAID
arrays can keep on running with failed devices. In fact, for RAID types
other than RAID1, removing a device would mean converting to a lower level
RAID (for example, from RAID6 to RAID5, or from RAID4 or RAID5 to RAID0).
However, in my case, the RAID is not converted, it's simply not available.
This is running in a virtual machine on VMware ESXi 7, LVM version:
2.03.07(2) (2019-11-30).
Am I missing something obvious? Appreciate any insights.
Thanks,
Andrei
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
