[CentOS] Strange behavior from software RAID
Somewhere, mdadm is cacheing information. Here is my /etc/mdadm.conf file: more /etc/mdadm.conf # mdadm.conf written out by anaconda DEVICE partitions MAILADDR root ARRAY /dev/md0 level=raid1 num-devices=4 metadata=0.90 UUID=55ff58b2:0abb5bad:42911890:5950dfce ARRAY /dev/md1 level=raid1 num-devices=2 metadata=0.90 UUID=315eaf5c:776c85bd:5fa8189c:68a99382 ARRAY /dev/md2 level=raid1 num-devices=2 metadata=0.90 UUID=5b017f95:b7e266cc:f17a7611:8b752a02 ARRAY /dev/md3 level=raid1 num-devices=2 metadata=0.90 UUID=4cc310ee:60201e16:c7017bd4:9feea350 ARRAY /dev/md4 level=raid1 num-devices=2 metadata=0.90 UUID=ea205046:3c6e78c6:ab84faa4:0da53c7c After a system re-boot, here is the contents of /proc/mdstat # cat /proc/mdstat Personalities : [raid1] md125 : active raid1 sdc3[0] 455482816 blocks [2/1] [U_] md0 : active raid1 sdd1[3] sdc1[0] sdb1[1] sda1[2] 1000320 blocks [4/4] [] md127 : active raid1 sdd3[1] sdb3[0] 971747648 blocks [2/2] [UU] md3 : active raid1 sdf1[1] sde1[0] 1003904 blocks [2/2] [UU] md4 : active raid1 sdf3[1] sde3[0] 1948491648 blocks [2/2] [UU] md1 : active raid1 sda3[1] 455482816 blocks [2/1] [_U] unused devices: There are six physical disks in this system: Disk /dev/sda: 500.1 GB, 500107862016 bytes Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes Disk /dev/sdc: 500.1 GB, 500107862016 bytes Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes Disk /dev/sde: 2000.3 GB, 2000398934016 bytes Disk /dev/sdf: 2000.3 GB, 2000398934016 bytes I used mdadm --examine /dev/sda1 to find the internal UUID for each of the physical volumes making up these volume groups /dev/sda1: Magic : a92b4efc Version : 0.90.00 UUID : 55ff58b2:0abb5bad:42911890:5950dfce /dev/sdb1: Magic : a92b4efc Version : 0.90.00 UUID : 55ff58b2:0abb5bad:42911890:5950dfce /dev/sdc1: Magic : a92b4efc Version : 0.90.00 UUID : 55ff58b2:0abb5bad:42911890:5950dfce /dev/sdd1: Magic : a92b4efc Version : 0.90.00 UUID : 55ff58b2:0abb5bad:42911890:5950dfce /dev/sda3: Magic : a92b4efc Version : 0.90.00 UUID : 315eaf5c:776c85bd:5fa8189c:68a99382 /dev/sdc3: Magic : a92b4efc Version : 0.90.00 UUID : 315eaf5c:776c85bd:5fa8189c:68a99382 /dev/sdb3: Magic : a92b4efc Version : 0.90.00 UUID : 5b017f95:b7e266cc:f17a7611:8b752a02 /dev/sdd3: Magic : a92b4efc Version : 0.90.00 UUID : 5b017f95:b7e266cc:f17a7611:8b752a02 /dev/sde1: Magic : a92b4efc Version : 0.90.00 UUID : 4cc310ee:60201e16:c7017bd4:9feea350 /dev/sdf1: Magic : a92b4efc Version : 0.90.00 UUID : 4cc310ee:60201e16:c7017bd4:9feea350 /dev/sde3: Magic : a92b4efc Version : 0.90.00 UUID : ea205046:3c6e78c6:ab84faa4:0da53c7c /dev/sdf3: Magic : a92b4efc Version : 0.90.00 UUID : ea205046:3c6e78c6:ab84faa4:0da53c7c As you can see, the UUID on the various PVs match the values in the /etc/mdadm.conf file. My question is What the heck is going on. When I boot the system, I end up with two unexpected, unconfigured volume groups. Where the heck are /dev/md125 and /dev/md127 coming from? They don't appear in /etc/mdadm.conf and if I re-boot they keep coming back. It appears that somewhere mdadm is keeping information. How can I get rid of it so the mdadm.conf file is used. Harold ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange behavior from software RAID
Here I am following up on my own post... It occurred to me that all of this stuff must be magic. How does it work when the mdadm.conf file is on a raid/LVM volume which is not available at boot time? I looked in the /boot filesystem, the only one which is available at boot time and there is nothing there, unless this data is actually saved in one of the kernel modules or other binary files... Harold ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange behavior from software RAID
Hello. mdadm.conf is inside initrd file. If you have updated the MD configuration you need to update your initrd file. http://wiki.centos.org/TipsAndTricks/CreateNewInitrd 2013/3/3 Harold Pritchett : > Here I am following up on my own post... > > It occurred to me that all of this stuff must be magic. > > How does it work when the mdadm.conf file is on a raid/LVM volume which is > not available at boot time? > > I looked in the /boot filesystem, the only one which is available at boot > time and there is nothing there, unless this data is actually saved in one of > the kernel modules or other > binary files... > > Harold > > ___ > CentOS mailing list > CentOS@centos.org > http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange behavior from software RAID
You can usually generate a new mdadm.conf using: rm /etc/mdadm.conf mdadm --detail --scan >> /etc/mdadm.conf On 03/02/2013 09:35 PM, Harold Pritchett wrote: > Somewhere, mdadm is cacheing information. Here is my /etc/mdadm.conf file: > > more /etc/mdadm.conf > # mdadm.conf written out by anaconda > DEVICE partitions > MAILADDR root > ARRAY /dev/md0 level=raid1 num-devices=4 metadata=0.90 > UUID=55ff58b2:0abb5bad:42911890:5950dfce > ARRAY /dev/md1 level=raid1 num-devices=2 metadata=0.90 > UUID=315eaf5c:776c85bd:5fa8189c:68a99382 > ARRAY /dev/md2 level=raid1 num-devices=2 metadata=0.90 > UUID=5b017f95:b7e266cc:f17a7611:8b752a02 > ARRAY /dev/md3 level=raid1 num-devices=2 metadata=0.90 > UUID=4cc310ee:60201e16:c7017bd4:9feea350 > ARRAY /dev/md4 level=raid1 num-devices=2 metadata=0.90 > UUID=ea205046:3c6e78c6:ab84faa4:0da53c7c > > After a system re-boot, here is the contents of /proc/mdstat > > # cat /proc/mdstat > Personalities : [raid1] > md125 : active raid1 sdc3[0] >455482816 blocks [2/1] [U_] > > md0 : active raid1 sdd1[3] sdc1[0] sdb1[1] sda1[2] >1000320 blocks [4/4] [] > > md127 : active raid1 sdd3[1] sdb3[0] >971747648 blocks [2/2] [UU] > > md3 : active raid1 sdf1[1] sde1[0] >1003904 blocks [2/2] [UU] > > md4 : active raid1 sdf3[1] sde3[0] >1948491648 blocks [2/2] [UU] > > md1 : active raid1 sda3[1] >455482816 blocks [2/1] [_U] > > unused devices: > > There are six physical disks in this system: > > Disk /dev/sda: 500.1 GB, 500107862016 bytes > Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes > Disk /dev/sdc: 500.1 GB, 500107862016 bytes > Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes > Disk /dev/sde: 2000.3 GB, 2000398934016 bytes > Disk /dev/sdf: 2000.3 GB, 2000398934016 bytes > > I used mdadm --examine /dev/sda1 to find the internal UUID for each of the > physical volumes making up these volume groups > > /dev/sda1: Magic : a92b4efc Version : 0.90.00 UUID : > 55ff58b2:0abb5bad:42911890:5950dfce > /dev/sdb1: Magic : a92b4efc Version : 0.90.00 UUID : > 55ff58b2:0abb5bad:42911890:5950dfce > /dev/sdc1: Magic : a92b4efc Version : 0.90.00 UUID : > 55ff58b2:0abb5bad:42911890:5950dfce > /dev/sdd1: Magic : a92b4efc Version : 0.90.00 UUID : > 55ff58b2:0abb5bad:42911890:5950dfce > /dev/sda3: Magic : a92b4efc Version : 0.90.00 UUID : > 315eaf5c:776c85bd:5fa8189c:68a99382 > /dev/sdc3: Magic : a92b4efc Version : 0.90.00 UUID : > 315eaf5c:776c85bd:5fa8189c:68a99382 > /dev/sdb3: Magic : a92b4efc Version : 0.90.00 UUID : > 5b017f95:b7e266cc:f17a7611:8b752a02 > /dev/sdd3: Magic : a92b4efc Version : 0.90.00 UUID : > 5b017f95:b7e266cc:f17a7611:8b752a02 > /dev/sde1: Magic : a92b4efc Version : 0.90.00 UUID : > 4cc310ee:60201e16:c7017bd4:9feea350 > /dev/sdf1: Magic : a92b4efc Version : 0.90.00 UUID : > 4cc310ee:60201e16:c7017bd4:9feea350 > /dev/sde3: Magic : a92b4efc Version : 0.90.00 UUID : > ea205046:3c6e78c6:ab84faa4:0da53c7c > /dev/sdf3: Magic : a92b4efc Version : 0.90.00 UUID : > ea205046:3c6e78c6:ab84faa4:0da53c7c > > As you can see, the UUID on the various PVs match the values in the > /etc/mdadm.conf file. > > My question is What the heck is going on. When I boot the system, I end up > with two unexpected, unconfigured volume groups. Where the heck are > /dev/md125 and /dev/md127 coming > from? They don't appear in /etc/mdadm.conf and if I re-boot they keep coming > back. It appears that somewhere mdadm is keeping information. How can I get > rid of it so the > mdadm.conf file is used. > > Harold > > > ___ > CentOS mailing list > CentOS@centos.org > http://lists.centos.org/mailman/listinfo/centos > ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Strange behavior from software RAID
On 03/02/2013 06:35 PM, Harold Pritchett wrote: > When I boot the system, I end up with two unexpected, unconfigured > volume groups. "RAID set" is a better term. The term "volume group" describes components of the LVM system, which is not directly related to md raid. > Where the heck are /dev/md125 and /dev/md127 coming > from? Well, md125 is the other half of md1. Check which of those two devices has the correct data. Destroy the other, then add that partition to the remaining RAID device. > They don't appear in /etc/mdadm.conf and if I re-boot they > keep coming back. It appears that somewhere mdadm is keeping > information. How can I get rid of it so the mdadm.conf file is > used. I think you'll need to do two things. First, automatically-detected RAID sets get an automatic minor number assigned. That number is stored in the RAID metadata, so if you want to change it, you'll need to manually update the metadata. I think that's done by: mdadm --stop /dev/md127 mdadm --assemble /dev/md2 --update=super-minor /dev/sdd3 /dev/sdb3 As Maxim suggested, you may also need to re-build your initrd. As always, make sure you have backups first. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
