Hello community, here is the log from the commit of package mdadm for openSUSE:Factory checked in at 2020-02-15 22:24:27 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/mdadm (Old) and /work/SRC/openSUSE:Factory/.mdadm.new.26092 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "mdadm" Sat Feb 15 22:24:27 2020 rev:120 rq:773542 version:4.1 Changes: -------- --- /work/SRC/openSUSE:Factory/mdadm/mdadm.changes 2019-11-03 11:20:31.576581265 +0100 +++ /work/SRC/openSUSE:Factory/.mdadm.new.26092/mdadm.changes 2020-02-15 22:24:48.263289744 +0100 @@ -1,0 +2,11 @@ +Fri Feb 7 11:31:21 UTC 2020 - Coly Li <col...@suse.com> + +- Create: add support for RAID0 layouts. (bsc#1162479) + 0015-Create-add-support-for-RAID0-layouts.patch +- Assemble: add support for RAID0 layouts. (bsc#1162479) + 0016-Assemble-add-support-for-RAID0-layouts.patch +- mdadm.8: add note information for raid0 growing operation + (bsc#1129900) + 1002-mdadm.8-add-note-information-for-raid0-growing-opera.patch + +------------------------------------------------------------------- New: ---- 0015-Create-add-support-for-RAID0-layouts.patch 0016-Assemble-add-support-for-RAID0-layouts.patch 1002-mdadm.8-add-note-information-for-raid0-growing-opera.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ mdadm.spec ++++++ --- /var/tmp/diff_new_pack.eWziTa/_old 2020-02-15 22:24:48.851290062 +0100 +++ /var/tmp/diff_new_pack.eWziTa/_new 2020-02-15 22:24:48.863290069 +0100 @@ -1,7 +1,7 @@ # # spec file for package mdadm # -# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2020 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -56,7 +56,10 @@ Patch22: 0012-mdcheck-when-mdcheck_start-is-enabled-enable-mdcheck.patch Patch23: 0013-mdcheck-use-to-pass-variable-to-mdcheck.patch Patch24: 0014-SUSE-mdadm_env.sh-handle-MDADM_CHECK_DURATION.patch +Patch25: 0015-Create-add-support-for-RAID0-layouts.patch +Patch26: 0016-Assemble-add-support-for-RAID0-layouts.patch Patch1001: 1001-display-timeout-status.patch +Patch1002: 1002-mdadm.8-add-note-information-for-raid0-growing-opera.patch %define _udevdir %(pkg-config --variable=udevdir udev) %define _systemdshutdowndir %{_unitdir}/../system-shutdown @@ -80,7 +83,10 @@ %patch22 -p1 %patch23 -p1 %patch24 -p1 +%patch25 -p1 +%patch26 -p1 %patch1001 -p1 +%patch1002 -p1 %build make %{?_smp_mflags} CC="%__cc" CXFLAGS="%{optflags} -Wno-error" SUSE=yes ++++++ 0015-Create-add-support-for-RAID0-layouts.patch ++++++ >From 329dfc28debb58ffe7bd1967cea00fc583139aca Mon Sep 17 00:00:00 2001 From: NeilBrown <ne...@suse.de> Date: Mon, 4 Nov 2019 14:27:49 +1100 Subject: [PATCH] Create: add support for RAID0 layouts. Git-commit: 329dfc28debb58ffe7bd1967cea00fc583139aca Patch-mainline: mdadm-4.1+ References: bsc#1162479 Since Linux 5.4 a layout is needed for RAID0 arrays with varying device sizes. This patch makes the layout of an array visible (via --examine) and sets the layout on newly created arrays. --layout=dangerous can be used to avoid setting a layout so that they array can be used on older kernels. Tested-by: dann frazier <dann.fraz...@canonical.com> Signed-off-by: NeilBrown <ne...@suse.de> Signed-off-by: Jes Sorensen <jsoren...@fb.com> --- Create.c | 11 +++++++++++ Detail.c | 5 +++++ maps.c | 12 ++++++++++++ md.4 | 14 ++++++++++++++ mdadm.8.in | 30 +++++++++++++++++++++++++++++- mdadm.c | 8 ++++++++ mdadm.h | 8 +++++++- super0.c | 6 ++++++ super1.c | 30 +++++++++++++++++++++++++++++- 9 files changed, 121 insertions(+), 3 deletions(-) diff --git a/Create.c b/Create.c index 292f92a..6f84e5b 100644 --- a/Create.c +++ b/Create.c @@ -51,6 +51,9 @@ static int default_layout(struct supertype *st, int level, int verbose) default: /* no layout */ layout = 0; break; + case 0: + layout = RAID0_ORIG_LAYOUT; + break; case 10: layout = 0x102; /* near=2, far=1 */ if (verbose > 0) @@ -950,6 +953,11 @@ int Create(struct supertype *st, char *mddev, if (rv) { pr_err("ADD_NEW_DISK for %s failed: %s\n", dv->devname, strerror(errno)); + if (errno == EINVAL && + info.array.level == 0) { + pr_err("Possibly your kernel doesn't support RAID0 layouts.\n"); + pr_err("Either upgrade, or use --layout=dangerous\n"); + } goto abort_locked; } break; @@ -1046,6 +1054,9 @@ int Create(struct supertype *st, char *mddev, if (ioctl(mdfd, RUN_ARRAY, ¶m)) { pr_err("RUN_ARRAY failed: %s\n", strerror(errno)); + if (errno == 524 /* ENOTSUP */ && + info.array.level == 0) + cont_err("Please use --layout=original or --layout=alternate\n"); if (info.array.chunk_size & (info.array.chunk_size-1)) { cont_err("Problem may be that chunk size is not a power of 2\n"); } diff --git a/Detail.c b/Detail.c index 24fa462..832485f 100644 --- a/Detail.c +++ b/Detail.c @@ -525,6 +525,11 @@ int Detail(char *dev, struct context *c) printf(" Layout : %s\n", str ? str : "-unknown-"); } + if (array.level == 0 && array.layout) { + str = map_num(r0layout, array.layout); + printf(" Layout : %s\n", + str ? str : "-unknown-"); + } if (array.level == 6) { str = map_num(r6layout, array.layout); printf(" Layout : %s\n", diff --git a/maps.c b/maps.c index 49b7f2c..a4fd279 100644 --- a/maps.c +++ b/maps.c @@ -73,6 +73,18 @@ mapping_t r6layout[] = { { NULL, UnSet } }; +/* raid0 layout is only needed because of a bug in 3.14 which changed + * the effective layout of raid0 arrays with varying device sizes. + */ +mapping_t r0layout[] = { + { "original", RAID0_ORIG_LAYOUT}, + { "alternate", RAID0_ALT_MULTIZONE_LAYOUT}, + { "1", 1}, /* aka ORIG */ + { "2", 2}, /* aka ALT */ + { "dangerous", 0}, + { NULL, UnSet}, +}; + mapping_t pers[] = { { "linear", LEVEL_LINEAR}, { "raid0", 0}, diff --git a/md.4 b/md.4 index e86707a..6fe2755 100644 --- a/md.4 +++ b/md.4 @@ -193,6 +193,20 @@ smallest device has been exhausted, the RAID0 driver starts collecting chunks into smaller stripes that only span the drives which still have remaining space. +A bug was introduced in linux 3.14 which changed the layout of blocks in +a RAID0 beyond the region that is striped over all devices. This bug +does not affect an array with all devices the same size, but can affect +other RAID0 arrays. + +Linux 5.4 (and some stable kernels to which the change was backported) +will not normally assemble such an array as it cannot know which layout +to use. There is a module parameter "raid0.default_layout" which can be +set to "1" to force the kernel to use the pre-3.14 layout or to "2" to +force it to use the 3.14-and-later layout. when creating a new RAID0 +array, +.I mdadm +will record the chosen layout in the metadata in a way that allows newer +kernels to assemble the array without needing a module parameter. .SS RAID1 diff --git a/mdadm.8.in b/mdadm.8.in index 9aec9f4..fc9b6a6 100644 --- a/mdadm.8.in +++ b/mdadm.8.in @@ -593,6 +593,8 @@ to change the RAID level in some cases. See LEVEL CHANGES below. This option configures the fine details of data layout for RAID5, RAID6, and RAID10 arrays, and controls the failure modes for .IR faulty . +It can also be used for working around a kernel bug with RAID0, but generally +doesn't need to be used explicitly. The layout of the RAID5 parity block can be one of .BR left\-asymmetric , @@ -652,7 +654,7 @@ option to set subsequent failure modes. "clear" or "none" will remove any pending or periodic failure modes, and "flush" will clear any persistent faults. -Finally, the layout options for RAID10 are one of 'n', 'o' or 'f' followed +The layout options for RAID10 are one of 'n', 'o' or 'f' followed by a small number. The default is 'n2'. The supported options are: .I 'n' @@ -677,6 +679,32 @@ devices in the array. It does not need to divide evenly into that number (e.g. it is perfectly legal to have an 'n2' layout for an array with an odd number of devices). +A bug introduced in Linux 3.14 means that RAID0 arrays +.B "with devices of differing sizes" +started using a different layout. This could lead to +data corruption. Since Linux 5.4 (and various stable releases that received +backports), the kernel will not accept such an array unless +a layout is explictly set. It can be set to +.RB ' original ' +or +.RB ' alternate '. +When creating a new array, +.I mdadm +will select +.RB ' original ' +by default, so the layout does not normally need to be set. +An array created for either +.RB ' original ' +or +.RB ' alternate ' +will not be recognized by an (unpatched) kernel prior to 5.4. To create +a RAID0 array with devices of differing sizes that can be used on an +older kernel, you can set the layout to +.RB ' dangerous '. +This will use whichever layout the running kernel supports, so the data +on the array may become corrupt when changing kernel from pre-3.14 to a +later kernel. + When an array is converted between RAID5 and RAID6 an intermediate RAID6 layout is used in which the second parity block (Q) is always on the last device. To convert a RAID5 to RAID6 and leave it in this new diff --git a/mdadm.c b/mdadm.c index 1fb8086..e438f9c 100644 --- a/mdadm.c +++ b/mdadm.c @@ -550,6 +550,14 @@ int main(int argc, char *argv[]) pr_err("raid level must be given before layout.\n"); exit(2); + case 0: + s.layout = map_name(r0layout, optarg); + if (s.layout == UnSet) { + pr_err("layout %s not understood for raid0.\n", + optarg); + exit(2); + } + break; case 5: s.layout = map_name(r5layout, optarg); if (s.layout == UnSet) { diff --git a/mdadm.h b/mdadm.h index 91f1338..9e98778 100644 --- a/mdadm.h +++ b/mdadm.h @@ -763,7 +763,8 @@ extern int restore_stripes(int *dest, unsigned long long *offsets, extern char *map_num(mapping_t *map, int num); extern int map_name(mapping_t *map, char *name); -extern mapping_t r5layout[], r6layout[], pers[], modes[], faultylayout[]; +extern mapping_t r0layout[], r5layout[], r6layout[], + pers[], modes[], faultylayout[]; extern mapping_t consistency_policies[], sysfs_array_states[]; extern char *map_dev_preferred(int major, int minor, int create, @@ -1758,6 +1759,11 @@ char *xstrdup(const char *str); #define makedev(M,m) (((M)<<8) | (m)) #endif +enum r0layout { + RAID0_ORIG_LAYOUT = 1, + RAID0_ALT_MULTIZONE_LAYOUT = 2, +}; + /* for raid4/5/6 */ #define ALGORITHM_LEFT_ASYMMETRIC 0 #define ALGORITHM_RIGHT_ASYMMETRIC 1 diff --git a/super0.c b/super0.c index 6b7c0e3..6af140b 100644 --- a/super0.c +++ b/super0.c @@ -1291,6 +1291,12 @@ static int validate_geometry0(struct supertype *st, int level, if (*chunk == UnSet) *chunk = DEFAULT_CHUNK; + if (level == 0 && layout != UnSet) { + if (verbose) + pr_err("0.90 metadata does not support layouts for RAID0\n"); + return 0; + } + if (!subdev) return 1; diff --git a/super1.c b/super1.c index 929466d..cedbb53 100644 --- a/super1.c +++ b/super1.c @@ -43,7 +43,7 @@ struct mdp_superblock_1 { __u64 ctime; /* lo 40 bits are seconds, top 24 are microseconds or 0*/ __u32 level; /* -4 (multipath), -1 (linear), 0,1,4,5 */ - __u32 layout; /* only for raid5 currently */ + __u32 layout; /* used for raid5, raid6, raid10, and raid0 */ __u64 size; /* used size of component devices, in 512byte sectors */ __u32 chunksize; /* in 512byte sectors */ @@ -144,6 +144,7 @@ struct misc_dev_info { #define MD_FEATURE_JOURNAL 512 /* support write journal */ #define MD_FEATURE_PPL 1024 /* support PPL */ #define MD_FEATURE_MUTLIPLE_PPLS 2048 /* support for multiple PPLs */ +#define MD_FEATURE_RAID0_LAYOUT 4096 /* layout is meaningful in RAID0 */ #define MD_FEATURE_ALL (MD_FEATURE_BITMAP_OFFSET \ |MD_FEATURE_RECOVERY_OFFSET \ |MD_FEATURE_RESHAPE_ACTIVE \ @@ -155,6 +156,7 @@ struct misc_dev_info { |MD_FEATURE_JOURNAL \ |MD_FEATURE_PPL \ |MD_FEATURE_MULTIPLE_PPLS \ + |MD_FEATURE_RAID0_LAYOUT \ ) static int role_from_sb(struct mdp_superblock_1 *sb) @@ -498,6 +500,11 @@ static void examine_super1(struct supertype *st, char *homehost) printf(" Events : %llu\n", (unsigned long long)__le64_to_cpu(sb->events)); printf("\n"); + if (__le32_to_cpu(sb->level) == 0 && + (sb->feature_map & __cpu_to_le32(MD_FEATURE_RAID0_LAYOUT))) { + c = map_num(r0layout, __le32_to_cpu(sb->layout)); + printf(" Layout : %s\n", c?c:"-unknown-"); + } if (__le32_to_cpu(sb->level) == 5) { c = map_num(r5layout, __le32_to_cpu(sb->layout)); printf(" Layout : %s\n", c?c:"-unknown-"); @@ -1646,6 +1653,7 @@ struct devinfo { int fd; char *devname; long long data_offset; + unsigned long long dev_size; mdu_disk_info_t disk; struct devinfo *next; }; @@ -1687,6 +1695,7 @@ static int add_to_super1(struct supertype *st, mdu_disk_info_t *dk, di->devname = devname; di->disk = *dk; di->data_offset = data_offset; + get_dev_size(fd, NULL, &di->dev_size); di->next = NULL; *dip = di; @@ -1888,10 +1897,25 @@ static int write_init_super1(struct supertype *st) unsigned long long sb_offset; unsigned long long data_offset; long bm_offset; + int raid0_need_layout = 0; for (di = st->info; di; di = di->next) { if (di->disk.state & (1 << MD_DISK_JOURNAL)) sb->feature_map |= __cpu_to_le32(MD_FEATURE_JOURNAL); + if (sb->level == 0 && sb->layout != 0) { + struct devinfo *di2 = st->info; + unsigned long long s1, s2; + s1 = di->dev_size; + if (di->data_offset != INVALID_SECTORS) + s1 -= di->data_offset; + s1 /= __le32_to_cpu(sb->chunksize); + s2 = di2->dev_size; + if (di2->data_offset != INVALID_SECTORS) + s2 -= di2->data_offset; + s2 /= __le32_to_cpu(sb->chunksize); + if (s1 != s2) + raid0_need_layout = 1; + } } for (di = st->info; di; di = di->next) { @@ -2039,6 +2063,10 @@ static int write_init_super1(struct supertype *st) sb->bblog_offset = 0; } + /* RAID0 needs a layout if devices aren't all the same size */ + if (raid0_need_layout) + sb->feature_map |= __cpu_to_le32(MD_FEATURE_RAID0_LAYOUT); + sb->sb_csum = calc_sb_1_csum(sb); rv = store_super1(st, di->fd); -- 2.16.4 ++++++ 0016-Assemble-add-support-for-RAID0-layouts.patch ++++++ >From 027c099fd1a31fb3815e592de75d0791a22353b4 Mon Sep 17 00:00:00 2001 From: NeilBrown <ne...@suse.de> Date: Mon, 4 Nov 2019 14:27:49 +1100 Subject: [PATCH] Assemble: add support for RAID0 layouts. Git-commit: 027c099fd1a31fb3815e592de75d0791a22353b4 Patch-mainline: mdadm-4.1+ References: bsc#1162479 If you have a RAID0 array with varying sized devices on a kernel before 5.4, you cannot assembling it on 5.4 or later without explicitly setting the layout. This is now possible with --update=layout-original (For 3.13 and earlier kernels) or --update=layout-alternate (for 3.14 and later kernels) Signed-off-by: NeilBrown <ne...@suse.de> Signed-off-by: Jes Sorensen <jsoren...@fb.com> --- Assemble.c | 8 ++++++++ md.4 | 7 +++++++ mdadm.8.in | 17 +++++++++++++++++ mdadm.c | 4 ++++ super1.c | 12 +++++++++++- 5 files changed, 47 insertions(+), 1 deletion(-) diff --git a/Assemble.c b/Assemble.c index b2e6914..6b5a7c8 100644 --- a/Assemble.c +++ b/Assemble.c @@ -1031,6 +1031,11 @@ static int start_array(int mdfd, pr_err("failed to add %s to %s: %s\n", devices[j].devname, mddev, strerror(errno)); + if (errno == EINVAL && content->array.level == 0 && + content->array.layout != 0) { + cont_err("Possibly your kernel doesn't support RAID0 layouts.\n"); + cont_err("Please upgrade.\n"); + } if (i < content->array.raid_disks * 2 || i == bestcnt) okcnt--; @@ -1220,6 +1225,9 @@ static int start_array(int mdfd, return 0; } pr_err("failed to RUN_ARRAY %s: %s\n", mddev, strerror(errno)); + if (errno == 524 /* ENOTSUP */ && + content->array.level == 0 && content->array.layout == 0) + cont_err("Please use --update=layout-original or --update=layout-alternate\n"); if (!enough(content->array.level, content->array.raid_disks, content->array.layout, 1, avail)) diff --git a/md.4 b/md.4 index 6fe2755..0712af2 100644 --- a/md.4 +++ b/md.4 @@ -208,6 +208,13 @@ array, will record the chosen layout in the metadata in a way that allows newer kernels to assemble the array without needing a module parameter. +To assemble an old array on a new kernel without using the module parameter, +use either the +.B "--update=layout-original" +option or the +.B "--update=layout-alternate" +option. + .SS RAID1 A RAID1 array is also known as a mirrored set (though mirrors tend to diff --git a/mdadm.8.in b/mdadm.8.in index fc9b6a6..6b63bb4 100644 --- a/mdadm.8.in +++ b/mdadm.8.in @@ -1213,6 +1213,8 @@ argument given to this flag can be one of .BR no\-bbl , .BR ppl , .BR no\-ppl , +.BR layout\-original , +.BR layout\-alternate , .BR metadata , or .BR super\-minor . @@ -1364,6 +1366,21 @@ The .B no\-ppl option will disable PPL in the superblock. +The +.B layout\-original +and +.B layout\-alternate +options are for RAID0 arrays in use before Linux 5.4. If the array was being +used with Linux 3.13 or earlier, then to assemble the array on a new kernel, +.B \-\-update=layout\-original +must be given. If the array was created and used with a kernel from Linux 3.14 to +Linux 5.3, then +.B \-\-update=layout\-alternate +must be given. This only needs to be given once. Subsequent assembly of the array +will happen normally. +For more information, see +.IR md (4). + .TP .BR \-\-freeze\-reshape Option is intended to be used in start-up scripts during initrd boot phase. diff --git a/mdadm.c b/mdadm.c index e438f9c..256a97e 100644 --- a/mdadm.c +++ b/mdadm.c @@ -795,6 +795,9 @@ int main(int argc, char *argv[]) continue; if (strcmp(c.update, "revert-reshape") == 0) continue; + if (strcmp(c.update, "layout-original") == 0 || + strcmp(c.update, "layout-alternate") == 0) + continue; if (strcmp(c.update, "byteorder") == 0) { if (ss) { pr_err("must not set metadata type with --update=byteorder.\n"); @@ -825,6 +828,7 @@ int main(int argc, char *argv[]) " 'summaries', 'homehost', 'home-cluster', 'byteorder', 'devicesize',\n" " 'no-bitmap', 'metadata', 'revert-reshape'\n" " 'bbl', 'no-bbl', 'force-no-bbl', 'ppl', 'no-ppl'\n" + " 'layout-original', 'layout-alternate'\n" ); exit(outf == stdout ? 0 : 2); diff --git a/super1.c b/super1.c index cedbb53..e0d80be 100644 --- a/super1.c +++ b/super1.c @@ -1550,7 +1550,17 @@ static int update_super1(struct supertype *st, struct mdinfo *info, sb->devflags |= FailFast1; else if (strcmp(update, "nofailfast") == 0) sb->devflags &= ~FailFast1; - else + else if (strcmp(update, "layout-original") == 0 || + strcmp(update, "layout-alternate") == 0) { + if (__le32_to_cpu(sb->level) != 0) { + pr_err("%s: %s only supported for RAID0\n", + devname?:"", update); + rv = -1; + } else { + sb->feature_map |= __cpu_to_le32(MD_FEATURE_RAID0_LAYOUT); + sb->layout = __cpu_to_le32(update[7] == 'o' ? 1 : 2); + } + } else rv = -1; sb->sb_csum = calc_sb_1_csum(sb); -- 2.16.4 ++++++ 1002-mdadm.8-add-note-information-for-raid0-growing-opera.patch ++++++ >From 224228a2a4db8f433e4ae8fc827e231e095aca45 Mon Sep 17 00:00:00 2001 From: Coly Li <col...@suse.de> Date: Thu, 19 Dec 2019 18:08:05 +0800 Subject: [PATCH v3] mdadm.8: add note information for raid0 growing operation Patch-mainline: Not yet, waiting for maintainer to merge References: bsc#1129900 When growing a raid0 device, if the new component disk size is not big enough, the grow operation may fail due to lack of backup space. The minimum backup space should be larger than LCM(old, new) * chunk-size * 2 where LCM() is the least common multiple of the old and new count of component disks, and "* 2" comes from the fact that mdadm refuses to use more than half of a spare device for backup space. There are users reporting such failure when they grew a raid0 array with small component disk. Neil Brown points out this is not a bug and how the failure comes. This patch adds note information into mdadm(8) man page in the Notes part of GROW MODE section, to explain the minimum size requirement of new component disk size or external backup size. Signed-off-by: Coly Li <col...@suse.de> Cc: NeilBrown <ne...@suse.de> Cc: Jes Sorensen <jsoren...@fb.com> Cc: Paul Menzel <pmen...@molgen.mpg.de> Cc: Wols Lists <antli...@youngman.org.uk> Cc: Nix <n...@esperi.org.uk> --- mdadm.8.in | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mdadm.8.in b/mdadm.8.in index 6b63bb4..bba4e25 100644 --- a/mdadm.8.in +++ b/mdadm.8.in @@ -2772,6 +2772,12 @@ option and it is transparent for assembly feature. .IP \(bu 4 Roaming between Windows(R) and Linux systems for IMSM metadata is not supported during grow process. +.IP \(bu 4 +When growing a raid0 device, the new component disk size (or external +backup size) should be larger than LCM(old, new) * chunk-size * 2, +where LCM() is the least common multiple of the old and new count of +component disks, and "* 2" comes from the fact that mdadm refuses to +use more than half of a spare device for backup space. .SS SIZE CHANGES Normally when an array is built the "size" is taken from the smallest -- 2.16.4