Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 09 of May 2012 19:18:07 David Sterba wrote: > On Thu, May 03, 2012 at 03:11:45PM +0200, Hubert Kario wrote: > > nice, didn't know about this. Such functionality would be nice to have. > > But then I don't think that a "recreate the array if the parameters are > > the > > same" is actually a good idea, lots of space for error. A pair of > > functions: > > > > btrfs dev zero-superblock > > btrfs dev restore-superblock > > As a user, I'm not sure what can I expect from the restore command. From > where does it restore? Eg. a file? > > As a tester I have use for a temporary clearing of a superblock on a > device, then mount it with -o degraded, work work, and then undo > clearing. So, my idea is like > > btrfs device zero-superblock --undo > > with the obvious sanity checks. A regular user would never need to call > this. Yes, that's a better idea. -- Hubert Kario QBS - Quality Business Software 02-656 Warszawa, ul. Ksawerów 30/85 tel. +48 (22) 646-61-51, 646-74-24 www.qbs.com.pl -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Thu, May 03, 2012 at 03:11:45PM +0200, Hubert Kario wrote: > nice, didn't know about this. Such functionality would be nice to have. > But then I don't think that a "recreate the array if the parameters are the > same" is actually a good idea, lots of space for error. A pair of functions: > > btrfs dev zero-superblock > btrfs dev restore-superblock As a user, I'm not sure what can I expect from the restore command. From where does it restore? Eg. a file? As a tester I have use for a temporary clearing of a superblock on a device, then mount it with -o degraded, work work, and then undo clearing. So, my idea is like btrfs device zero-superblock --undo with the obvious sanity checks. A regular user would never need to call this. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 02 of May 2012 19:36:29 David Sterba wrote:
> On Wed, May 02, 2012 at 06:42:16PM +0200, Hubert Kario wrote:
> > > I'm not sure if this is useful and sensible usecase, clearing superblock
> > > is a one-time action anyway, so it's more for the sake of tool
> > > flexibility.
> >
> > Clearing superblock is not a light decision and should generally be
> > performed just before formatting the partition with some other fs or
> > physical volume for LVM. IMHO recoverability of "cleared" superblock is a
> > function hardly anyone would use.
>
> googled, a few users asking about recovering from md zero-superblock, and
> the solution was to recreate the array, md is said to be smart and
> recognize traces of previous array and will not destroy it if the
> parameters are same. Point for md, btrfs does not do this.
nice, didn't know about this. Such functionality would be nice to have.
But then I don't think that a "recreate the array if the parameters are the
same" is actually a good idea, lots of space for error. A pair of functions:
btrfs dev zero-superblock
btrfs dev restore-superblock
would be a better solution IMO
> > > To your implementation: I think adding a function doing the superblock
> > > reset would be enough here. Something like this (in pseudocode):
> > >
> > > for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) {
> > >
> > > bytenr = btrfs_sb_offset(i);
> > > "break if bytenr > device size"
> > > memset(superblock buffer, CLEARPATTERN, sizeof(...))
> > >
> > > }
> > > write_all_supers(root);
> >
> > That's exactly what btrfs_prepare_device does. And it's a function run by
> > btfs just before btrfs dev add and by mkfs. Duplicating its code would be
> > a bad idea.
>
> Not 'exactly' IMO:
>
> * calls TRIM/discard on the device
> * zeroes first 2 megabytes
> * zeroes all reachable superblocks
> * zeroes last 2 megabytes
>
> Too many undocumented and unobvious side-efects.
True. But close enough ;)
> Code duplication can be avoided by factoring the 'zero superblock' into
> a function and calling it from btrfs_prepare_device().
Then there's also the "actually zero" vs "reversibly destroy" difference but
it's trivial to fix using a single option.
Regards
--
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawerów 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wed, May 02, 2012 at 06:42:16PM +0200, Hubert Kario wrote:
> A similar function in mdadm is called zero superblock so I just re used the
> name (according to the principle of least surprise). Users, even admins,
> generally don't read kernel code...
I intended to point out that the functionality is there under different
name, but the 'least surprise' and tool compatibility plays for
'zero-superblock' naming.
> > I'm not sure if this is useful and sensible usecase, clearing superblock
> > is a one-time action anyway, so it's more for the sake of tool
> > flexibility.
>
> Clearing superblock is not a light decision and should generally be performed
> just before formatting the partition with some other fs or physical volume
> for
> LVM. IMHO recoverability of "cleared" superblock is a function hardly anyone
> would use.
googled, a few users asking about recovering from md zero-superblock, and
the solution was to recreate the array, md is said to be smart and
recognize traces of previous array and will not destroy it if the
parameters are same. Point for md, btrfs does not do this.
> > To your implementation: I think adding a function doing the superblock
> > reset would be enough here. Something like this (in pseudocode):
> >
> > for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) {
> > bytenr = btrfs_sb_offset(i);
> > "break if bytenr > device size"
> > memset(superblock buffer, CLEARPATTERN, sizeof(...))
> > }
> > write_all_supers(root);
>
> That's exactly what btrfs_prepare_device does. And it's a function run by
> btfs
> just before btrfs dev add and by mkfs. Duplicating its code would be a bad
> idea.
Not 'exactly' IMO:
* calls TRIM/discard on the device
* zeroes first 2 megabytes
* zeroes all reachable superblocks
* zeroes last 2 megabytes
Too many undocumented and unobvious side-efects.
Code duplication can be avoided by factoring the 'zero superblock' into
a function and calling it from btrfs_prepare_device() .
david
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 02 of May 2012 16:28:43 David Sterba wrote:
> On Tue, May 01, 2012 at 02:40:29PM +0200, Hubert Kario wrote:
> > +static const char * const cmd_zero_dev_usage[] = {
> > + "btrfs device zero-superblock [ ...]",
>
> FYI, this step is named 'clear superblock' in kernel code as done after the
> device is removed, and I suggest to consider to name the command like
> 'clear-superblock' or 'clear-sb'.
A similar function in mdadm is called zero superblock so I just re used the
name (according to the principle of least surprise). Users, even admins,
generally don't read kernel code...
> Also, kernel clears only the "superblock magic" string, ie. the
> "_BHRfS_M" bytes. This leaves the rest of the superblock intact, for
> possible recovery when it's cleared accidentally.
That's when a device is removed from the filesystem, not when a filesystem is
just not used any more and you want to re-purpose the devices.
> I had prototyped a similar utility (in perl, so nothing for progs
> inclusion for now) and rewrote the magic string with _BHRf$_M ie. the
> S -> $ for visual similarity with the action performed. This allows to
> detect cleared superblocks and activate them back eventually.
>
> I'm not sure if this is useful and sensible usecase, clearing superblock
> is a one-time action anyway, so it's more for the sake of tool
> flexibility.
Clearing superblock is not a light decision and should generally be performed
just before formatting the partition with some other fs or physical volume for
LVM. IMHO recoverability of "cleared" superblock is a function hardly anyone
would use.
> To your implementation: I think adding a function doing the superblock
> reset would be enough here. Something like this (in pseudocode):
>
> for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) {
> bytenr = btrfs_sb_offset(i);
> "break if bytenr > device size"
> memset(superblock buffer, CLEARPATTERN, sizeof(...))
> }
> write_all_supers(root);
That's exactly what btrfs_prepare_device does. And it's a function run by btfs
just before btrfs dev add and by mkfs. Duplicating its code would be a bad
idea.
Regards,
--
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawerów 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wed, May 02, 2012 at 04:28:43PM +0200, David Sterba wrote:
> I had prototyped a similar utility (in perl, so nothing for progs
> inclusion for now)
attached.
david
#!/usr/bin/perl
# clear btrfs signature from a device
use Fcntl;
use constant BTRFS_SUPER_INFO_OFFSET => 64 * 1024;
use constant BTRFS_SUPER_INFO_SIZE => 4096;
use constant BTRFS_SUPER_MIRROR_MAX => 3;
use constant BTRFS_SUPER_MIRROR_SHIFT => 12;
use constant BTRFS_MAGIC => "_BHRfS_M";
use constant BTRFS_DEAD => '_BHRf$_M';
sub btrfs_sb_offset($) {
my $mirror =$_[0];
my $start = 16 * 1024;
if ($mirror>0) {
return $start << (BTRFS_SUPER_MIRROR_SHIFT * $mirror);
}
return BTRFS_SUPER_INFO_OFFSET;
}
my $dbg=1;
my $savesb=0;
# main
my $dev=$ARGV[0];
my $size;
if(!-b $dev) {
print("Not a block device: $dev\n");
$size=(stat($dev))[7];
} else {
$size=`blockdev --getsize64 "$dev"`;
}
sysopen(F, $dev, O_EXCL | O_RDWR) or die("Cannot open $dev exclusively: $!");
print("Device size: $size\n") if($dbg);
for(my $i=0;$i<6;$i++) {
my $off=btrfs_sb_offset($i);
if($off > $size) {
print("Offset for SB $i beyond EOF\n") if($dbg);
last;
}
print("Offset $i is $off\n") if($dbg);
sysseek(F, $off, 0);
sysread(F, $buf, BTRFS_SUPER_INFO_SIZE);
if($savesb) {
open(Q,">SB$i");
print Q ($buf);
close(Q);
}
my $sbmagic=substr($buf, 0x40, length(BTRFS_MAGIC));
print("SB magic: $sbmagic\n") if($dbg);
if(BTRFS_MAGIC eq $sbmagic) {
print("Found a valid signature of superblock $i\n");
sysseek(F, $off + 0x40, 0);
print("Clearing...\n");
syswrite(F, BTRFS_DEAD, length(BTRFS_DEAD));
} elsif(BTRFS_DEAD eq $sbmagic) {
print("Found a signature of a dead superblock $i\n");
} else {
print("Superblock $i does not look like a btrfs one\n");
}
}
close(F);
print("Syncing dev\n");
if (!-b $dev) {
system("fsync \'$dev\'");
} else {
system("blockdev --flushbufs \'$dev\'");
}
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Tue, May 01, 2012 at 02:40:29PM +0200, Hubert Kario wrote:
> +static const char * const cmd_zero_dev_usage[] = {
> + "btrfs device zero-superblock [ ...]",
FYI, this step is named 'clear superblock' in kernel code as done after the
device is removed, and I suggest to consider to name the command like
'clear-superblock' or 'clear-sb'.
Also, kernel clears only the "superblock magic" string, ie. the
"_BHRfS_M" bytes. This leaves the rest of the superblock intact, for
possible recovery when it's cleared accidentally.
I had prototyped a similar utility (in perl, so nothing for progs
inclusion for now) and rewrote the magic string with _BHRf$_M ie. the
S -> $ for visual similarity with the action performed. This allows to
detect cleared superblocks and activate them back eventually.
I'm not sure if this is useful and sensible usecase, clearing superblock
is a one-time action anyway, so it's more for the sake of tool
flexibility.
To your implementation: I think adding a function doing the superblock
reset would be enough here. Something like this (in pseudocode):
for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) {
bytenr = btrfs_sb_offset(i);
"break if bytenr > device size"
memset(superblock buffer, CLEARPATTERN, sizeof(...))
}
write_all_supers(root);
david
> + "Remove btrfs filesystem superblock from devices.",
> + "WARNING! This command will make filesystem residing on the devices",
> + "completely unmountable!",
> + NULL
> +};
> +
> +static int cmd_zero_dev(int argc, char **argv)
> +{
> + int fd;
> + char *file;
> + int arg_processed;
> + int ret = 0;
> + int n;
> + u64 device_len;
> + int mixed_mode_needed = 1; /* keep btrfs_prepare_device() quiet */
> + const int ZERO_END = 1;
> +
> + if( argc < 2 ) {
> + usage(cmd_zero_dev_usage);
> + }
> +
> + for(arg_processed = 1; arg_processed < argc; arg_processed++) {
> + file = argv[arg_processed];
> +
> + fd = open(file, O_RDWR);
> + if (fd < 0) {
> + fprintf(stderr, "Unable to open %s\n", file);
> + ret |= 1;
> + continue;
> + }
> +
> + n = btrfs_prepare_device(fd, file, ZERO_END, &device_len,
> + &mixed_mode_needed);
> + if (n) {
> + fprintf(stderr, "Error when zeroing out %s\n", file);
> + ret |= n;
> + }
> +
> + close(fd);
> + }
> +
> + return ret;
> +}
> +
> const struct cmd_group device_cmd_group = {
> device_cmd_group_usage, NULL, {
> { "add", cmd_add_dev, cmd_add_dev_usage, NULL, 0 },
> { "delete", cmd_rm_dev, cmd_rm_dev_usage, NULL, 0 },
> { "scan", cmd_scan_dev, cmd_scan_dev_usage, NULL, 0 },
> + { "zero-superblock", cmd_zero_dev, cmd_zero_dev_usage, NULL, 0
> },
> { 0, 0, 0, 0, 0 }
> }
> };
> diff --git a/man/btrfs.8.in b/man/btrfs.8.in
> index be478e0..a840f7e 100644
> --- a/man/btrfs.8.in
> +++ b/man/btrfs.8.in
> @@ -39,6 +39,8 @@ btrfs \- control a btrfs filesystem
> .PP
> \fBbtrfs\fP \fBdevice delete\fP\fI [...] \fP
> .PP
> +\fBbtrfs\fP \fBdevice zero-superblock\fP\fI [...] \fP
> +.PP
> \fBbtrfs\fP \fBscrub start\fP [-Bdqru] {\fI\fP|\fI\fP}
> .PP
> \fBbtrfs\fP \fBscrub cancel\fP {\fI\fP|\fI\fP}
> @@ -230,6 +232,11 @@ Finally, if \fB--all-devices\fP is passed, all the
> devices under /dev are
> scanned.
> .TP
>
> +\fBdevice zero-superblock\fR\fI [..]\fR
> +The space on the disk where btrfs metadata can reside is overwritten with
> +zeros.
> +.TP
> +
> \fBscrub start\fP [-Bdqru] {\fI\fP|\fI\fP}
> Start a scrub on all devices of the filesystem identified by \fI\fR or
> on
> a single \fI\fR. Without options, scrub is started as a background
> --
> 1.7.10
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 1/5] btrfs: add command to zero out superblock
Signed-off-by: Hubert Kario
diff --git a/cmds-device.c b/cmds-device.c
index db625a6..05a549c 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -246,11 +246,58 @@ static int cmd_scan_dev(int argc, char **argv)
return 0;
}
+static const char * const cmd_zero_dev_usage[] = {
+ "btrfs device zero-superblock [ ...]",
+ "Remove btrfs filesystem superblock from devices.",
+ "WARNING! This command will make filesystem residing on the devices",
+ "completely unmountable!",
+ NULL
+};
+
+static int cmd_zero_dev(int argc, char **argv)
+{
+ int fd;
+ char *file;
+ int arg_processed;
+ int ret = 0;
+ int n;
+ u64 device_len;
+ int mixed_mode_needed = 1; /* keep btrfs_prepare_device() quiet */
+ const int ZERO_END = 1;
+
+ if( argc < 2 ) {
+ usage(cmd_zero_dev_usage);
+ }
+
+ for(arg_processed = 1; arg_processed < argc; arg_processed++) {
+ file = argv[arg_processed];
+
+ fd = open(file, O_RDWR);
+ if (fd < 0) {
+ fprintf(stderr, "Unable to open %s\n", file);
+ ret |= 1;
+ continue;
+ }
+
+ n = btrfs_prepare_device(fd, file, ZERO_END, &device_len,
+ &mixed_mode_needed);
+ if (n) {
+ fprintf(stderr, "Error when zeroing out %s\n", file);
+ ret |= n;
+ }
+
+ close(fd);
+ }
+
+ return ret;
+}
+
const struct cmd_group device_cmd_group = {
device_cmd_group_usage, NULL, {
{ "add", cmd_add_dev, cmd_add_dev_usage, NULL, 0 },
{ "delete", cmd_rm_dev, cmd_rm_dev_usage, NULL, 0 },
{ "scan", cmd_scan_dev, cmd_scan_dev_usage, NULL, 0 },
+ { "zero-superblock", cmd_zero_dev, cmd_zero_dev_usage, NULL, 0
},
{ 0, 0, 0, 0, 0 }
}
};
diff --git a/man/btrfs.8.in b/man/btrfs.8.in
index be478e0..a840f7e 100644
--- a/man/btrfs.8.in
+++ b/man/btrfs.8.in
@@ -39,6 +39,8 @@ btrfs \- control a btrfs filesystem
.PP
\fBbtrfs\fP \fBdevice delete\fP\fI [...] \fP
.PP
+\fBbtrfs\fP \fBdevice zero-superblock\fP\fI [...] \fP
+.PP
\fBbtrfs\fP \fBscrub start\fP [-Bdqru] {\fI\fP|\fI\fP}
.PP
\fBbtrfs\fP \fBscrub cancel\fP {\fI\fP|\fI\fP}
@@ -230,6 +232,11 @@ Finally, if \fB--all-devices\fP is passed, all the devices
under /dev are
scanned.
.TP
+\fBdevice zero-superblock\fR\fI [..]\fR
+The space on the disk where btrfs metadata can reside is overwritten with
+zeros.
+.TP
+
\fBscrub start\fP [-Bdqru] {\fI\fP|\fI\fP}
Start a scrub on all devices of the filesystem identified by \fI\fR or on
a single \fI\fR. Without options, scrub is started as a background
--
1.7.10
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
