Re: [PATCH] btrfs-progs: change mans to describe the third copy of superblock
At Thu, 19 Oct 2017 17:05:18 +0800, Qu Wenruo wrote: > > > > On 2017年10月19日 16:34, Misono, Tomohiro wrote: > > On 2017/10/19 16:45, Satoru Takeuchi wrote: > >> Some tools can select which superblock these commands use by "-s > >> " > >> option. Although this option says the valid values are 0-2, we can set 3 > >> if filesystem is very large. > >> > > > > Hello, > > Wiki says there are 4 superblocks. However in the implementation > > BTRFS_SUPER_MIROR_MAX > > is 3 and 0 indicates the block at 64K (disk-io.h of btrfs-progs), therefore > > I think > > there is no 4th superblock actually. > > Kernel implementation also shows that it will only update up to 3 > superblocks: > > --- > if (max_mirrors == 0) > max_mirrors = BTRFS_SUPER_MIRROR_MAX; > > for (i = 0; i < max_mirrors; i++) { > bytenr = btrfs_sb_offset(i); > if (bytenr + BTRFS_SUPER_INFO_SIZE >= > device->commit_total_bytes) > break; > --- > > And BTRFS_SUPER_MIRROR_MAX is 3: > --- > #define BTRFS_SUPER_MIRROR_MAX 3 > --- > > So even you can set any value and btrfs_sb_offset() can calculate the > super block offset, you will just read out some garbage. My fault, sorry. I should read source more carefully. And thank you both to let me know my mistake. Thanks, Satoru > > Thanks, > Qu > > > > Regards, > > Tomohiro > > > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > > the body of a message to majord...@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Unmountable fs - missing generation?
At Thu, 19 Oct 2017 12:03:08 +0900, satoru takeuchi wrote: > > Resend it since I forgot to CC linux-btrfs ML >Larkin > > On Oct 17, 2017, at 0:16, Larkin Lowrey <llow...@nuclearwinter.com> > wrote: > > I am unable to mount one my my filesystems. The superblock thinks > the latest generation is 2220927 but I can't seem to find a root > with that number. I can find 2220926 and 2220928 but not 2220927. > Is there anything that I can do to recover this FS? > > > `btrfs-select-super` may help you. Please see the following steps. > > 1. Backup the current unmountable fs image if possible. > 2. Salvage your files as much as possilbe with reading the following > document > if possible. > > https://btrfs.wiki.kernel.org/index.php/Restore > > 3. Execute `btrfs-select-super -s 1 /dev/Cached/Backups`. Please note > that > this command changes the contents of /dev/Cached/Backups. So if this > command fails. Things would get worse. I forgot to tell one important point. You can only run above mentioned command iff 1st copy of superblock is valid. It can be confirmed by `btrfs inspect-internal dump-super (or btrfs-dump-super)` as follows. * valid case ``` $ sudo btrfs inspect dump-super -a /dev/sdb4 ... superblock: bytenr=67108864, device=/dev/sdb4 # 1st copy of superblock - csum0x423bcd19 [match] bytenr 67108864 flags 0x1 ... ``` * invalid case ``` $ sudo btrfs inspect dump-super -a /dev/sdb4 ... superblock: bytenr=67108864, device=/dev/sdb4 - ERROR: bad magic on superblock on /dev/sdb4 at 67108864 ... ``` Thanks, Satoru > Thanks, > Satoru > > > # btrfs check /dev/Cached/Backups > checksum verify failed on 159057884594176 found 15284E33 wanted > C8C5B54E > checksum verify failed on 159057884594176 found 15284E33 wanted > C8C5B54E > checksum verify failed on 159057884594176 found 472037C9 wanted > 9ACDCCB4 > checksum verify failed on 159057884594176 found 472037C9 wanted > 9ACDCCB4 > Csum didn't match > Couldn't setup extent tree > Couldn't open file system > > # btrfs-find-root -g 2220927 /dev/Cached/Backups > Couldn't setup extent tree > Couldn't setup device tree > Superblock thinks the generation is 2220927 > Superblock thinks the level is 2 > > Found tree root at 159057884577792 gen 2220927 level 2 > Well block 101489031790592(gen: 2220928 level: 2) seems good, but > generation/level doesn't match, want gen: 2220927 level: 2 > > # btrfs check --tree-root 159057884577792 /dev/Cached/Backups > checksum verify failed on 159057884594176 found 15284E33 wanted > C8C5B54E > checksum verify failed on 159057884594176 found 15284E33 wanted > C8C5B54E > checksum verify failed on 159057884594176 found 472037C9 wanted > 9ACDCCB4 > checksum verify failed on 159057884594176 found 472037C9 wanted > 9ACDCCB4 > Csum didn't match > Couldn't setup extent tree > Couldn't open file system > > # btrfs check --tree-root 101489031790592 /dev/Cached/Backups > parent transid verify failed on 101489031790592 wanted 2220927 > found 2220928 > parent transid verify failed on 101489031790592 wanted 2220927 > found 2220928 > parent transid verify failed on 101489031790592 wanted 2220927 > found 2220928 > parent transid verify failed on 101489031790592 wanted 2220927 > found 2220928 > Ignoring transid failure > parent transid verify failed on 159057595138048 wanted 2220927 > found 2220920 > parent transid verify failed on 159057595138048 wanted 2220927 > found 2220920 > parent transid verify failed on 159057595138048 wanted 2220927 > found 2220920 > parent transid verify failed on 159057595138048 wanted 2220927 > found 2220920 > Ignoring transid failure > parent transid verify failed on 158652658122752 wanted 2220927 > found 2220911 > parent transid verify failed on 158652658122752 wanted 2220927 > found 2220911 > parent transid verify failed on 158652658122752 wanted 2220927 > found 2220911 > parent transid verify failed on 158652658122752 wanted 2220927 > found 2220911 > Ignoring transid failure > Checking filesystem on /dev/Cached/Backups > UUID: 1b213dfd-6486-47d8-8459-bc5825882023 > checking extents > parent transid verify failed on 116329711550464 wanted 2220928 > found 2220921 > parent transid verify failed on 116329711550464 wanted 2220928 > fo
[PATCH] btrfs-progs: change mans to describe the third copy of superblock
Some tools can select which superblock these commands use by "-s " option. Although this option says the valid values are 0-2, we can set 3 if filesystem is very large. Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com> --- Documentation/btrfs-check.asciidoc| 2 +- Documentation/btrfs-restore.asciidoc | 2 +- Documentation/btrfs-select-super.asciidoc | 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Documentation/btrfs-check.asciidoc b/Documentation/btrfs-check.asciidoc index fbf4884..a557cff 100644 --- a/Documentation/btrfs-check.asciidoc +++ b/Documentation/btrfs-check.asciidoc @@ -74,7 +74,7 @@ run in read-only mode, this option exists to calm potential panic when users are going to run the checker -s|--super :: -use 'superblock'th superblock copy, valid values are 0, 1 or 2 if the +use 'superblock'th superblock copy, valid values are 0, 1, 2 or 3 if the respective superblock offset is within the device size + This can be used to use a different starting point if some of the primary diff --git a/Documentation/btrfs-restore.asciidoc b/Documentation/btrfs-restore.asciidoc index 090dcc5..c19e0e2 100644 --- a/Documentation/btrfs-restore.asciidoc +++ b/Documentation/btrfs-restore.asciidoc @@ -63,7 +63,7 @@ use to read the root tree only restore files that are under specified subvolume root pointed by -u|--super :: -use given superblock mirror identified by , it can be 0,1 or 2 +use given superblock mirror identified by , it can be 0, 1, 2 or 3 -r|--root :: only restore files that are under a specified subvolume whose objectid is diff --git a/Documentation/btrfs-select-super.asciidoc b/Documentation/btrfs-select-super.asciidoc index 6e94a03..7f96bd8 100644 --- a/Documentation/btrfs-select-super.asciidoc +++ b/Documentation/btrfs-select-super.asciidoc @@ -32,13 +32,14 @@ Superblock copies exist in the following offsets on the device: - primary: '64KiB' (65536) - 1st copy: '64MiB' (67108864) - 2nd copy: '256GiB' (274877906944) +- 3rd copy: '1PiB' (1125899906842624) A superblock size is '4KiB' (4096). OPTIONS --- -s|--super :: -use 'superblock'th superblock copy, valid values are 0 1 or 2 if the +use 'superblock'th superblock copy, valid values are 0, 1, 2 or 3 if the respective superblock offset is within the device size SEE ALSO -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs-progs: allow "none" to disable compression for convenience
At Tue, 19 Sep 2017 17:14:27 +0200, David Sterba wrote: > > On Mon, Sep 18, 2017 at 09:41:17AM +0900, Satoru Takeuchi wrote: > > At Sun, 17 Sep 2017 14:08:40 +0100, > > Mike Fleetwood wrote: > > > > > > On 17 September 2017 at 01:36, Satoru Takeuchi > > > <satoru.takeu...@gmail.com> wrote: > > > > It's messy to use "" to disable compression. Introduce the new value > > > > "no" > > > > which can also be used for this purpose. > > > > > > From an English language point of view, "none" would be better. None > > > says the absence of, where as no is more general negative. > > > > Thank you for your comment. How about is it? > > > > --- > > It's messy to use "" to disable compression. Introduce the new value "none" > > which can also be used for this purpose. > > I'd allow both values, 'no' and 'none', similar to the mount options, > that also accept both (technically, the 'no' + anything is accepted for > disabling compression). But only "no" is writtein in man 5 btrfs. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RESEND] btrfs-progs: allow "no" to disable compression for convenience
At Sat, 14 Oct 2017 18:19:14 +0200, Koen Kooi wrote: > > Op 14-10-17 om 14:54 schreef Satoru Takeuchi: > > It's messy to use "" to disable compression. Introduce the new value "no" > > which can also be used for this purpose. > > Wouldn't 'none' be a better fit? I consider "no" is better because we use `nocompress` or `compress=no` mount option to disable compression. I want to unify the expression. Thanks, Satoru > > regards, > > Koen > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: use BLK_STS defines where needed
At Sat, 14 Oct 2017 08:35:56 +0800, Anand Jain wrote: > > At few places we could use BLK_STS_OK and BLK_STS_NOSUPP. > > Signed-off-by: Anand JainReviewed-by: Satoru Taekeuchi > --- > fs/btrfs/compression.c | 3 ++- > fs/btrfs/inode.c | 4 ++-- > fs/btrfs/volumes.c | 2 +- > 3 files changed, 5 insertions(+), 4 deletions(-) > > diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c > index b51d23f5cafa..43d4b51556ec 100644 > --- a/fs/btrfs/compression.c > +++ b/fs/btrfs/compression.c > @@ -239,7 +239,8 @@ static void end_compressed_bio_write(struct bio *bio) >cb->start, >cb->start + cb->len - 1, >NULL, > - bio->bi_status ? 0 : 1); > + bio->bi_status ? > + BLK_STS_OK : BLK_STS_NOTSUPP); > cb->compressed_pages[0]->mapping = NULL; > > end_compressed_writeback(inode, cb); > diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c > index 128f3e58634f..9aa03c89ad86 100644 > --- a/fs/btrfs/inode.c > +++ b/fs/btrfs/inode.c > @@ -8360,7 +8360,7 @@ static void btrfs_endio_direct_read(struct bio *bio) > if (dip->flags & BTRFS_DIO_ORIG_BIO_SUBMITTED) { > err = btrfs_subio_endio_read(inode, io_bio, err); > if (!err) > - bio->bi_status = 0; > + bio->bi_status = BLK_STS_OK; > } > > unlock_extent(_I(inode)->io_tree, dip->logical_offset, > @@ -8478,7 +8478,7 @@ static void btrfs_end_dio_bio(struct bio *bio) > if (dip->errors) { > bio_io_error(dip->orig_bio); > } else { > - dip->dio_bio->bi_status = 0; > + dip->dio_bio->bi_status = BLK_STS_OK; > bio_endio(dip->orig_bio); > } > out: > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index a61405c2cd31..15e017af756c 100644 > --- a/fs/btrfs/volumes.c > +++ b/fs/btrfs/volumes.c > @@ -6020,7 +6020,7 @@ static void btrfs_end_bio(struct bio *bio) >* this bio is actually up to date, we didn't >* go over the max number of errors >*/ > - bio->bi_status = 0; > + bio->bi_status = BLK_STS_OK; > } > > btrfs_end_bbio(bbio, bio); > -- > 2.13.1 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH RESEND] btrfs-progs: allow "no" to disable compression for convenience
It's messy to use "" to disable compression. Introduce the new value "no" which can also be used for this purpose. Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com> --- Documentation/btrfs-property.asciidoc | 2 +- props.c | 6 -- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Documentation/btrfs-property.asciidoc b/Documentation/btrfs-property.asciidoc index 7ed6a7d..97b90d6 100644 --- a/Documentation/btrfs-property.asciidoc +++ b/Documentation/btrfs-property.asciidoc @@ -43,7 +43,7 @@ read-only flag of subvolume: true or false label label of device compression -compression setting for an inode: lzo, zlib, zstd, or "" (empty string) +compression setting for an inode: lzo, zlib, zstd, no, or "" (empty string). Both no and "" are for disabling compression. *list* [-t ] :: Lists available properties with their descriptions for the given object. diff --git a/props.c b/props.c index a7e3e96..a2df868 100644 --- a/props.c +++ b/props.c @@ -142,9 +142,11 @@ static int prop_compression(enum prop_object_type type, memcpy(xattr_name + XATTR_BTRFS_PREFIX_LEN, name, strlen(name)); xattr_name[XATTR_BTRFS_PREFIX_LEN + strlen(name)] = '\0'; - if (value) + if (value) { + if (!strcmp(value, "no")) + value = ""; sret = fsetxattr(fd, xattr_name, value, strlen(value), 0); - else + } else sret = fgetxattr(fd, xattr_name, NULL, 0); if (sret < 0) { ret = -errno; -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs: doc: update help/document of btrfs device remove
At Tue, 3 Oct 2017 17:12:39 +0900, Misono, Tomohiro wrote: > > This patch updates help/document of "btrfs device remove" in two points: > > 1. Add explanation of 'missing' for 'device remove'. This is only > written in wikipage currently. > (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices) > > 2. Add example of device removal in the man document. This is because > that explanation of "remove" says "See the example section below", but > there is no example of removal currently. > > Signed-off-by: Tomohiro Misono> --- > Documentation/btrfs-device.asciidoc | 19 +++ > cmds-device.c | 10 +- > 2 files changed, 28 insertions(+), 1 deletion(-) > > diff --git a/Documentation/btrfs-device.asciidoc > b/Documentation/btrfs-device.asciidoc > index 88822ec..dc523a9 100644 > --- a/Documentation/btrfs-device.asciidoc > +++ b/Documentation/btrfs-device.asciidoc > @@ -75,6 +75,10 @@ The operation can take long as it needs to move all data > from the device. > It is possible to delete the device that was used to mount the filesystem. > The > device entry in mount table will be replaced by another device name with the > lowest device id. > ++ > +If device is mounted as degraded mode (-o degraded), special term "missing" > +can be used for . In that case, the first device that is described by > +the filesystem metadata, but not presented at the mount time will be removed. > > *delete* | [|...] :: > Alias of remove kept for backward compatibility > @@ -206,6 +210,21 @@ data or the block groups occupy the whole first device. > The device size of '/dev/sdb' as seen by the filesystem remains unchanged, > but > the logical space from 50-100GiB will be unused. > > + REMOVE DEVICE It's a part of "TYPICAL USECASES" section. So it's also necessary to modify the following sentence === See the example section below. === to as follow. === See the *TYPICAL USECASES* section below. === Or just removing the above mentioned sentence is also OK since there is "See the section *TYPICAL USECASES* for some examples." in "DEVICE MANAGEMENT" section. > + > +Device removal must satisfy the profile constraints, otherwise the command > +fails. For example: > + > + $ btrfs device remove /dev/sda /mnt > + $ ERROR: error removing device '/dev/sda': unable to go below two devices > on raid1 s/^$ ERROR/ERROR/ > + > + > +In order to remove a device, you need to convert profile in this case: > + > + $ btrfs balance start -mconvert=dup /mnt > + $ btrfs balance start -dconvert=single /mnt It's simpler to convert both the RAID configuration of data and metadata by the following one command. $ btrfs balance -mconvert=dup -dconvert=single /mnt > + $ btrfs device remove /dev/sda /mnt > + > DEVICE STATS > > > diff --git a/cmds-device.c b/cmds-device.c > index 4337eb2..6cb53ff 100644 > --- a/cmds-device.c > +++ b/cmds-device.c > @@ -224,9 +224,16 @@ static int _cmd_device_remove(int argc, char **argv, > return !!ret; > } > > +#define COMMON_USAGE_REMOVE_DELETE \ > + "", \ > + "If 'missing' is specified for , the first device that is", \ > + "described by the filesystem metadata, but not presented at the", \ > + "mount time will be removed." > + > static const char * const cmd_device_remove_usage[] = { > "btrfs device remove | [|...] ", > "Remove a device from a filesystem", > + COMMON_USAGE_REMOVE_DELETE, > NULL > }; > > @@ -237,7 +244,8 @@ static int cmd_device_remove(int argc, char **argv) > > static const char * const cmd_device_delete_usage[] = { > "btrfs device delete | [|...] ", > - "Remove a device from a filesystem", > + "Remove a device from a filesystem (alias of \"btrfs device remove\")", > + COMMON_USAGE_REMOVE_DELETE, > NULL > }; This snippet is not related to the description of this patch. Dividing this patch is better. Thanks, Satoru > > -- > 2.9.5 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: USB upgrade fun
At Sun, 08 Oct 2017 17:58:10 +0800, Kai Hendry wrote: > > Hi there, > > My /mnt/raid1 suddenly became full somewhat expectedly, so I bought 2 > new USB 4TB hard drives (one WD, one Seagate) to upgrade to. > > After adding sde and sdd I started to see errors in dmesg [2]. > https://s.natalian.org/2017-10-07/raid1-newdisks.txt > [2] https://s.natalian.org/2017-10-07/btrfs-errors.txt These messages are harmless. Qu is tuckling on this problem. > > Sidenote: I've since learnt that removing a drive actually deletes the > contents of the drive? I don't want that. I was hoping to put that drive > into cold storage. How do I remove a drive without losing data from a > RAID1 configuration? Please let me clarify what you said. Do you worry about losing filesystem data in removed device, in this case /dev/sdc1? To be more specific, if /mnt/raid1/file is in /dev/sdc1 and lose this file by removing this device? If so, don't worry. When removing /dev/sdc1, the filesystem data exists in this device is moved to other devices, /dev/sdb1, /dev/sdd1, or /dev/sde1. Just FYI, `btrfs replace /dev/sdc1 /dev/sdd1 /mnt/raid1` is more suitable in your case. > > > I assumed it had to perhaps with the USB bus on my NUC5CPYB being maxed > out, and to expedite the sync, I tried to remove one of the older 2TB > sdc1. However the load went crazy and my system went completely > unstable. I shutdown the machine and after an hour I hard powered it > down since it seemed to hang (it's headless). Because all data in /dev/sdc1, in this case totally 1.81TiB(data) + 6.00GiB(metadata) + 32MiB(system) should be moved to remaining devices. > > > After a reboot it failed, namely because "nofail" wasn't in my fstab and > systemd is pedantic by default. After managing to get it booting into my > system without /mnt/raid1 I faced these "open ctree failed" issues. > After running btrfs check on all the drives and getting nowhere, I > decided to unplug the new drives and I discovered that when I take out > the new 4TB WD drive, I could mount it with -o degraded. > > dmesg errors with the WD include "My Passport" Wrong diagnostic page; > asked for 1 got 8 "Failed to get diagnostic page 0xffea" which > raised my suspicions. The model number btw is WDBYFT0040BYI-WESN > > Anyway, I'm back up and running with 2x2TB (one of them didn't finish > removing, I don't know which) & 1x4TB. > > [1] https://s.natalian.org/2017-10-08/usb-btrfs.txt > > I've decided to send the WD back for a refund. I've decided I want keep > the 2x2TB and RAID1 with the new 1x4TB disk. So 4TB total. btrfs > complains now of "Some devices missing" [1]. How do I fix this > situation? Probably `btrfs device remove missing /mnt/raid1` works. Thanks, Satoru > > Any tips how to name this individual disks? hdparm -I isn't a lot to go > on. > > [hendry@nuc ~]$ sudo hdparm -I /dev/sdb1 | grep Model > Model Number: ST4000LM024-2AN17V > [hendry@nuc ~]$ sudo hdparm -I /dev/sdc1 | grep Model > Model Number: ST2000LM003 HN-M201RAD > [hendry@nuc ~]$ sudo hdparm -I /dev/sdd1 | grep Model > Model Number: ST2000LM005 HN-M201AAD > > > Ok, thanks. Hope you can guide me, > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: Remove WARN_ON for unaligned device created before v4.13 and adds more user friendly output
At Sat, 23 Sep 2017 10:19:26 +0900, Satoru Takeuchi wrote: > > At Wed, 20 Sep 2017 15:18:43 +0900, > Qu Wenruo wrote: > > > > Commit 7dfb8be11b5d ("btrfs: Round down values which are written for > > total_bytes_size") is fixing the unaligned device size caused by > > adding/shrinking device. > > > > It added a new WARN_ON() when device size is unaligned. > > This is fine for new device added to btrfs using v4.13 kernel, but not > > existing device whose total_bytes is already unaligned. > > > > And the WARN_ON() will get triggered every time a block group get > > created/removed on the unaligned device. > > > > This patch will remove the WARN_ON(), and warn user more gently what's > > happening and how to fix it. > > > > Reported-by: Rich Rauenzahn <r...@shroop.net> > > Fixes: 7dfb8be11b5d ("btrfs: Round down values which are written for > > total_bytes_size") > > Signed-off-by: Qu Wenruo <quwenruo.bt...@gmx.com> > > --- > > fs/btrfs/ctree.h | 1 - > > fs/btrfs/volumes.c | 16 > > 2 files changed, 12 insertions(+), 5 deletions(-) > > > > diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h > > index 5a8933da39a7..4de9269e435a 100644 > > --- a/fs/btrfs/ctree.h > > +++ b/fs/btrfs/ctree.h > > @@ -1562,7 +1562,6 @@ static inline void > > btrfs_set_device_total_bytes(struct extent_buffer *eb, > > { > > BUILD_BUG_ON(sizeof(u64) != > > sizeof(((struct btrfs_dev_item *)0))->total_bytes); > > - WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); > > btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), val); > > } > > > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > > index 0e8f16c305df..afae25df6a8c 100644 > > --- a/fs/btrfs/volumes.c > > +++ b/fs/btrfs/volumes.c > > @@ -6472,15 +6472,23 @@ static int read_one_chunk(struct btrfs_fs_info > > *fs_info, struct btrfs_key *key, > > return 0; > > } > > > > -static void fill_device_from_item(struct extent_buffer *leaf, > > -struct btrfs_dev_item *dev_item, > > -struct btrfs_device *device) > > +static void fill_device_from_item(struct btrfs_fs_info *fs_info, > > + struct extent_buffer *leaf, > > + struct btrfs_dev_item *dev_item, > > + struct btrfs_device *device) > > { > > unsigned long ptr; > > > > device->devid = btrfs_device_id(leaf, dev_item); > > device->disk_total_bytes = btrfs_device_total_bytes(leaf, dev_item); > > device->total_bytes = device->disk_total_bytes; > > + if (!IS_ALIGNED(device->total_bytes, fs_info->sectorsize)) { > > + btrfs_warn(fs_info, > > + "devid %llu has unaligned total bytes %llu", > > + device->devid, device->disk_total_bytes); > > + btrfs_warn(fs_info, > > + "please shrink the device a little and resize back > > to fix it"); > > + } > > How about telling uses to know device->total_bytes should be alligned > to fs_info->sectorsize here? > > Thanks, I should make my comment clearer, sorry. === + if (!IS_ALIGNED(device->total_bytes, fs_info->sectorsize)) { + btrfs_warn(fs_info, + "devid %llu: total bytes %llu should be aligned to %u bytes", + device->devid, device->disk_total_bytes, fs_info->sectorsize); + btrfs_warn(fs_info, + "please shrink the device a little and resize back to fix it"); + } === Thanks, Satoru > Satoru > > > device->commit_total_bytes = device->disk_total_bytes; > > device->bytes_used = btrfs_device_bytes_used(leaf, dev_item); > > device->commit_bytes_used = device->bytes_used; > > @@ -6625,7 +6633,7 @@ static int read_one_dev(struct btrfs_fs_info *fs_info, > > return -EINVAL; > > } > > > > - fill_device_from_item(leaf, dev_item, device); > > + fill_device_from_item(fs_info, leaf, dev_item, device); > > device->in_fs_metadata = 1; > > if (device->writeable && !device->is_tgtdev_for_dev_replace) { > > device->fs_devices->total_rw_bytes += device->total_bytes; > > -- > > 2.14.1 > > > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > > the body of a message to majord...@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: Remove WARN_ON for unaligned device created before v4.13 and adds more user friendly output
At Wed, 20 Sep 2017 15:18:43 +0900, Qu Wenruo wrote: > > Commit 7dfb8be11b5d ("btrfs: Round down values which are written for > total_bytes_size") is fixing the unaligned device size caused by > adding/shrinking device. > > It added a new WARN_ON() when device size is unaligned. > This is fine for new device added to btrfs using v4.13 kernel, but not > existing device whose total_bytes is already unaligned. > > And the WARN_ON() will get triggered every time a block group get > created/removed on the unaligned device. > > This patch will remove the WARN_ON(), and warn user more gently what's > happening and how to fix it. > > Reported-by: Rich Rauenzahn> Fixes: 7dfb8be11b5d ("btrfs: Round down values which are written for > total_bytes_size") > Signed-off-by: Qu Wenruo > --- > fs/btrfs/ctree.h | 1 - > fs/btrfs/volumes.c | 16 > 2 files changed, 12 insertions(+), 5 deletions(-) > > diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h > index 5a8933da39a7..4de9269e435a 100644 > --- a/fs/btrfs/ctree.h > +++ b/fs/btrfs/ctree.h > @@ -1562,7 +1562,6 @@ static inline void btrfs_set_device_total_bytes(struct > extent_buffer *eb, > { > BUILD_BUG_ON(sizeof(u64) != >sizeof(((struct btrfs_dev_item *)0))->total_bytes); > - WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); > btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), val); > } > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index 0e8f16c305df..afae25df6a8c 100644 > --- a/fs/btrfs/volumes.c > +++ b/fs/btrfs/volumes.c > @@ -6472,15 +6472,23 @@ static int read_one_chunk(struct btrfs_fs_info > *fs_info, struct btrfs_key *key, > return 0; > } > > -static void fill_device_from_item(struct extent_buffer *leaf, > - struct btrfs_dev_item *dev_item, > - struct btrfs_device *device) > +static void fill_device_from_item(struct btrfs_fs_info *fs_info, > + struct extent_buffer *leaf, > + struct btrfs_dev_item *dev_item, > + struct btrfs_device *device) > { > unsigned long ptr; > > device->devid = btrfs_device_id(leaf, dev_item); > device->disk_total_bytes = btrfs_device_total_bytes(leaf, dev_item); > device->total_bytes = device->disk_total_bytes; > + if (!IS_ALIGNED(device->total_bytes, fs_info->sectorsize)) { > + btrfs_warn(fs_info, > +"devid %llu has unaligned total bytes %llu", > +device->devid, device->disk_total_bytes); > + btrfs_warn(fs_info, > +"please shrink the device a little and resize back > to fix it"); > + } How about telling uses to know device->total_bytes should be alligned to fs_info->sectorsize here? Thanks, Satoru > device->commit_total_bytes = device->disk_total_bytes; > device->bytes_used = btrfs_device_bytes_used(leaf, dev_item); > device->commit_bytes_used = device->bytes_used; > @@ -6625,7 +6633,7 @@ static int read_one_dev(struct btrfs_fs_info *fs_info, > return -EINVAL; > } > > - fill_device_from_item(leaf, dev_item, device); > + fill_device_from_item(fs_info, leaf, dev_item, device); > device->in_fs_metadata = 1; > if (device->writeable && !device->is_tgtdev_for_dev_replace) { > device->fs_devices->total_rw_bytes += device->total_bytes; > -- > 2.14.1 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: use btrfs_op instead of bio_op in __btrfs_map_block
At Tue, 19 Sep 2017 17:50:09 -0600, Liu Bo wrote: > > This seems to be a leftover of commit cf8cddd38bab ("btrfs: don't > abuse REQ_OP_* flags for btrfs_map_block"). > > It should use btrfs_op() helper to provide one of 'enum btrfs_map_op' > types. > > Signed-off-by: Liu Bo <bo.li....@oracle.com> Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com> I checked all callers of the following functions and there is no leftover. - btrfs_map_block - btrfs_map_sblock - __btrfs_map_block Thanks, Satoru > --- > fs/btrfs/volumes.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index bd679bc..bd7b75d3 100644 > --- a/fs/btrfs/volumes.c > +++ b/fs/btrfs/volumes.c > @@ -6229,7 +6229,7 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info > *fs_info, struct bio *bio, > map_length = length; > > btrfs_bio_counter_inc_blocked(fs_info); > - ret = __btrfs_map_block(fs_info, bio_op(bio), logical, > + ret = __btrfs_map_block(fs_info, btrfs_op(bio), logical, > _length, , mirror_num, 1); > if (ret) { > btrfs_bio_counter_dec(fs_info); > -- > 2.9.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds
At Tue, 19 Sep 2017 16:41:52 +0900, Misono, Tomohiro wrote: > > "btrfs subvolume create/delete" outputs the message of "Create/Delete > subvolume ..." even when an operation fails. > Since it is confusing, let's outputs the message only when an operation > succeeds. > > Signed-off-by: Tomohiro Misono <misono.tomoh...@jp.fujitsu.com> Current message as follows is odd as you said. ``` Create subvolume './test' ERROR: cannot create subvolume: No such file or directory ``` It's ambiguous for users to know whether creating subvolume succeeded or not. I tested this patch with injecting error on ioctl() for subvol creation/deletion. Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com> Tested-by: Satoru Takeuchi <satoru.takeu...@gmail.com> > --- > cmds-subvolume.c | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/cmds-subvolume.c b/cmds-subvolume.c > index 666f6e0..6d4b0fe 100644 > --- a/cmds-subvolume.c > +++ b/cmds-subvolume.c > @@ -189,7 +189,6 @@ static int cmd_subvol_create(int argc, char **argv) > if (fddst < 0) > goto out; > > - printf("Create subvolume '%s/%s'\n", dstdir, newname); > if (inherit) { > struct btrfs_ioctl_vol_args_v2 args; > > @@ -213,6 +212,7 @@ static int cmd_subvol_create(int argc, char **argv) > error("cannot create subvolume: %s", strerror(errno)); > goto out; > } > + printf("Create subvolume '%s/%s'\n", dstdir, newname); > > retval = 0; /* success */ > out: > @@ -337,9 +337,6 @@ again: > goto out; > } > > - printf("Delete subvolume (%s): '%s/%s'\n", > - commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) > - ? "commit" : "no-commit", dname, vname); > memset(, 0, sizeof(args)); > strncpy_null(args.name, vname); > res = ioctl(fd, BTRFS_IOC_SNAP_DESTROY, ); > @@ -349,6 +346,9 @@ again: > ret = 1; > goto out; > } > + printf("Delete subvolume (%s): '%s/%s'\n", > + commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) > + ? "commit" : "no-commit", dname, vname); > > if (commit_mode == 1) { > res = wait_for_commit(fd); > -- > 2.9.5 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs-progs: allow "none" to disable compression for convenience
At Tue, 19 Sep 2017 17:14:27 +0200, David Sterba wrote: > > On Mon, Sep 18, 2017 at 09:41:17AM +0900, Satoru Takeuchi wrote: > > At Sun, 17 Sep 2017 14:08:40 +0100, > > Mike Fleetwood wrote: > > > > > > On 17 September 2017 at 01:36, Satoru Takeuchi > > > <satoru.takeu...@gmail.com> wrote: > > > > It's messy to use "" to disable compression. Introduce the new value > > > > "no" > > > > which can also be used for this purpose. > > > > > > From an English language point of view, "none" would be better. None > > > says the absence of, where as no is more general negative. > > > > Thank you for your comment. How about is it? > > > > --- > > It's messy to use "" to disable compression. Introduce the new value "none" > > which can also be used for this purpose. > > I'd allow both values, 'no' and 'none', similar to the mount options, > that also accept both (technically, the 'no' + anything is accepted for > disabling compression). As a result of reading "man 5 btrfs", now I prefer "no". It's used to mean disabling compression there. On the other hand, "none" is not used at all. From man 5 btrfs: === ... FILE ATTRIBUTES ... compress, compress=type, compress-force, compress-force=type (default: off) Control BTRFS file data compression. Type may be specified as zlib, lzo or no (for no compression, used for remounting). If no type is specified, zlib is used. If compress-force is specified, all files will be compressed, whether or not they compress well. ... X no compression, permanently turn off compression on the given file, other compression mount options will not affect that ... === So David, please apply my v1 patcth if it looks good for you. Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs Issues
At Wed, 13 Sep 2017 11:53:35 -0400, Ruoxin Jiang wrote: > > [1 ] > Hello, > > We are researchers from Columbia University, New York. As part of our > current research we have found some semantic discrepancies between > btrfs and other popular filesystems. > > We have attached two cases. The first one involves an invalid O_DIRECT > write() that fails back to buffered write instead of failing with > error EINVAL. In directory 2, we discovered that btrfs calculates > write_bytes in __btrfs_buffered_write differently from that in > generic_perform_writes in fs/mmap.c. This can cause inconsistent > behavior between btrfs and other filesystems when program invokes the > same writev/write() syscall. > > In each directory, you will find a Readme.md describing the issue and > pointing to the code that may cause the problem. For your convenience, > we also included test programs (min.cpp) and instructions in Readme to > help reproduce the issues. > > We would appreciate very much if you could confirm the two cases at > your conveniences. I took a look at your test programs, btrfs_issues/{1,2}/min.cpp. It looks very hard to read since you call syscalls in odd ways and all flags are hardcoded as literal hexadecimal numbers. Could rewrite these program to improve readability? In addition, I have two questions about btrfs_issues/1/min.cpp. 1. Why you set 'filename' as the 1st argument of mmap()? 2. What's the purpose of mmap() call? I guess mmap() is not related to issue 1. Thanks, Satoru > > Thanks, > Amy > [2 btrfs_issues.tar.gz ] -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs-progs: Output time elapsed for each major tree it checked
At Mon, 18 Sep 2017 09:47:29 +0900, Qu Wenruo wrote: > > Marc reported that "btrfs check --repair" runs much faster than "btrfs > check", which is quite weird. > > This patch will add time elapsed for each major tree it checked, for > both original mode and lowmem mode, so we can have a clue what's going > wrong. > > Reported-by: Marc MERLIN <m...@merlins.org> > Signed-off-by: Qu Wenruo <quwenruo.bt...@gmx.com> > --- > v2: > Add prefix for each time report, as error message may make the output > hard to understand. Suggested by Satoru. Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com> > --- > cmds-check.c | 21 +++-- > utils.h | 24 > 2 files changed, 43 insertions(+), 2 deletions(-) > > diff --git a/cmds-check.c b/cmds-check.c > index 006edbde..f1074c73 100644 > --- a/cmds-check.c > +++ b/cmds-check.c > @@ -5318,13 +5318,16 @@ static int do_check_fs_roots(struct btrfs_fs_info > *fs_info, > struct cache_tree *root_cache) > { > int ret; > + struct timer timer; > > if (!ctx.progress_enabled) > fprintf(stderr, "checking fs roots\n"); > + start_timer(); > if (check_mode == CHECK_MODE_LOWMEM) > ret = check_fs_roots_v2(fs_info); > else > ret = check_fs_roots(fs_info, root_cache); > + printf("fs roots checked in %d seconds\n", stop_timer()); > > return ret; > } > @@ -11584,14 +11587,16 @@ out: > static int do_check_chunks_and_extents(struct btrfs_fs_info *fs_info) > { > int ret; > + struct timer timer; > > if (!ctx.progress_enabled) > fprintf(stderr, "checking extents\n"); > + start_timer(); > if (check_mode == CHECK_MODE_LOWMEM) > ret = check_chunks_and_extents_v2(fs_info); > else > ret = check_chunks_and_extents(fs_info); > - > + printf("extents checked in %d seconds\n", stop_timer()); > return ret; > } > > @@ -12772,6 +12777,7 @@ int cmd_check(int argc, char **argv) > int qgroups_repaired = 0; > unsigned ctree_flags = OPEN_CTREE_EXCLUSIVE; > int force = 0; > + struct timer timer; > > while(1) { > int c; > @@ -12953,8 +12959,11 @@ int cmd_check(int argc, char **argv) > if (repair) > ctree_flags |= OPEN_CTREE_PARTIAL; > > + printf("opening btrfs filesystem\n"); > + start_timer(); > info = open_ctree_fs_info(argv[optind], bytenr, tree_root_bytenr, > chunk_root_bytenr, ctree_flags); > + printf("btrfs filesystem opened in %d seconds\n", stop_timer()); > if (!info) { > error("cannot open file system"); > ret = -EIO; > @@ -13115,8 +13124,10 @@ int cmd_check(int argc, char **argv) > else > fprintf(stderr, "checking free space cache\n"); > } > + start_timer(); > ret = check_space_cache(root); > err |= !!ret; > + printf("free space cache (tree) checked in %d seconds\n", > stop_timer()); > if (ret) { > if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE)) > error("errors found in free space tree"); > @@ -13140,18 +13151,22 @@ int cmd_check(int argc, char **argv) > } > > fprintf(stderr, "checking csums\n"); > + start_timer(); > ret = check_csums(root); > err |= !!ret; > + printf("csums tree checked in %d seconds\n", stop_timer()); > if (ret) { > error("errors found in csum tree"); > goto out; > } > > - fprintf(stderr, "checking root refs\n"); > /* For low memory mode, check_fs_roots_v2 handles root refs */ > if (check_mode != CHECK_MODE_LOWMEM) { > + fprintf(stderr, "checking root refs\n"); > + start_timer(); > ret = check_root_refs(root, _cache); > err |= !!ret; > + printf("root refs checked in %d seconds\n", stop_timer()); > if (ret) { > error("errors found in root refs"); > goto out; > @@ -13186,8 +13201,10 @@ int cmd_check(int argc, char **argv) > > if (info->quota_enabled) { > fprintf(stderr, "checking quota groups\n"); > + start_timer(); > ret = qgroup_verify_all(info); &
[PATCH v2] btrfs-progs: allow "none" to disable compression for convenience
At Sun, 17 Sep 2017 14:08:40 +0100, Mike Fleetwood wrote: > > On 17 September 2017 at 01:36, Satoru Takeuchi > <satoru.takeu...@gmail.com> wrote: > > It's messy to use "" to disable compression. Introduce the new value "no" > > which can also be used for this purpose. > > From an English language point of view, "none" would be better. None > says the absence of, where as no is more general negative. Thank you for your comment. How about is it? --- It's messy to use "" to disable compression. Introduce the new value "none" which can also be used for this purpose. Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com> --- Documentation/btrfs-property.asciidoc | 2 +- props.c | 6 -- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Documentation/btrfs-property.asciidoc b/Documentation/btrfs-property.asciidoc index 7ed6a7d..786af9b 100644 --- a/Documentation/btrfs-property.asciidoc +++ b/Documentation/btrfs-property.asciidoc @@ -43,7 +43,7 @@ read-only flag of subvolume: true or false label label of device compression -compression setting for an inode: lzo, zlib, zstd, or "" (empty string) +compression setting for an inode: lzo, zlib, zstd, none, or "" (empty string). Both none and "" are for disabling compression. *list* [-t ] :: Lists available properties with their descriptions for the given object. diff --git a/props.c b/props.c index a7e3e96..8d85181 100644 --- a/props.c +++ b/props.c @@ -142,9 +142,11 @@ static int prop_compression(enum prop_object_type type, memcpy(xattr_name + XATTR_BTRFS_PREFIX_LEN, name, strlen(name)); xattr_name[XATTR_BTRFS_PREFIX_LEN + strlen(name)] = '\0'; - if (value) + if (value) { + if (!strcmp(value, "none")) + value = ""; sret = fsetxattr(fd, xattr_name, value, strlen(value), 0); - else + } else sret = fgetxattr(fd, xattr_name, NULL, 0); if (sret < 0) { ret = -errno; -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: cleanup 'start' subtraction from try uncompressed inline extent
At Fri, 15 Sep 2017 01:57:26 +0300, Timofey Titovets wrote: > > Was added in: > c8b978188c9a0fd3d535c13debd19d522b726f1f > "Btrfs: Add zlib compression support" > Survive to near time (from 08.10.2008). > > Because 'start' checked for zero before branch, so it's > safe to remove that subtraction. > > Signed-off-by: Timofey Titovets <nefelim...@gmail.com> Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com> > --- > fs/btrfs/inode.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c > index 02ef32149c15..81123408e82e 100644 > --- a/fs/btrfs/inode.c > +++ b/fs/btrfs/inode.c > @@ -570,7 +570,7 @@ static noinline void compress_file_range(struct inode > *inode, > cont: > if (start == 0) { > /* lets try to make an inline extent */ > - if (ret || total_in < (actual_end - start)) { > + if (ret || total_in < actual_end) { > /* we didn't compress the entire range, try >* to make an uncompressed inline extent. >*/ > -- > 2.14.1 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs: Output time elapsed for each major tree it checked
At Mon, 11 Sep 2017 14:26:23 +0900, Qu Wenruo wrote: > > Marc reported that "btrfs check --repair" runs much faster than "btrfs > check", which is quite weird. > > This patch will add time elapsed for each major tree it checked, for > both original mode and lowmem mode, so we can have a clue what's going > wrong. > > Reported-by: Marc MERLIN> Signed-off-by: Qu Wenruo > --- > cmds-check.c | 21 +++-- > utils.h | 24 > 2 files changed, 43 insertions(+), 2 deletions(-) > > diff --git a/cmds-check.c b/cmds-check.c > index 006edbde..fee806cd 100644 > --- a/cmds-check.c > +++ b/cmds-check.c > @@ -5318,13 +5318,16 @@ static int do_check_fs_roots(struct btrfs_fs_info > *fs_info, > struct cache_tree *root_cache) > { > int ret; > + struct timer timer; > > if (!ctx.progress_enabled) > fprintf(stderr, "checking fs roots\n"); > + start_timer(); > if (check_mode == CHECK_MODE_LOWMEM) > ret = check_fs_roots_v2(fs_info); > else > ret = check_fs_roots(fs_info, root_cache); > + printf("done in %d seconds\n", stop_timer()); How about adding the prefixes to show which part of check/repair is done before showing the elaplsed time, like "checking fs roots done in XX seconds" and "checking extents done in XX seconds"? Current message, "done in XX seconds" for all parts, is hard to understand which part takes such the time, especially when something wrong happens in some parts. For example, please see the following example of repairing the image based on tests/fsck-tests/002-bad-transid/default_case.img === ... opening btrfs filesystem parent transid verify failed on 29360128 wanted 9 found 755944791 parent transid verify failed on 29360128 wanted 9 found 755944791 parent transid verify failed on 29360128 wanted 9 found 755944791 parent transid verify failed on 29360128 wanted 9 found 755944791 Ignoring transid failure done in 0 seconds ... checking free space cache cache and super generation don't match, space cache will be invalidated done in 0 seconds ... === Thanks, Satoru > > return ret; > } > @@ -11584,14 +11587,16 @@ out: > static int do_check_chunks_and_extents(struct btrfs_fs_info *fs_info) > { > int ret; > + struct timer timer; > > if (!ctx.progress_enabled) > fprintf(stderr, "checking extents\n"); > + start_timer(); > if (check_mode == CHECK_MODE_LOWMEM) > ret = check_chunks_and_extents_v2(fs_info); > else > ret = check_chunks_and_extents(fs_info); > - > + printf("done in %d seconds\n", stop_timer()); > return ret; > } > > @@ -12772,6 +12777,7 @@ int cmd_check(int argc, char **argv) > int qgroups_repaired = 0; > unsigned ctree_flags = OPEN_CTREE_EXCLUSIVE; > int force = 0; > + struct timer timer; > > while(1) { > int c; > @@ -12953,8 +12959,11 @@ int cmd_check(int argc, char **argv) > if (repair) > ctree_flags |= OPEN_CTREE_PARTIAL; > > + printf("opening btrfs filesystem\n"); > + start_timer(); > info = open_ctree_fs_info(argv[optind], bytenr, tree_root_bytenr, > chunk_root_bytenr, ctree_flags); > + printf("done in %d seconds\n", stop_timer()); > if (!info) { > error("cannot open file system"); > ret = -EIO; > @@ -13115,8 +13124,10 @@ int cmd_check(int argc, char **argv) > else > fprintf(stderr, "checking free space cache\n"); > } > + start_timer(); > ret = check_space_cache(root); > err |= !!ret; > + printf("done in %d seconds\n", stop_timer()); > if (ret) { > if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE)) > error("errors found in free space tree"); > @@ -13140,18 +13151,22 @@ int cmd_check(int argc, char **argv) > } > > fprintf(stderr, "checking csums\n"); > + start_timer(); > ret = check_csums(root); > err |= !!ret; > + printf("done in %d seconds\n", stop_timer()); > if (ret) { > error("errors found in csum tree"); > goto out; > } > > - fprintf(stderr, "checking root refs\n"); > /* For low memory mode, check_fs_roots_v2 handles root refs */ > if (check_mode != CHECK_MODE_LOWMEM) { > + fprintf(stderr, "checking root refs\n"); > + start_timer(); > ret = check_root_refs(root, _cache); > err |= !!ret; > + printf("done in %d seconds\n", stop_timer()); > if (ret) { > error("errors found in root refs"); > goto out; > @@ -13186,8 +13201,10 @@ int cmd_check(int argc, char **argv) > > if (info->quota_enabled) { > fprintf(stderr, "checking
[PATCH] btrfs-progs: allow "no" to disable compression for convenience
It's messy to use "" to disable compression. Introduce the new value "no" which can also be used for this purpose. Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com> --- Documentation/btrfs-property.asciidoc | 2 +- props.c | 6 -- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Documentation/btrfs-property.asciidoc b/Documentation/btrfs-property.asciidoc index 7ed6a7d..97b90d6 100644 --- a/Documentation/btrfs-property.asciidoc +++ b/Documentation/btrfs-property.asciidoc @@ -43,7 +43,7 @@ read-only flag of subvolume: true or false label label of device compression -compression setting for an inode: lzo, zlib, zstd, or "" (empty string) +compression setting for an inode: lzo, zlib, zstd, no, or "" (empty string). Both no and "" are for disabling compression. *list* [-t ] :: Lists available properties with their descriptions for the given object. diff --git a/props.c b/props.c index a7e3e96..a2df868 100644 --- a/props.c +++ b/props.c @@ -142,9 +142,11 @@ static int prop_compression(enum prop_object_type type, memcpy(xattr_name + XATTR_BTRFS_PREFIX_LEN, name, strlen(name)); xattr_name[XATTR_BTRFS_PREFIX_LEN + strlen(name)] = '\0'; - if (value) + if (value) { + if (!strcmp(value, "no")) + value = ""; sret = fsetxattr(fd, xattr_name, value, strlen(value), 0); - else + } else sret = fgetxattr(fd, xattr_name, NULL, 0); if (sret < 0) { ret = -errno; -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs: prevent to set invalid default subvolid
`btrfs sub set-default` succeeds to set an ID which isn't corresponding to any fs/file tree. If such the bad ID is set to a filesystem, we can't mount this filesystem without specifying `subvol` or `subvolid` mount options. Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com> --- fs/btrfs/ioctl.c | 4 1 file changed, 4 insertions(+) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index ae8fbf9..8e1bc19 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -4057,6 +4057,10 @@ static long btrfs_ioctl_default_subvol(struct file *file, void __user *argp) ret = PTR_ERR(new_root); goto out; } + if (!is_fstree(new_root->objectid)) { + ret = -ENOENT; + goto out; + } path = btrfs_alloc_path(); if (!path) { -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs: convert all mount option checking code to use btrfs_test_opt
Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com> --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/super.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index e2d7e86..ea9b0d2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -652,7 +652,7 @@ static int cache_block_group(struct btrfs_block_group_cache *cache, cache->cached = BTRFS_CACHE_FAST; spin_unlock(>lock); - if (fs_info->mount_opt & BTRFS_MOUNT_SPACE_CACHE) { + if (btrfs_test_opt(fs_info, SPACE_CACHE)) { mutex_lock(_ctl->mutex); ret = load_free_space_cache(fs_info, cache); diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 0b7a1d8..26f0a38 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -292,7 +292,7 @@ void __btrfs_panic(struct btrfs_fs_info *fs_info, const char *function, vaf.va = errstr = btrfs_decode_error(errno); - if (fs_info && (fs_info->mount_opt & BTRFS_MOUNT_PANIC_ON_FATAL_ERROR)) + if (fs_info && (btrfs_test_opt(fs_info, PANIC_ON_FATAL_ERROR))) panic(KERN_CRIT "BTRFS panic (device %s) in %s:%d: %pV (errno=%d %s)\n", s_id, function, line, , errno, errstr); -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fix inconsistent device between /proc/pid/maps and stat
2017-03-24 20:58 GMT+09:00 David Sterba <dste...@suse.cz>: > On Tue, Mar 21, 2017 at 10:53:09AM +0900, Satoru Takeuchi wrote: >> There have been some discussions about inconsistent device between >> /proc/pid/maps and stat(2). >> >> http://thr3ads.net/btrfs-devel/2011/05/2346176-RFC-PATCH-0-2-btrfs-vfs-Return-same-device-in-stat-2-and-proc-pid-maps >> https://www.spinics.net/lists/linux-btrfs/msg09044.html >> https://patchwork.kernel.org/patch/2825842/ >> https://patchwork.kernel.org/patch/2831525/ >> >> It's important since it breaks user programs like lsof(8). There was a >> patche by Mark to fix this problem. >> However, unfortunately, that patch is not merged so far. > > And no variant of the get_map_dev will ever be merged. Reworking this > requires extensions to the superblock and subvolume structures, making > it more generic and suitable for VFS. Thank you for your comment. I'm going to reconsider how to fix this problem. Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] fix inconsistent device between /proc/pid/maps and stat
There have been some discussions about inconsistent device between /proc/pid/maps and stat(2). http://thr3ads.net/btrfs-devel/2011/05/2346176-RFC-PATCH-0-2-btrfs-vfs-Return-same-device-in-stat-2-and-proc-pid-maps https://www.spinics.net/lists/linux-btrfs/msg09044.html https://patchwork.kernel.org/patch/2825842/ https://patchwork.kernel.org/patch/2831525/ It's important since it breaks user programs like lsof(8). There was a patche by Mark to fix this problem. However, unfortunately, that patch is not merged so far. NOTE: 1) This patch is inspired by the Mark's one and I tweak it as follows. - move a flag for this fix from sb->s_flags to kernel internal sb->s_iflags - change that flag's name from MS_STAT_FOR_DEV to SB_I_LOGICALVOL, to make its meaning clearer 2) This patch is for Chris's for-linus-4.11 (commit e1699d2d7bf6e6cce3e1baff19f9dd4595a58664 ("")). It should modify to apply to 4.11-rcX because of statx patches changes struct inode_operations->getattr() interface. For more information about this problem, please refer to the patchat the end of this mail. --- Subject: [PATCH] fix inconsistent devie between /proc/pid/maps and stat /proc/pid/maps returns each device as follows. === dev = inode->i_sb->s_dev; === However, stat(2) returns it as follows. === stat->dev = BTRFS_I(inode)->root->anon_dev; === It results in device mismatch and this inconsistency breaks users programs like lsof(8) as follows. Without this patch: === $ mount | grep /home/vagrant/mnt /dev/vda5 on /home/vagrant/mnt type btrfs (rw,relatime,noacl,space_cache,subvolid=5,subvol=/) $ /home/vagrant/mnt/lsof | grep /home/vagrant/mnt lsof 19292 root txt REG 0,44 163136257 /home/sat/mnt/lsof lsof 19292 root mem REG 0,42 257 /home/sat/mnt/lsof (path dev=0,44) lsof 19294 root txt REG 0,44 163136257 /home/sat/mnt/lsof lsof 19294 root mem REG 0,42 257 /home/sat/mnt/lsof (path dev=0,44) === With this patch: === $ mount | grep /home/vagrant/mnt /dev/vda5 on /home/vagrant/mnt type btrfs (rw,relatime,noacl,space_cache,subvolid=5,subvol=/) $ /home/vagrant/mnt/lsof | grep /home/vagrant/mnt lsof 752 root txt REG 0,44 163224 263 /home/vagrant/mnt/lsof lsof 754 root txt REG 0,44 163224 263 /home/vagrant/mnt/lsof === Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com> CC: Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/btrfs/super.c | 1 + fs/proc/generic.c| 13 + fs/proc/internal.h | 1 + fs/proc/nommu.c | 2 +- fs/proc/task_mmu.c | 2 +- fs/proc/task_nommu.c | 2 +- include/linux/fs.h | 1 + 7 files changed, 19 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index da687dc79cce..2ccfdb107ba0 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1133,6 +1133,7 @@ static int btrfs_fill_super(struct super_block *sb, #endif sb->s_flags |= MS_I_VERSION; sb->s_iflags |= SB_I_CGROUPWB; + sb->s_iflags |= SB_I_LOGICALVOL; err = open_ctree(sb, fs_devices, (char *)data); if (err) { btrfs_err(fs_info, "open_ctree failed"); diff --git a/fs/proc/generic.c b/fs/proc/generic.c index f6a01f09f79d..d38cd77b297c 100644 --- a/fs/proc/generic.c +++ b/fs/proc/generic.c @@ -646,3 +646,16 @@ void *PDE_DATA(const struct inode *inode) return __PDE_DATA(inode); } EXPORT_SYMBOL(PDE_DATA); + +dev_t proc_get_map_dev(struct dentry *dentry) +{ + struct inode *inode = dentry->d_inode; + struct kstat kstat; + + if (inode->i_sb->s_iflags & SB_I_LOGICALVOL && + inode->i_op->getattr && + inode->i_op->getattr(NULL, dentry, ) == 0) + return kstat.dev; + + return inode->i_sb->s_dev; +} diff --git a/fs/proc/internal.h b/fs/proc/internal.h index 2de5194ba378..bf0f11fc209b 100644 --- a/fs/proc/internal.h +++ b/fs/proc/internal.h @@ -190,6 +190,7 @@ static inline struct proc_dir_entry *pde_get(struct proc_dir_entry *pde) return pde; } extern void pde_put(struct proc_dir_entry *); +dev_t proc_get_map_dev(struct dentry *); static inline bool is_empty_pde(const struct proc_dir_entry *pde) { diff --git a/fs/proc/nommu.c b/fs/proc/nommu.c index 75634379f82e..e9c29864a50e 100644 --- a/fs/proc/nommu.c +++ b/fs/proc/nommu.c @@ -46,7 +46,7 @@ static int nommu_region_show(struct seq_file *m, struct vm_region *region) if (file) { struct inode *inode = file_inode(region->vm_file); - dev = inode->i_sb->s_dev; + dev = proc_get_map_dev(vma->vm_file
Re: [PATCH v12.1 01/15] btrfs: expand cow_file_range() to support in-band dedup and subpage-blocksize
On 2016/07/12 1:41, David Sterba wrote: > On Mon, Jul 11, 2016 at 11:05:29AM +0800, Qu Wenruo wrote: >> From: Wang Xiaoguang>> >> Extract cow_file_range() new parameters for both in-band dedupe and >> subpage sector size patchset. >> >> This should make conflict of both patchset to minimal, and reduce the >> effort needed to rebase them. > > Great, thanks. I did a test merge, there are still conflicts but they > seem to be resolvable more easily, picking the changes from the right > patchset. I haven't tested it so there's more work, but the point was to > get the conflict surface down. > > There's another candidate, btrfs_set_extent_delalloc as it adds a > parameter, can you please send a similar patch for that? He's going to attend LinuxCon Japan (Jul 13 - Jul 15). So I guess he will reply several days later (next week?). Thanks, Satoru > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs: add option to run balance as daemon
On 2016/06/22 0:16, Austin S. Hemmelgarn wrote: > Currently, balance operations are run synchronously in the foreground. > This is nice for interactive management, but is kind of crappy when you > start looking at automation and similar things. > > This patch adds an option to `btrfs balance start` to tell it to > daemonize prior to running the balance operation, thus allowing us to > preform balances asynchronously. The two biggest use cases I have for > this are starting a balance on a remote server without establishing a > full shell session, and being able to background the balance in a > recovery shell (which usually has no job control) so I can still get > progress information. > > Because it simply daemonizes prior to calling the balance ioctl, this > doesn't actually need any kernel support. > > Signed-off-by: Austin S. Hemmelgarn> --- > This works as is, but there are two specific things I would love to > eventually fix but don't have the time to fix right now: > * There is no way to get any feedback from the balance operation. > * Because of how everything works, trying to start a new balance with > --background while one iw already running won't return an error but > won't queue or start a new balance either. > > The first one is more a utility item than anything else, and probably > would not be hard to add. Ideally, it should be output to a user > specified file, and this should work even for a normal foreground balance. > > The second is very much a UX issue, but can't be easily sovled without > doing some creative process monitoring from the parrent processes. > > Documentation/btrfs-balance.asciidoc | 2 ++ > cmds-balance.c | 43 > +++- > 2 files changed, 44 insertions(+), 1 deletion(-) > > diff --git a/Documentation/btrfs-balance.asciidoc > b/Documentation/btrfs-balance.asciidoc > index 7df40b9..f487dbb 100644 > --- a/Documentation/btrfs-balance.asciidoc > +++ b/Documentation/btrfs-balance.asciidoc > @@ -85,6 +85,8 @@ act on system chunks (requires '-f'), see `FILTERS` section > for details about 'f > be verbose and print balance filter arguments > -f > force reducing of metadata integrity, eg. when going from 'raid1' to 'single' > +--background > +run the balance operation asynchronously in the background > > *status* [-v] :: > Show status of running or paused balance. > diff --git a/cmds-balance.c b/cmds-balance.c > index 708bbf4..66169b7 100644 > --- a/cmds-balance.c > +++ b/cmds-balance.c > @@ -20,6 +20,9 @@ > #include > #include > #include > +#include > +#include > +#include > #include > > #include "kerncompat.h" > @@ -510,6 +513,7 @@ static const char * const cmd_balance_start_usage[] = { > "-v be verbose", > "-f force reducing of metadata integrity", > "--full-balance do not print warning and do not delay start", > + "--background run the balance as a background process", > NULL > }; > > @@ -520,6 +524,7 @@ static int cmd_balance_start(int argc, char **argv) > , NULL }; > int force = 0; > int verbose = 0; > + int background = 0; > unsigned start_flags = 0; > int i; > > @@ -527,7 +532,8 @@ static int cmd_balance_start(int argc, char **argv) > > optind = 1; > while (1) { > - enum { GETOPT_VAL_FULL_BALANCE = 256 }; > + enum { GETOPT_VAL_FULL_BALANCE = 256, > + GETOPT_VAL_BACKGROUND = 257 }; > static const struct option longopts[] = { > { "data", optional_argument, NULL, 'd'}, > { "metadata", optional_argument, NULL, 'm' }, > @@ -536,6 +542,8 @@ static int cmd_balance_start(int argc, char **argv) > { "verbose", no_argument, NULL, 'v' }, > { "full-balance", no_argument, NULL, > GETOPT_VAL_FULL_BALANCE }, > + { "background", no_argument, NULL, > + GETOPT_VAL_BACKGROUND }, > { NULL, 0, NULL, 0 } > }; > > @@ -574,6 +582,9 @@ static int cmd_balance_start(int argc, char **argv) > case GETOPT_VAL_FULL_BALANCE: > start_flags |= BALANCE_START_NOWARN; > break; > + case GETOPT_VAL_BACKGROUND: > + background = 1; > + break; > default: > usage(cmd_balance_start_usage); > } > @@ -626,6 +637,36 @@ static int cmd_balance_start(int argc, char **argv) > args.flags |= BTRFS_BALANCE_FORCE; > if (verbose) > dump_ioctl_balance_args(); > + if (background) { > + switch (fork()) { > + case (-1): > + error("Unable to fork to run balance in
[PATCH] btrfs-progs: dedupe-inband enable/reconfigure: force option does not take argument
--- This patch can be applied to integration-20160704(2355a7e5dcdf122d1924) --- cmds-dedupe-ib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-dedupe-ib.c b/cmds-dedupe-ib.c index 342784c..dbb30ab 100644 --- a/cmds-dedupe-ib.c +++ b/cmds-dedupe-ib.c @@ -132,7 +132,7 @@ static int enable_reconfig_dedupe(int argc, char **argv, int reconf) { "hash-algorithm", required_argument, NULL, 'a'}, { "limit-hash", required_argument, NULL, 'l'}, { "limit-memory", required_argument, NULL, 'm'}, - { "force", required_argument, NULL, 'f'}, + { "force", no_argument, NULL, 'f'}, { NULL, 0, NULL, 0} }; -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: fi show: print error message if no valid Btrfs is specified
* Before this patch === # ./btrfs fi show foo # "foo" doesn't mean any valid Btrfs # # no error message # echo $? 1 === * After this patch === # ./btrfs fi show foo ERROR: foo is not a valid Btrfs # # echo $? 1 === Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-filesystem.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/cmds-filesystem.c b/cmds-filesystem.c index 88867a3..90f3c49 100644 --- a/cmds-filesystem.c +++ b/cmds-filesystem.c @@ -898,9 +898,10 @@ devs_only: list_for_each_entry(fs_devices, _uuids, list) print_one_uuid(fs_devices, unit_mode); - if (search && !found) + if (search && !found) { + error("%s is not a valid Btrfs", search); ret = 1; - + } while (!list_empty(_uuids)) { fs_devices = list_entry(all_uuids.next, struct btrfs_fs_devices, list); -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v11 00/13] Btrfs dedupe framework
disk backend bug >> Add handle for multiple hash on same bytenr corner case to fix abort >> trans error >> Increase dedup rate by enhancing delayed ref handler for both backend. >> Move dedup_add() to run_delayed_ref() time, to fix abort trans error. >> Increase dedup block size up limit to 8M. >> v4: >> Add dedup prop for disabling dedup for given files/dirs. >> Merge inmem_search() and ondisk_search() into generic_search() to save >> some code >> Fix another delayed_ref related bug. >> Use the same mutex for both inmem and ondisk backend. >> Move dedup_add() back to btrfs_finish_ordered_io() to increase dedup >> rate. >> v5: >> Reuse compress routine for much simpler dedup function. >> Slightly improved performance due to above modification. >> Fix race between dedup enable/disable >> Fix for false ENOSPC report >> v6: >> Further enable/disable race window fix. >> Minor format change according to checkpatch. >> v7: >> Fix one concurrency bug with balance. >> Slightly modify return value from -EINVAL to -EOPNOTSUPP for >> btrfs_dedup_ioctl() to allow progs to distinguish unsupported commands >> and wrong parameter. >> Rebased to integration-4.6. >> v8: >> Rename 'dedup' to 'dedupe'. >> Add support to allow dedupe and compression work at the same time. >> Fix several balance related bugs. Special thanks to Satoru Takeuchi, >> who exposed most of them. >> Small dedupe hit case performance improvement. >> v9: >> Re-order the patchset to completely separate pure in-memory and any >> on-disk format change. >> Fold bug fixes into its original patch. >> v10: >> Adding back missing bug fix patch. >> Reduce on-disk item size. >> Hide dedupe ioctl under CONFIG_BTRFS_DEBUG. >> v11: >> Remove other backend and props support to focus on the framework and >> in-memory backend. Suggested by David. >> Better disable and buffered write race protection. >> Comprehensive fix to dedupe metadata ENOSPC problem. >> >> Qu Wenruo (3): >> btrfs: delayed-ref: Add support for increasing data ref under spinlock >> btrfs: dedupe: Inband in-memory only de-duplication implement >> btrfs: relocation: Enhance error handling to avoid BUG_ON >> >> Wang Xiaoguang (10): >> btrfs: dedupe: Introduce dedupe framework and its header >> btrfs: dedupe: Introduce function to initialize dedupe info >> btrfs: dedupe: Introduce function to add hash into in-memory tree >> btrfs: dedupe: Introduce function to remove hash from in-memory tree >> btrfs: dedupe: Introduce function to search for an existing hash >> btrfs: dedupe: Implement btrfs_dedupe_calc_hash interface >> btrfs: ordered-extent: Add support for dedupe >> btrfs: dedupe: Add ioctl for inband dedupelication >> btrfs: improve inode's outstanding_extents computation >> btrfs: dedupe: fix false ENOSPC >> >> fs/btrfs/Makefile | 2 +- >> fs/btrfs/ctree.h| 25 +- >> fs/btrfs/dedupe.c | 710 >> >> fs/btrfs/dedupe.h | 210 + >> fs/btrfs/delayed-ref.c | 30 +- >> fs/btrfs/delayed-ref.h | 8 + >> fs/btrfs/disk-io.c | 4 + >> fs/btrfs/extent-tree.c | 83 +- >> fs/btrfs/extent_io.c| 63 +++- >> fs/btrfs/extent_io.h| 15 +- >> fs/btrfs/file.c | 26 +- >> fs/btrfs/free-space-cache.c | 5 +- >> fs/btrfs/inode-map.c| 4 +- >> fs/btrfs/inode.c| 434 ++- >> fs/btrfs/ioctl.c| 80 - >> fs/btrfs/ordered-data.c | 46 ++- >> fs/btrfs/ordered-data.h | 14 + >> fs/btrfs/relocation.c | 46 ++- >> fs/btrfs/sysfs.c| 2 + >> include/uapi/linux/btrfs.h | 41 +++ >> 20 files changed, 1701 insertions(+), 147 deletions(-) >> create mode 100644 fs/btrfs/dedupe.c >> create mode 100644 fs/btrfs/dedupe.h >> > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Confusing output from fi us/df
On 2016/06/21 8:30, Marc Grondin wrote: > Hi everyone, > > > I have a btrfs filesystem ontop of a 4x1tb mdraid raid5 array and I've > been getting confusing output on metadata usage. Seems that even tho > both data and metadata are in single profile metadata is reporting > double the space(as if it was in dupe profile) > > > > root@thebeach /h/marcg> uname -a > Linux thebeach 4.6.2-gentoo-GMAN #1 SMP Sat Jun 11 22:32:27 ADT 2016 > x86_64 Intel(R) Core(TM) i5-2320 CPU @ 3.00GHz GenuineIntel GNU/Linux > root@thebeach /h/marcg> btrfs --version > btrfs-progs v4.5.3 > root@thebeach /h/marcg> btrfs fi us /media/Storage2 > Overall: > Device size: 2.73TiB > Device allocated: 1.71TiB > Device unallocated: 1.02TiB > Device missing: 0.00B > Used: 1.38TiB > Free (estimated): 1.34TiB (min: 1.34TiB) > Data ratio: 1.00 > Metadata ratio: 1.00 > Global reserve: 512.00MiB (used: 0.00B) > > > Data,single: Size:1.71TiB, Used:1.38TiB > /dev/mapper/storage2 1.71TiB > > > Metadata,single: Size:3.00GiB, Used:1.53GiB > /dev/mapper/storage2 3.00GiB > > > System,single: Size:32.00MiB, Used:208.00KiB > /dev/mapper/storage2 32.00MiB > > > Unallocated: > /dev/mapper/storage2 1.02TiB > root@thebeach /h/marcg> btrfs fi df /media/Storage2 > Data, single: total=1.71TiB, used=1.38TiB > System, single: total=32.00MiB, used=208.00KiB > Metadata, single: total=3.00GiB, used=1.53GiB > GlobalReserve, single: total=512.00MiB, used=0.00B > root@thebeach /h/marcg> > > > I'm not sure if this is known and if it's btrfs-progs related or if it > is actually allocating that space. Could you tell me the location where you think metadata is reporting double the space? from fi us: > Metadata,single: Size:3.00GiB, Used:1.53GiB > /dev/mapper/storage2 3.00GiB from fi df: > Metadata,single: Size:3.00GiB, Used:1.53GiB > /dev/mapper/storage2 3.00GiB As far as I can see, Btrfs just allocates 3.00 GiB from /dev/mapper/storage2, Metadata,single size is the same as it (not double), and 1.53 GiB is used. The following is in my case where data is single and meta is dup. from fi us: Metadata,DUP: Size:384.00MiB, Used:221.36MiB /dev/vda3 768.00MiB from fi df: Metadata, DUP: total=384.00MiB, used=221.36MiB Here Btrfs allocates 768.0MiB from /dev/vda3 and it's twice as large as the size of Metadata,DUP(384.00MiB). I guess it means that "metadata is reporting double the space" as you said and your case it not the case. CMIIW. Thanks, Satoru > > > Thank you for reading. > > > Marc > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: Initialize stripesize to the value of sectorsize
On 2016/06/17 1:38, Chandan Rajendra wrote: > stripesize should ideally be set to the value of sectorsize. However > previous versions of btrfs-progs/mkfs.btrfs had set stripesize to a > value of 4096. On machines with PAGE_SIZE other than 4096, This could > lead to the following scenario, > > - /dev/loop0, /dev/loop1 and /dev/loop2 are mounted as a single > filesystem. The filesystem was created by an older version of mkfs.btrfs > which set stripesize to 4k. > - losetup -a >/dev/loop0: [0030]:19477 (/root/disk-imgs/file-0.img) >/dev/loop1: [0030]:16577 (/root/disk-imgs/file-1.img) >/dev/loop2: [64770]:3423229 (/root/disk-imgs/file-2.img) > - /etc/mtab lists only /dev/loop0 > - losetup /dev/loop4 /root/disk-imgs/file-1.img > The new mkfs.btrfs invoked as 'mkfs.btrfs -f /dev/loop4' succeeds even > though /dev/loop1 has already been mounted and has > /root/disk-imgs/file-1.img as its backing file. > > The above behaviour occurs because check_super() function returns an > error code (due to stripesize not being set to 4096) and hence > check_mounted_where() function treats /dev/loop1 as a disk containing a > filesystem other than Btrfs. > > Hence as a workaround this commit allows 4096 as a valid stripesize. > > Signed-off-by: Chandan Rajendra> --- > disk-io.c | 4 +++- > mkfs.c| 1 + > 2 files changed, 4 insertions(+), 1 deletion(-) > > diff --git a/disk-io.c b/disk-io.c > index 77eb0a6..1ac7631 100644 > --- a/disk-io.c > +++ b/disk-io.c > @@ -1476,7 +1476,9 @@ static int check_super(struct btrfs_super_block *sb) > error("invalid bytes_used %llu", btrfs_super_bytes_used(sb)); > goto error_out; > } > - if (btrfs_super_stripesize(sb) != 4096) { > + > +if ((btrfs_super_stripesize(sb) != 4096) Just one trivial comment. Tab was converted to some spaces. Thanks, Satoru > + && (btrfs_super_stripesize(sb) != btrfs_super_sectorsize(sb))) { > error("invalid stripesize %u", btrfs_super_stripesize(sb)); > goto error_out; > } > diff --git a/mkfs.c b/mkfs.c > index a3a3c14..697bdc2 100644 > --- a/mkfs.c > +++ b/mkfs.c > @@ -1482,6 +1482,7 @@ int main(int argc, char **argv) > } > > sectorsize = max(sectorsize, (u32)sysconf(_SC_PAGESIZE)); > + stripesize = sectorsize; > saved_optind = optind; > dev_cnt = argc - optind; > if (dev_cnt == 0) > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs: doc: correct the destination of btrfs-receive
On 2016/06/14 18:16, Hugo Mills wrote: > On Tue, Jun 14, 2016 at 10:51:33AM +0200, David Sterba wrote: >> On Tue, Jun 14, 2016 at 02:50:19PM +0900, Satoru Takeuchi wrote: >>> We can set not only btrfs mount point but also any path belong to >>> btrfs mount point as btrfs-receive's destination. >>> >>> Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> >> >> The patches from you have a consistent whitespace damage, I've fixed >> the btrfs-crc but now that I see it again I suspect some error on your >> side. The problem is on my side. I'm sorry. >> >>> @@ -7,14 +7,14 @@ btrfs-receive - receive subvolumes from send stream >>> >>> SYNOPSIS >>> >>> -*btrfs receive* [options] >>> +*btrfs receive* [options] >>> >>> DESCRIPTION >>> --- >>> >>> Receive a stream of changes and replicate one or more subvolumes that were >>> previously used with *btrfs send* The received subvolumes are stored to >>> -'mount'. >>> +'path'. >>> >>> *btrfs receive* will fail int the following cases: >>> >>> @@ -37,7 +37,7 @@ by default, btrfs receive uses standard input to receive >>> the stream, >>> use this option to read from a file instead >>> >>> -C|--chroot:: >>> -confine the process to 'mount' using `chroot`(1) >>> +confine the process to 'path' using `chroot`(1) >>> >>> -e:: >>> terminate after receiving an 'end cmd' marker in the stream. >> >> ie. all the context lines start with two spaces instead of one. I'll >> apply this patch manually but please have a look. > >Looking at this, I suspect it's a consequence of sending it as > "Content-Type: format=flowed; delsp=yes". I'm not sure which of those > two options is the culprit. When I look at the message in my client > (mutt), it looks absolutely fine. When I pipe it to hexdump, the > double-spacing is apparent. You're right. These are added to charset="iso-2022-jp" plain text mail since thunderbird 45. I disabled the setting that appends the above mentioned options. So, probably the following sample patch doesn't have strange spaces. === We can set not only btrfs mount points but also any paths belong to btrfs mount point as btrfs-receive's destination. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- Documentation/btrfs-receive.asciidoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Documentation/btrfs-receive.asciidoc b/Documentation/btrfs-receive.asciidoc index fbbded2..e246603 100644 --- a/Documentation/btrfs-receive.asciidoc +++ b/Documentation/btrfs-receive.asciidoc @@ -7,14 +7,14 @@ btrfs-receive - receive subvolumes from send stream SYNOPSIS -*btrfs receive* [options] +*btrfs receive* [options] DESCRIPTION --- Receive a stream of changes and replicate one or more subvolumes that were previously used with *btrfs send* The received subvolumes are stored to -'mount'. +'path'. *btrfs receive* will fail int the following cases: @@ -37,7 +37,7 @@ by default, btrfs receive uses standard input to receive the stream, use this option to read from a file instead -C|--chroot:: -confine the process to 'mount' using `chroot`(1) +confine the process to 'path' using `chroot`(1) -e:: terminate after receiving an 'end cmd' marker in the stream. -- 2.5.5 === Thanks, Satoru > >Hugo. > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: doc: correct the destination of btrfs-receive
We can set not only btrfs mount point but also any path belong to btrfs mount point as btrfs-receive's destination. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- Documentation/btrfs-receive.asciidoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Documentation/btrfs-receive.asciidoc b/Documentation/btrfs-receive.asciidoc index fbbded2..e246603 100644 --- a/Documentation/btrfs-receive.asciidoc +++ b/Documentation/btrfs-receive.asciidoc @@ -7,14 +7,14 @@ btrfs-receive - receive subvolumes from send stream SYNOPSIS -*btrfs receive* [options] +*btrfs receive* [options] DESCRIPTION --- Receive a stream of changes and replicate one or more subvolumes that were previously used with *btrfs send* The received subvolumes are stored to -'mount'. +'path'. *btrfs receive* will fail int the following cases: @@ -37,7 +37,7 @@ by default, btrfs receive uses standard input to receive the stream, use this option to read from a file instead -C|--chroot:: -confine the process to 'mount' using `chroot`(1) +confine the process to 'path' using `chroot`(1) -e:: terminate after receiving an 'end cmd' marker in the stream. -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/5] btrfs-progs: btrfs-crc: make argc check more strict
Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- btrfs-crc.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/btrfs-crc.c b/btrfs-crc.c index c2b5f00..d433ff3 100644 --- a/btrfs-crc.c +++ b/btrfs-crc.c @@ -69,12 +69,14 @@ int main(int argc, char **argv) str = argv[optind]; if (!loop) { - if (check_argc_min(argc - optind, 1)) + if (check_argc_exact(argc - optind, 1)) print_usage(255); printf("%12u - %s\n", crc32c(~1, str, strlen(str)), str); return 0; } + if (check_argc_exact(argc - optind, 0)) + print_usage(255); buf = malloc(length); if (!buf) -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/5] btrfs-progs: btrfs-crc: improve usage message
- If -c is set, filename argument is ignored. - Describe about -h option Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- btrfs-crc.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/btrfs-crc.c b/btrfs-crc.c index 55a4c61..c2b5f00 100644 --- a/btrfs-crc.c +++ b/btrfs-crc.c @@ -26,10 +26,12 @@ void print_usage(int status) { printf("usage: btrfs-crc filename\n"); printf("print out the btrfs crc for \"filename\"\n"); - printf("usage: btrfs-crc filename -c crc [-s seed] [-l length]\n"); + printf("usage: btrfs-crc -c crc [-s seed] [-l length]\n"); printf("brute force search for file names with the given crc\n"); printf(" -s seedthe random seed (default: random)\n"); printf(" -l length the length of the file names (default: 10)\n"); + printf("usage: btrfs-crc -h\n"); + printf("print this message\n"); exit(status); } -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/5] btrfs-progs: btrfs-crc: print usage on receiving invalid arguments
Usage is only printed if -h option is set. However it's nice to do it when wrong option is set or the number of argument is wrong. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- btrfs-crc.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/btrfs-crc.c b/btrfs-crc.c index 86dfe05..55a4c61 100644 --- a/btrfs-crc.c +++ b/btrfs-crc.c @@ -22,7 +22,7 @@ #include "crc32c.h" #include "utils.h" -void print_usage(void) +void print_usage(int status) { printf("usage: btrfs-crc filename\n"); printf("print out the btrfs crc for \"filename\"\n"); @@ -30,7 +30,7 @@ void print_usage(void) printf("brute force search for file names with the given crc\n"); printf(" -s seedthe random seed (default: random)\n"); printf(" -l length the length of the file names (default: 10)\n"); - exit(1); + exit(status); } int main(int argc, char **argv) @@ -57,9 +57,9 @@ int main(int argc, char **argv) seed = atol(optarg); break; case 'h': - print_usage(); + print_usage(1); case '?': - return 255; + print_usage(255); } } @@ -68,7 +68,7 @@ int main(int argc, char **argv) if (!loop) { if (check_argc_min(argc - optind, 1)) - return 255; + print_usage(255); printf("%12u - %s\n", crc32c(~1, str, strlen(str)), str); return 0; -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/5] btrfs-progs: btrfs-crc should be ignored by git
It's a binary built from btrfs-crc.c Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index a27cb0d..aaf9702 100644 --- a/.gitignore +++ b/.gitignore @@ -33,6 +33,7 @@ btrfs-zero-log btrfs-corrupt-block btrfs-select-super btrfs-calc-size +btrfs-crc btrfstune libbtrfs.a libbtrfs.so -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/5] btrfs-progs: btrfs-crc: fix build error
Remove the following build error. $ make btrfs-crc [CC] btrfs-crc.o [LD] btrfs-crc btrfs-crc.o: In function `usage': /home/sat/src/btrfs-progs/btrfs-crc.c:26: multiple definition of `usage' help.o:/home/sat/src/btrfs-progs/help.c:125: first defined here collect2: error: ld returned 1 exit status Makefile:294: recipe for target 'btrfs-crc' failed make: *** [btrfs-crc] Error 1 = Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- btrfs-crc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-crc.c b/btrfs-crc.c index 723e0b7..86dfe05 100644 --- a/btrfs-crc.c +++ b/btrfs-crc.c @@ -22,7 +22,7 @@ #include "crc32c.h" #include "utils.h" -void usage(void) +void print_usage(void) { printf("usage: btrfs-crc filename\n"); printf("print out the btrfs crc for \"filename\"\n"); @@ -57,7 +57,7 @@ int main(int argc, char **argv) seed = atol(optarg); break; case 'h': - usage(); + print_usage(); case '?': return 255; } -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Very quick embarrassing q: /var/lib permissions.
On 2016/06/02 10:20, al wrote: Satoru Takeuchi jp.fujitsu.com> writes: Thank you, sir! I wonder if you would let me have the permissions (only) of any of the files you have inside your equivalent directory. Here it is. === # ls -l /var/lib/btrfs/ total 4 -rw---. 1 root root 413 Jun 1 08:19 scrub.status. === Effectively I'm asking (for evidence) if I'm using the command with escalated permissions when this is unnecessary. I wonder why you ask these question. Probably it's enough to read the file permissions of your system... Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Very quick embarrassing q: /var/lib permissions.
On 2016/06/01 6:53, al wrote: Please can someone run: # ls -l /var/lib/ | grep btrfs for me (and post to directly or via list as they think fit). $ ls -l /var/lib | grep btrfs drwxr-xr-x 1 root root 196 May 18 10:34 btrfs This directory will be created when you run "btrfs scrub start" for the first time. Thanks, Satoru Thank you. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: About in-band dedupe for v4.7
On 2016/05/11 10:40, Qu Wenruo wrote: Chris Mason wrote on 2016/05/10 20:37 -0400: On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: Hi, Chris, Josef and David, As merge window for v4.7 is coming, it would be good to hear your ideas about the inband dedupe. We are addressing the ENOSPC problem which Josef pointed out, and we believe the final fix patch would come out at the beginning of the merge window.(Next week) If it's fine, would you please consider to merge the in-memory backend patchset for v4.7 as an experimental feature? Most of the patch won't be changed from v10 patchset, only ENOSPC fix will be updated, and ioctl patchset will introduce a new Kconfig option of "btrfs experimental features" for inband dedupe. (With explain about unstable ioctl/on-disk format for experimental features) If you are all OK to merge inband dedupe in-memory backend, I'll prepare the new v11 patchset for this merge. We have to balance the part where we really want the features to come in, and we want to lower the load on you to continue porting them. But, I really do agree that we need strong test suites included with every major feature like this. -chris That's fine. We're running all generic and btrfs test case with dedupe enabled, > by modifying xfstest to call "btrfs dedeup enable" just after mount, to ensure dedupe won't corrupt any existing test case. How about writing the xfstests patch that we can run the above tests without modifying xfstests by hand? Then it becomes easy for everyone to test dedupe. Thanks, Satoru And we are also enriching our dedupe test cases to reduce the stability concern on the new feature. BTW, does it mean inband dedup won't catch v4.7 merge window and no need to submit a v11 patchset for this merge window? Thanks, Qu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] btrfs-progs: prop: remove conditions which never be satisfied
parse_args() always set at least one parameter, 'object', for {get,list} subcommands. In addition, it always set all three parameters, 'object', 'name', and 'value' for set subcommand. So the following conditions can be removed. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-property.c | 12 1 file changed, 12 deletions(-) diff --git a/cmds-property.c b/cmds-property.c index 46be8f3..e59882b 100644 --- a/cmds-property.c +++ b/cmds-property.c @@ -385,10 +385,6 @@ static int cmd_property_get(int argc, char **argv) parse_args(argc, argv, cmd_property_get_usage, , , , NULL, 1); - if (!object) { - error("invalid arguments"); - usage(cmd_property_get_usage); - } if (name) ret = setget_prop(types, object, name, NULL); @@ -416,10 +412,6 @@ static int cmd_property_set(int argc, char **argv) parse_args(argc, argv, cmd_property_set_usage, , , , , 3); - if (!object || !name || !value) { - error("invalid arguments"); - usage(cmd_property_set_usage); - } ret = setget_prop(types, object, name, value); @@ -442,10 +434,6 @@ static int cmd_property_list(int argc, char **argv) parse_args(argc, argv, cmd_property_list_usage, , , NULL, NULL, 1); - if (!object) { - error("invalid arguments"); - usage(cmd_property_list_usage); - } ret = dump_props(types, object, 1); -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3] btrfs-progs: prop: simplify parse_args()
Since parameter is mandatory for all subcommands, 'object' is always set by parse_args()'s callers. In addition, on setting '*name' and '*value', if 'optind < argc' is satisfied here, they are always set by parse_args()'s callers. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-property.c | 13 + 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/cmds-property.c b/cmds-property.c index 48a8945..46be8f3 100644 --- a/cmds-property.c +++ b/cmds-property.c @@ -298,7 +298,7 @@ static void parse_args(int argc, char **argv, { int ret; char *type_str = NULL; - int max_nonopt_args = 0; + int max_nonopt_args = 1; optind = 1; while (1) { @@ -315,8 +315,6 @@ static void parse_args(int argc, char **argv, } } - if (object) - max_nonopt_args++; if (name) max_nonopt_args++; if (value) @@ -345,14 +343,13 @@ static void parse_args(int argc, char **argv, } } - if (object && optind < argc) - *object = argv[optind++]; - if (name && optind < argc) + *object = argv[optind++]; + if (optind < argc) *name = argv[optind++]; - if (value && optind < argc) + if (optind < argc) *value = argv[optind++]; - if (!*types && object && *object) { + if (!*types) { ret = autodetect_object_types(*object, types); if (ret < 0) { error("failed to detect object type: %s", -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/3] btrfs-progs: prop: convert error messages to use error()
props.c uses 'fprintf(stderr, "ERROR: ...")' as its error messages, however we have generic error() function. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- props.c | 21 + 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/props.c b/props.c index 5b74932..5b4e26e 100644 --- a/props.c +++ b/props.c @@ -48,7 +48,7 @@ static int prop_read_only(enum prop_object_type type, fd = open(object, O_RDONLY); if (fd < 0) { ret = -errno; - fprintf(stderr, "ERROR: open %s failed. %s\n", + error("open %s failed. %s", object, strerror(-ret)); goto out; } @@ -56,7 +56,7 @@ static int prop_read_only(enum prop_object_type type, ret = ioctl(fd, BTRFS_IOC_SUBVOL_GETFLAGS, ); if (ret < 0) { ret = -errno; - fprintf(stderr, "ERROR: failed to get flags for %s. %s\n", + error("failed to get flags for %s. %s", object, strerror(-ret)); goto out; } @@ -76,14 +76,14 @@ static int prop_read_only(enum prop_object_type type, flags = flags & ~BTRFS_SUBVOL_RDONLY; } else { ret = -EINVAL; - fprintf(stderr, "ERROR: invalid value for property.\n"); + error("invalid value for property."); goto out; } ret = ioctl(fd, BTRFS_IOC_SUBVOL_SETFLAGS, ); if (ret < 0) { ret = -errno; - fprintf(stderr, "ERROR: failed to set flags for %s. %s\n", + error("failed to set flags for %s. %s", object, strerror(-ret)); goto out; } @@ -130,8 +130,7 @@ static int prop_compression(enum prop_object_type type, fd = open_file_or_dir3(object, , open_flags); if (fd == -1) { ret = -errno; - fprintf(stderr, "ERROR: open %s failed. %s\n", - object, strerror(-ret)); + error("open %s failed. %s", object, strerror(-ret)); goto out; } @@ -151,9 +150,8 @@ static int prop_compression(enum prop_object_type type, if (sret < 0) { ret = -errno; if (ret != -ENOATTR) - fprintf(stderr, - "ERROR: failed to %s compression for %s. %s\n", - value ? "set" : "get", object, strerror(-ret)); + error("failed to %s compression for %s. %s", + value ? "set" : "get", object, strerror(-ret)); else ret = 0; goto out; @@ -169,9 +167,8 @@ static int prop_compression(enum prop_object_type type, sret = fgetxattr(fd, xattr_name, buf, len); if (sret < 0) { ret = -errno; - fprintf(stderr, - "ERROR: failed to get compression for %s. %s\n", - object, strerror(-ret)); + error("failed to get compression for %s. %s", + object, strerror(-ret)); goto out; } fprintf(stdout, "compression=%.*s\n", (int)len, buf); -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question: raid1 behaviour on failure
On 2016/04/23 16:07, Matthias Bodenbinder wrote: Here is my newest test. The backports provide a 4.5 kernel: kernel: 4.5.0-0.bpo.1-amd64 btrfs-tools: 4.4-1~bpo8+1 This time the raid1 is automatically unmounted after I unplug the device and it can not be mounted while the device is missing. See below. Matthias As I said before, I consider this problem is not caused by Btrfs, but by hardware. Please see the following comments. 1) turn on the Fantec case: Apr 23 08:45:38 rakete kernel: usb 3-1: new SuperSpeed USB device number 2 using xhci_hcd Apr 23 08:45:38 rakete kernel: usb 3-1: New USB device found, idVendor=152d, idProduct=0567 Apr 23 08:45:38 rakete kernel: usb 3-1: New USB device strings: Mfr=10, Product=11, SerialNumber=5 Apr 23 08:45:38 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge Apr 23 08:45:38 rakete kernel: usb 3-1: Manufacturer: JMicron Apr 23 08:45:38 rakete kernel: usb 3-1: SerialNumber: 152D00539000 Apr 23 08:45:38 rakete mtp-probe[3641]: checking bus 3, device 2: "/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1" Apr 23 08:45:38 rakete mtp-probe[3641]: bus: 3, device: 2 was not an MTP device Apr 23 08:45:38 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device detected Apr 23 08:45:38 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d pid 0567: 500 Apr 23 08:45:38 rakete kernel: scsi host8: usb-storage 3-1:1.0 Apr 23 08:45:38 rakete kernel: usbcore: registered new interface driver usb-storage Apr 23 08:45:38 rakete kernel: usbcore: registered new interface driver uas Apr 23 08:45:39 rakete kernel: scsi 8:0:0:0: Direct-Access WDC WD20 02FAEX-007BA00125 PQ: 0 ANSI: 6 Apr 23 08:45:39 rakete kernel: scsi 8:0:0:1: Direct-Access WDC WD75 00AACS-00C7B00125 PQ: 0 ANSI: 6 Apr 23 08:45:39 rakete kernel: scsi 8:0:0:2: Direct-Access WDC WD50 01AALS-00L3B20125 PQ: 0 ANSI: 6 Apr 23 08:45:39 rakete kernel: scsi 8:0:0:3: Direct-Access SAMSUNG SP2504C 0125 PQ: 0 ANSI: 6 Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: Attached scsi generic sg6 type 0 Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB) Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: Attached scsi generic sg7 type 0 Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Write Protect is off Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Mode Sense: 67 00 10 08 Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: Attached scsi generic sg8 type 0 Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] 1465149168 512-byte logical blocks: (750 GB/699 GiB) Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Write Protect is off Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Mode Sense: 67 00 10 08 Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] 976773168 512-byte logical blocks: (500 GB/466 GiB) Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: Attached scsi generic sg9 type 0 Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] No Caching mode page found Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Assuming drive cache: write through Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] Write Protect is off Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] Mode Sense: 67 00 10 08 Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] 488395055 512-byte logical blocks: (250 GB/233 GiB) Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] No Caching mode page found Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Assuming drive cache: write through Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] Write Protect is off Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] Mode Sense: 67 00 10 08 Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] No Caching mode page found Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] Assuming drive cache: write through Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] No Caching mode page found Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] Assuming drive cache: write through Apr 23 08:45:39 rakete kernel: sdf: sdf1 Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Attached SCSI disk Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Attached SCSI disk Apr 23 08:45:40 rakete kernel: sd 8:0:0:2: [sdh] Attached SCSI disk Apr 23 08:45:40 rakete kernel: sd 8:0:0:3: [sdi] Attached SCSI disk When you turned on Fantec case, four disks, WD20(sdf), WD75(sdg), WD50(sgh), and SP2504C(sgi) were attached. It's a matter of course. Apr 23 08:45:40 rakete kernel: BTRFS: device fsid 16d5891f-5d52-4b29-8591-588ddf11e73d devid 1 transid 89 /dev/sdg Apr 23 08:45:40 rakete kernel: BTRFS: device fsid 16d5891f-5d52-4b29-8591-588ddf11e73d devid 2 transid 89 /dev/sdh Apr 23 08:45:40 rakete kernel: BTRFS: device fsid 16d5891f-5d52-4b29-8591-588ddf11e73d devid 3 transid 89 /dev/sdi Apr 23 08:45:40 rakete kernel: EXT4-fs (sdf1): mounted filesystem with ordered data mode. Opts: (null) Apr 23 08:45:40 rakete udisksd[2422]: Mounted /dev/sdf1 at /media/matthias/BACKUP on behalf of uid 1000 7# mount /mnt/raid1/
Re: Question: raid1 behaviour on failure
On 2016/04/22 14:32, Qu Wenruo wrote: Satoru Takeuchi wrote on 2016/04/22 11:21 +0900: On 2016/04/21 20:58, Qu Wenruo wrote: On 04/21/2016 03:45 PM, Satoru Takeuchi wrote: On 2016/04/21 15:23, Satoru Takeuchi wrote: On 2016/04/20 14:17, Matthias Bodenbinder wrote: Am 18.04.2016 um 09:22 schrieb Qu Wenruo: BTW, it would be better to post the dmesg for better debug. So here we. I did the same test again. Here is a full log of what i did. It seems to be mean like a bug in btrfs. Sequenz of events: 1. mount the raid1 (2 disc with different size) 2. unplug the biggest drive (hotplug) 3. try to copy something to the degraded raid1 4. plugin the device again (hotplug) This scenario does not work. The disc array is NOT redundant! I can not work with it while a drive is missing and I can not reattach the device so that everything works again. The btrfs module crashes during the test. I am using LMDE2 with backports: btrfs-tools 4.4-1~bpo8+1 linux-image-4.4.0-0.bpo.1-amd64 Matthias rakete - root - /root 1# mount /mnt/raid1/ Journal: Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto defrag Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space caching is enabled Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents rakete - root - /mnt/raid1 3# ll insgesamt 0 drwxrwxr-x 1 root root 36 Nov 14 2014 AfterShot2(64-bit) drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc drwxr-xr-x 1 root root 108 Mär 24 07:31 var 4# btrfs fi show Label: none uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d Total devices 3 FS bytes used 1.60GiB devid1 size 698.64GiB used 3.03GiB path /dev/sdg devid2 size 465.76GiB used 3.03GiB path /dev/sdh devid3 size 232.88GiB used 0.00B path /dev/sdi unplug device sdg: Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8. Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info about processes that Apr 20 07:03:05 rakete umount[16405]: use the device is found by lsof(8) or fuser(1).) Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process exited, code=exited status=32 Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1. Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device number 3 using xhci_hcd Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found, idVendor=152d, idProduct=0567 Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings: Mfr=10, Product=11, SerialNumber=5 Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000 Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device detected Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d pid 0567: 500 Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0 Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3: "/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1" Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an MTP device Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC WD20 02FAEX-007BA00125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC WD50 01AALS-00L3B20125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access SAMSUNG SP2504C 0125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte logical blocks: (500 GB/466 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page found Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive cache: write through Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte logical blocks: (250 GB/233 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page found
Re: Question: raid1 behaviour on failure
On 2016/04/21 20:58, Qu Wenruo wrote: On 04/21/2016 03:45 PM, Satoru Takeuchi wrote: On 2016/04/21 15:23, Satoru Takeuchi wrote: On 2016/04/20 14:17, Matthias Bodenbinder wrote: Am 18.04.2016 um 09:22 schrieb Qu Wenruo: BTW, it would be better to post the dmesg for better debug. So here we. I did the same test again. Here is a full log of what i did. It seems to be mean like a bug in btrfs. Sequenz of events: 1. mount the raid1 (2 disc with different size) 2. unplug the biggest drive (hotplug) 3. try to copy something to the degraded raid1 4. plugin the device again (hotplug) This scenario does not work. The disc array is NOT redundant! I can not work with it while a drive is missing and I can not reattach the device so that everything works again. The btrfs module crashes during the test. I am using LMDE2 with backports: btrfs-tools 4.4-1~bpo8+1 linux-image-4.4.0-0.bpo.1-amd64 Matthias rakete - root - /root 1# mount /mnt/raid1/ Journal: Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto defrag Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space caching is enabled Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents rakete - root - /mnt/raid1 3# ll insgesamt 0 drwxrwxr-x 1 root root 36 Nov 14 2014 AfterShot2(64-bit) drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc drwxr-xr-x 1 root root 108 Mär 24 07:31 var 4# btrfs fi show Label: none uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d Total devices 3 FS bytes used 1.60GiB devid1 size 698.64GiB used 3.03GiB path /dev/sdg devid2 size 465.76GiB used 3.03GiB path /dev/sdh devid3 size 232.88GiB used 0.00B path /dev/sdi unplug device sdg: Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8. Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info about processes that Apr 20 07:03:05 rakete umount[16405]: use the device is found by lsof(8) or fuser(1).) Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process exited, code=exited status=32 Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1. Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device number 3 using xhci_hcd Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found, idVendor=152d, idProduct=0567 Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings: Mfr=10, Product=11, SerialNumber=5 Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000 Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device detected Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d pid 0567: 500 Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0 Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3: "/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1" Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an MTP device Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC WD20 02FAEX-007BA00125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC WD50 01AALS-00L3B20125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access SAMSUNG SP2504C 0125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte logical blocks: (500 GB/466 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page found Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive cache: write through Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte logical blocks: (250 GB/233 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page found Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Assuming drive cache: write through Apr 2
Re: Question: raid1 behaviour on failure
On 2016/04/21 15:23, Satoru Takeuchi wrote: On 2016/04/20 14:17, Matthias Bodenbinder wrote: Am 18.04.2016 um 09:22 schrieb Qu Wenruo: BTW, it would be better to post the dmesg for better debug. So here we. I did the same test again. Here is a full log of what i did. It seems to be mean like a bug in btrfs. Sequenz of events: 1. mount the raid1 (2 disc with different size) 2. unplug the biggest drive (hotplug) 3. try to copy something to the degraded raid1 4. plugin the device again (hotplug) This scenario does not work. The disc array is NOT redundant! I can not work with it while a drive is missing and I can not reattach the device so that everything works again. The btrfs module crashes during the test. I am using LMDE2 with backports: btrfs-tools 4.4-1~bpo8+1 linux-image-4.4.0-0.bpo.1-amd64 Matthias rakete - root - /root 1# mount /mnt/raid1/ Journal: Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto defrag Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space caching is enabled Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents rakete - root - /mnt/raid1 3# ll insgesamt 0 drwxrwxr-x 1 root root 36 Nov 14 2014 AfterShot2(64-bit) drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc drwxr-xr-x 1 root root 108 Mär 24 07:31 var 4# btrfs fi show Label: none uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d Total devices 3 FS bytes used 1.60GiB devid1 size 698.64GiB used 3.03GiB path /dev/sdg devid2 size 465.76GiB used 3.03GiB path /dev/sdh devid3 size 232.88GiB used 0.00B path /dev/sdi unplug device sdg: Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8. Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info about processes that Apr 20 07:03:05 rakete umount[16405]: use the device is found by lsof(8) or fuser(1).) Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process exited, code=exited status=32 Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1. Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device number 3 using xhci_hcd Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found, idVendor=152d, idProduct=0567 Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings: Mfr=10, Product=11, SerialNumber=5 Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000 Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device detected Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d pid 0567: 500 Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0 Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3: "/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1" Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an MTP device Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC WD20 02FAEX-007BA00125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC WD50 01AALS-00L3B20125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access SAMSUNG SP2504C 0125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte logical blocks: (500 GB/466 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page found Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive cache: write through Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte logical blocks: (250 GB/233 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page found Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Assuming drive cache: write through Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] Write Protect is
Re: Question: raid1 behaviour on failure
On 2016/04/20 14:17, Matthias Bodenbinder wrote: Am 18.04.2016 um 09:22 schrieb Qu Wenruo: BTW, it would be better to post the dmesg for better debug. So here we. I did the same test again. Here is a full log of what i did. It seems to be mean like a bug in btrfs. Sequenz of events: 1. mount the raid1 (2 disc with different size) 2. unplug the biggest drive (hotplug) 3. try to copy something to the degraded raid1 4. plugin the device again (hotplug) This scenario does not work. The disc array is NOT redundant! I can not work with it while a drive is missing and I can not reattach the device so that everything works again. The btrfs module crashes during the test. I am using LMDE2 with backports: btrfs-tools 4.4-1~bpo8+1 linux-image-4.4.0-0.bpo.1-amd64 Matthias rakete - root - /root 1# mount /mnt/raid1/ Journal: Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto defrag Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space caching is enabled Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents rakete - root - /mnt/raid1 3# ll insgesamt 0 drwxrwxr-x 1 root root 36 Nov 14 2014 AfterShot2(64-bit) drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc drwxr-xr-x 1 root root 108 Mär 24 07:31 var 4# btrfs fi show Label: none uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d Total devices 3 FS bytes used 1.60GiB devid1 size 698.64GiB used 3.03GiB path /dev/sdg devid2 size 465.76GiB used 3.03GiB path /dev/sdh devid3 size 232.88GiB used 0.00B path /dev/sdi unplug device sdg: Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8. Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 243826688, lost sync page write Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal superblock for sdf1-8. Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info about processes that Apr 20 07:03:05 rakete umount[16405]: use the device is found by lsof(8) or fuser(1).) Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process exited, code=exited status=32 Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1. Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device number 3 using xhci_hcd Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found, idVendor=152d, idProduct=0567 Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings: Mfr=10, Product=11, SerialNumber=5 Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000 Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device detected Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d pid 0567: 500 Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0 Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3: "/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1" Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an MTP device Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC WD20 02FAEX-007BA00125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC WD50 01AALS-00L3B20125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access SAMSUNG SP2504C 0125 PQ: 0 ANSI: 6 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8 type 0 Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte logical blocks: (500 GB/466 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page found Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive cache: write through Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08 Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte logical blocks: (250 GB/233 GiB) Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page found Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Assuming drive cache: write through Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] Write Protect is off Apr 20 07:03:25 rakete kernel: sd
[PATCH] btrfs-progs: prop: remove an unnecessary condition on parse_args
>From commit c742debab11f ('btrfs-progs: fix a regression that "property" with -t option doesn't work'), the number of arguments is checked strictly. So the following condition never be satisfied. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-property.c | 5 - 1 file changed, 5 deletions(-) diff --git a/cmds-property.c b/cmds-property.c index eed5f4a..48a8945 100644 --- a/cmds-property.c +++ b/cmds-property.c @@ -352,11 +352,6 @@ static void parse_args(int argc, char **argv, if (value && optind < argc) *value = argv[optind++]; - if (optind != argc) { - error("unexpected agruments found"); - usage(usage_str); - } - if (!*types && object && *object) { ret = autodetect_object_types(*object, types); if (ret < 0) { -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation
On 2016/04/20 7:25, Mark Fasheh wrote: This has been broken since Linux v4.1. We may have worked out a solution on the btrfs list but in the meantime sending a test to expose the issue seems like a good idea. Signed-off-by: Mark Fasheh--- tests/btrfs/122 | 88 +++ tests/btrfs/group | 1 + You forgot to add tests/btrfs/122.out. 2 files changed, 89 insertions(+) create mode 100755 tests/btrfs/122 diff --git a/tests/btrfs/122 b/tests/btrfs/122 new file mode 100755 index 000..b7e9e4b --- /dev/null +++ b/tests/btrfs/122 @@ -0,0 +1,88 @@ +#! /bin/bash +# FS QA Test No. btrfs/122 +# +# Test that qgroup counts are valid after snapshot creation. This has +# been broken in btrfs since Linux v4.1 +# +#--- +# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#--- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch + +rm -f $seqres.full + +# Force a small leaf size to make it easier to blow out our root +# subvolume tree +_scratch_mkfs "--nodesize 16384" nodesize 16384 is the default value. Do you intend other value, for example 4096? +_scratch_mount +_run_btrfs_util_prog quota enable $SCRATCH_MNT + +mkdir "$SCRATCH_MNT/snaps" + +# First make some simple snapshots - the bug was initially reproduced like this +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty1" +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/empty2" + +# This forces the fs tree out past level 0, adding at least one tree +# block which must be properly accounted for when we make our next +# snapshots. +mkdir "$SCRATCH_MNT/data" +for i in `seq 0 640`; do +$XFS_IO_PROG -f -c "pwrite 0 1M" "$SCRATCH_MNT/data/file$i" > /dev/null 2>&1 +done; ";" after "done" is not necessary. Thanks, Satoru + +# Snapshot twice. +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap1" +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap2" + +_scratch_unmount + +# generate a qgroup report and look for inconsistent groups +$BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV 2>&1 | \ + grep -q -E "Counts for qgroup.*are different" +if [ $? -ne 0 ]; then + status=0 +fi + +exit diff --git a/tests/btrfs/group b/tests/btrfs/group index 9403daa..f7e8cff 100644 --- a/tests/btrfs/group +++ b/tests/btrfs/group @@ -122,3 +122,4 @@ 119 auto quick snapshot metadata qgroup 120 auto quick snapshot metadata 121 auto quick snapshot qgroup +122 auto quick snapshot qgroup -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RFE: btrfs subvolume list -t, include marker for snapshots, and whether they are ro
On 2016/04/15 10:55, Chris Murphy wrote: Hi, I'm realizing instead of doing 'btrfs subvolume -t' and then 'btrfs subvolume -tr' and comparing, it would be better if -t just had a column for whether a subvolume is ro. And maybe it's useful to know if a subvolume is a snapshot or not (?). I'm not super picky about the last one. But being able to see if it's ro or not is a nice to have. In particular on openSUSE with snapper where there's a prolific amount of snapshots, and the naming convention doesn't indicate whether something is ro or not. Although I'm not sure whether such changes is acceptable or not, just FYI, you can list subvols with ro flags by the following script. #!/bin/ruby IO.popen("btrfs sub list -t " + ARGV[0]) {|p_list| puts p_list.gets.chomp + "ro" puts p_list.gets.chomp + "--" ro = '' p_list.each { |l| l.chomp! path = l.split[3] IO.popen("btrfs property get #{ARGV[0]+path} | sed -nre 's/^ro=(true|false)$/\\1/p'") { |p_prop| ro = p_prop.gets ro = "false" if ro.nil? } puts l + "\t" + ro } } # NOTE: It's not well tested. sample output: # ./sub-list-with-ro.rb / ID gen top level pathro -- --- - -- 257 48672 5 rootfalse 259 48623 257 var/lib/machinesfalse 452 36540 257 root/snaps/root-2016-03-18 true 457 45676 257 root/snaps/root-2016-04-08 true Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: mkfs: fix an error when using DUP on multidev fs
To accept DUP on multidev fs, in addition to the following commit, we need to mark DUP as an allowed data/metadata profile. commit 42f1279bf8e9 ("btrfs-progs: mkfs: allow DUP on multidev fs, only warn") * actual result = # ./mkfs.btrfs -f -m DUP -d DUP /dev/sdb1 /dev/sdb2 btrfs-progs v4.5-24-ga35b7e6 See http://btrfs.wiki.kernel.org for more information. WARNING: DUP is not recommended on filesystem with multiple devices ERROR: unable to create FS with metadata profile DUP (have 2 devices but 1 devices are required) = * expected result = # ./mkfs.btrfs -f -m dup -d dup /dev/sdb1 /dev/sdb2 WARNING: DUP is not recommended on filesystem with multiple devices btrfs-progs v4.5-25-g1a10a3c See http://btrfs.wiki.kernel.org for more information. Label: (null) UUID: 010d72ff-c87c-4516-8916-5e635719d110 Node size: 16384 Sector size:4096 Filesystem size:28.87GiB Block group profiles: Data: DUP 1.01GiB Metadata: DUP 1.01GiB System: DUP 12.00MiB SSD detected: no Incompat features: extref, skinny-metadata Number of devices: 2 Devices: IDSIZE PATH 1 953.00MiB /dev/sdb1 227.94GiB /dev/sdb2 == Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to devel branch (commit: a35b7e6ee122) --- utils.c | 1 - 1 file changed, 1 deletion(-) diff --git a/utils.c b/utils.c index 75ce6ea..7e45702 100644 --- a/utils.c +++ b/utils.c @@ -2455,7 +2455,6 @@ int test_num_disk_vs_raid(u64 metadata_profile, u64 data_profile, case 2: allowed |= BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1 | BTRFS_BLOCK_GROUP_RAID5; - break; case 1: allowed |= BTRFS_BLOCK_GROUP_DUP; } -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/5] btrfs-progs: "inspect-internal subvolid-resolve" doesn't work
"inspect-internal subvolid-resolve" doesn't work from the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") It's because 1st argument, subvolid, is also used for the pathname of filesystem. 2nd argument should be used for this purpose instead. * actual result == # ./btrfs inspect-internal subvolid-resolve 260 /btrfs ERROR: cannot access '260': No such file or directory == * expected result == # btrfs inspect-internal subvolid-resolve 260 /btrfs snap ====== Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-inspect.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-inspect.c b/cmds-inspect.c index 04e1ae8..6045985 100644 --- a/cmds-inspect.c +++ b/cmds-inspect.c @@ -273,7 +273,7 @@ static int cmd_inspect_subvolid_resolve(int argc, char **argv) if (check_argc_exact(argc - optind, 2)) usage(cmd_inspect_subvolid_resolve_usage); - fd = btrfs_open_dir(argv[optind], , 1); + fd = btrfs_open_dir(argv[optind + 1], , 1); if (fd < 0) { ret = -ENOENT; goto out; -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/5] btrfs-progs: "qgroup assign" can't handle options
"qgroup assign" is considered as working without any options from the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") However, we can pass options to this command. * actual result == # ./btrfs qgroup assign --rescan 0/260 1/261 /btrfs btrfs qgroup assign: unrecognized option '--rescan' usage: btrfs qgroup assign [options] Assign SRC as the child qgroup of DST --rescan schedule qutoa rescan if needed --no-rescan == * expected result == # ./btrfs qgroup assign --rescan 0/260 1/261 /btrfs # ====== Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-qgroup.c | 62 --- 1 file changed, 25 insertions(+), 37 deletions(-) diff --git a/cmds-qgroup.c b/cmds-qgroup.c index 7ae2253..ebd66ef 100644 --- a/cmds-qgroup.c +++ b/cmds-qgroup.c @@ -32,7 +32,8 @@ static const char * const qgroup_cmd_group_usage[] = { NULL }; -static int _cmd_qgroup_assign(int assign, int argc, char **argv) +static int _cmd_qgroup_assign(int assign, int argc, char **argv, + const char * const *usage_str) { int ret = 0; int fd; @@ -41,28 +42,31 @@ static int _cmd_qgroup_assign(int assign, int argc, char **argv) struct btrfs_ioctl_qgroup_assign_args args; DIR *dirstream = NULL; - while (1) { - enum { GETOPT_VAL_RESCAN = 256 }; - static const struct option long_options[] = { - { "rescan", no_argument, NULL, GETOPT_VAL_RESCAN }, - { NULL, 0, NULL, 0 } - }; - int c = getopt_long(argc, argv, "", long_options, NULL); - - if (c < 0) - break; - switch (c) { - case GETOPT_VAL_RESCAN: - rescan = 1; - break; - default: - /* Usage printed by the caller */ - return -1; + if (assign) { + while (1) { + enum { GETOPT_VAL_RESCAN = 256 }; + static const struct option long_options[] = { + { "rescan", no_argument, NULL, GETOPT_VAL_RESCAN }, + { NULL, 0, NULL, 0 } + }; + int c = getopt_long(argc, argv, "", long_options, NULL); + + if (c < 0) + break; + switch (c) { + case GETOPT_VAL_RESCAN: + rescan = 1; + break; + default: + usage(usage_str); + } } + } else { + clean_args_no_options(argc, argv, usage_str); } if (check_argc_exact(argc - optind, 3)) - return -1; + usage(usage_str); memset(, 0, sizeof(args)); args.assign = assign; @@ -208,15 +212,7 @@ static const char * const cmd_qgroup_assign_usage[] = { static int cmd_qgroup_assign(int argc, char **argv) { - int ret; - - clean_args_no_options(argc, argv, cmd_qgroup_assign_usage); - - ret = _cmd_qgroup_assign(1, argc, argv); - - if (ret < 0) - usage(cmd_qgroup_assign_usage); - return ret; + return _cmd_qgroup_assign(1, argc, argv, cmd_qgroup_assign_usage); } static const char * const cmd_qgroup_remove_usage[] = { @@ -227,15 +223,7 @@ static const char * const cmd_qgroup_remove_usage[] = { static int cmd_qgroup_remove(int argc, char **argv) { - int ret; - - clean_args_no_options(argc, argv, cmd_qgroup_remove_usage); - - ret = _cmd_qgroup_assign(0, argc, argv); - - if (ret < 0) - usage(cmd_qgroup_remove_usage); - return ret; + return _cmd_qgroup_assign(0, argc, argv, cmd_qgroup_remove_usage); } static const char * const cmd_qgroup_create_usage[] = { -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/5] btrfs-progs: "sub get-default" doesn't work
On 2016/03/17 3:29, David Sterba wrote: Hi, btrfs-progs 4.5-rc1 have been released. The ETA for final release is this Friday, so please test and report if you find problems. Small fixes or documentation updates are welcome. Please apply this patchset. Especially [1/5]~[4/5] fix the regressions caused by the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") I read whole this commit carefully and probably fixed all problems in this commit by 1a521af, c742deb, and this patchset. Satoru --- "sub get-default" does't work from the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") * actual result == # ./btrfs sub get-default /btrfs btrfs subvolume get-default: too few arguments usage: btrfs subvolume get-default Get the default subvolume of a filesystem == * expected result == # btrfs sub get-default /btrfs ID 5 (FS_TREE) ====== Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to devel branch (commit: 40dc7c504cf0) --- cmds-subvolume.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-subvolume.c b/cmds-subvolume.c index 32caaa5..3953d7c 100644 --- a/cmds-subvolume.c +++ b/cmds-subvolume.c @@ -790,7 +790,7 @@ static int cmd_subvol_get_default(int argc, char **argv) clean_args_no_options(argc, argv, cmd_subvol_get_default_usage); - if (check_argc_exact(argc - optind, 2)) + if (check_argc_exact(argc - optind, 1)) usage(cmd_subvol_get_default_usage); subvol = argv[1]; -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/5] btrfs-progs: qgroup assign can't handle --no-rescan option
* actual result == # btrfs qgroup assign --no-rescan 0/260 1/261 /btrfs btrfs qgroup assign: unrecognized option '--no-rescan' usage: btrfs qgroup assign [options] Assign SRC as the child qgroup of DST --rescan schedule qutoa rescan if needed --no-rescan == * expected result == # ./btrfs qgroup assign --no-rescan 0/260 1/261 /btrfs # == Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-qgroup.c | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/cmds-qgroup.c b/cmds-qgroup.c index ebd66ef..4b4149d 100644 --- a/cmds-qgroup.c +++ b/cmds-qgroup.c @@ -44,9 +44,13 @@ static int _cmd_qgroup_assign(int assign, int argc, char **argv, if (assign) { while (1) { - enum { GETOPT_VAL_RESCAN = 256 }; + enum { + GETOPT_VAL_RESCAN = 256, + GETOPT_VAL_NO_RESCAN = 257, + }; static const struct option long_options[] = { { "rescan", no_argument, NULL, GETOPT_VAL_RESCAN }, + { "no-rescan", no_argument, NULL, GETOPT_VAL_NO_RESCAN }, { NULL, 0, NULL, 0 } }; int c = getopt_long(argc, argv, "", long_options, NULL); @@ -57,6 +61,9 @@ static int _cmd_qgroup_assign(int assign, int argc, char **argv, case GETOPT_VAL_RESCAN: rescan = 1; break; + case GETOPT_VAL_NO_RESCAN: + rescan = 0; + break; default: usage(usage_str); } @@ -206,7 +213,7 @@ static const char * const cmd_qgroup_assign_usage[] = { "Assign SRC as the child qgroup of DST", "", "--rescan schedule qutoa rescan if needed", - "--no-rescan", + "--no-rescandon't schedule quota rescan", NULL }; -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/5] btrfs-progs: "qgroup create/destroy" don't work
"qgroup create/destroy" don't work from the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") * actual result == # ./btrfs qgroup create 1 /btrfs/sub btrfs qgroup create: too few arguments usage: btrfs qgroup create Create a subvolume quota group. == # btrfs qgroup create 1 /btrfs/sub # ./btrfs qgroup destroy 1 /btrfs/sub btrfs qgroup destroy: too few arguments usage: btrfs qgroup destroy Destroy a quota group. == * expected result == # btrfs qgroup create 1 /btrfs/sub # btrfs qgroup destroy 1 /btrfs/sub/ ====== Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-qgroup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-qgroup.c b/cmds-qgroup.c index c4a0c6b..7ae2253 100644 --- a/cmds-qgroup.c +++ b/cmds-qgroup.c @@ -124,7 +124,7 @@ static int _cmd_qgroup_create(int create, int argc, char **argv) struct btrfs_ioctl_qgroup_create_args args; DIR *dirstream = NULL; - if (check_argc_exact(argc - optind, 3)) + if (check_argc_exact(argc - optind, 2)) return -1; memset(, 0, sizeof(args)); -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2 v2] btrfs-progs: Fix a regression that "property" with -t option doesn't work
Sorry, I forgot to add "btrfs-progs: " in the subject line. --- "property" is considered as working without any options from the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") However, we can pass -t option to this command. * actual result == $ ./btrfs prop list -t f /btrfs btrfs property list: invalid option -- 't' usage: btrfs property list [-t ] Lists available properties with their descriptions for the given object. Please see the help of 'btrfs property get' for a description of objects and object types. == * expected result == $ ./btrfs prop list -t f /btrfs label Set/get label of device. ====== Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com --- v2: Reflect David's comment. - Check the number of non-option arguments --- cmds-property.c | 35 --- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/cmds-property.c b/cmds-property.c index 5b4da26..eed5f4a 100644 --- a/cmds-property.c +++ b/cmds-property.c @@ -294,10 +294,11 @@ out: static void parse_args(int argc, char **argv, const char * const *usage_str, int *types, char **object, - char **name, char **value) + char **name, char **value, int min_nonopt_args) { int ret; char *type_str = NULL; + int max_nonopt_args = 0; optind = 1; while (1) { @@ -314,6 +315,17 @@ static void parse_args(int argc, char **argv, } } + if (object) + max_nonopt_args++; + if (name) + max_nonopt_args++; + if (value) + max_nonopt_args++; + + if (check_argc_min(argc - optind, min_nonopt_args) || + check_argc_max(argc - optind, max_nonopt_args)) + usage(usage_str); + *types = 0; if (type_str) { if (!strcmp(type_str, "s") || !strcmp(type_str, "subvol")) { @@ -379,13 +391,8 @@ static int cmd_property_get(int argc, char **argv) char *name = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_get_usage); - - if (check_argc_min(argc, 2) || check_argc_max(argc, 5)) - usage(cmd_property_get_usage); - parse_args(argc, argv, cmd_property_get_usage, , , , - NULL); + NULL, 1); if (!object) { error("invalid arguments"); usage(cmd_property_get_usage); @@ -415,13 +422,8 @@ static int cmd_property_set(int argc, char **argv) char *value = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_set_usage); - - if (check_argc_min(argc, 4) || check_argc_max(argc, 6)) - usage(cmd_property_set_usage); - parse_args(argc, argv, cmd_property_set_usage, , - , , ); + , , , 3); if (!object || !name || !value) { error("invalid arguments"); usage(cmd_property_set_usage); @@ -446,13 +448,8 @@ static int cmd_property_list(int argc, char **argv) char *object = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_list_usage); - - if (check_argc_min(argc, 2) || check_argc_max(argc, 4)) - usage(cmd_property_list_usage); - parse_args(argc, argv, cmd_property_list_usage, - , , NULL, NULL); + , , NULL, NULL, 1); if (!object) { error("invalid arguments"); usage(cmd_property_list_usage); -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2 v2] fix a regression that "property" with -t option doesn't work
"property" is considered as working without any options from the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") However, we can pass -t option to this command. * actual result == $ ./btrfs prop list -t f /btrfs btrfs property list: invalid option -- 't' usage: btrfs property list [-t ] Lists available properties with their descriptions for the given object. Please see the help of 'btrfs property get' for a description of objects and object types. == * expected result == $ ./btrfs prop list -t f /btrfs label Set/get label of device. ====== Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com --- v2: Reflect David's comment. - Check the number of non-option arguments --- cmds-property.c | 35 --- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/cmds-property.c b/cmds-property.c index 5b4da26..eed5f4a 100644 --- a/cmds-property.c +++ b/cmds-property.c @@ -294,10 +294,11 @@ out: static void parse_args(int argc, char **argv, const char * const *usage_str, int *types, char **object, - char **name, char **value) + char **name, char **value, int min_nonopt_args) { int ret; char *type_str = NULL; + int max_nonopt_args = 0; optind = 1; while (1) { @@ -314,6 +315,17 @@ static void parse_args(int argc, char **argv, } } + if (object) + max_nonopt_args++; + if (name) + max_nonopt_args++; + if (value) + max_nonopt_args++; + + if (check_argc_min(argc - optind, min_nonopt_args) || + check_argc_max(argc - optind, max_nonopt_args)) + usage(usage_str); + *types = 0; if (type_str) { if (!strcmp(type_str, "s") || !strcmp(type_str, "subvol")) { @@ -379,13 +391,8 @@ static int cmd_property_get(int argc, char **argv) char *name = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_get_usage); - - if (check_argc_min(argc, 2) || check_argc_max(argc, 5)) - usage(cmd_property_get_usage); - parse_args(argc, argv, cmd_property_get_usage, , , , - NULL); + NULL, 1); if (!object) { error("invalid arguments"); usage(cmd_property_get_usage); @@ -415,13 +422,8 @@ static int cmd_property_set(int argc, char **argv) char *value = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_set_usage); - - if (check_argc_min(argc, 4) || check_argc_max(argc, 6)) - usage(cmd_property_set_usage); - parse_args(argc, argv, cmd_property_set_usage, , - , , ); + , , , 3); if (!object || !name || !value) { error("invalid arguments"); usage(cmd_property_set_usage); @@ -446,13 +448,8 @@ static int cmd_property_list(int argc, char **argv) char *object = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_list_usage); - - if (check_argc_min(argc, 2) || check_argc_max(argc, 4)) - usage(cmd_property_list_usage); - parse_args(argc, argv, cmd_property_list_usage, - , , NULL, NULL); + , , NULL, NULL, 1); if (!object) { error("invalid arguments"); usage(cmd_property_list_usage); -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] btrfs-progs: Describe optarg of -m option in the manpage of receive
Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to devel branch (commit: 4685a560811a) --- Documentation/btrfs-receive.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/btrfs-receive.asciidoc b/Documentation/btrfs-receive.asciidoc index 84b85c1..758eebe 100644 --- a/Documentation/btrfs-receive.asciidoc +++ b/Documentation/btrfs-receive.asciidoc @@ -43,7 +43,7 @@ or on EOF. --max-errors :: Terminate as soon as N errors happened while processing commands from the send stream. Default value is 1. A value of 0 means no limit. --m:: +-m :: The root mount point of the destination fs. + By default the mountpoint is searched in /proc/self/mounts. -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/4] btrfs-progs: fix a reression that "property" with -t option doesn't work
On 2016/03/14 21:23, David Sterba wrote: On Mon, Mar 14, 2016 at 09:12:36AM +0900, Satoru Takeuchi wrote: --- a/cmds-property.c +++ b/cmds-property.c @@ -379,9 +379,7 @@ static int cmd_property_get(int argc, char **argv) char *name = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_get_usage); - - if (check_argc_min(argc, 2) || check_argc_max(argc, 5)) + if (check_argc_min(argc, 2)) usage(cmd_property_get_usage); parse_args(argc, argv, cmd_property_get_usage, , , , We still need to check the number of non-option arguments here, when the optind is set from parse_args. OK, I'll send a patch which checks the number after getopt. Thanks, Satoru @@ -415,9 +413,7 @@ static int cmd_property_set(int argc, char **argv) - if (check_argc_min(argc, 4) || check_argc_max(argc, 6)) + if (check_argc_min(argc, 4)) usage(cmd_property_set_usage); ... parse_args(argc, argv, cmd_property_set_usage, , @@ -446,9 +442,7 @@ static int cmd_property_list(int argc, char **argv) - if (check_argc_min(argc, 2) || check_argc_max(argc, 4)) + if (check_argc_min(argc, 2)) ... -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs: Simplify conditions about compress while mapping btrfs flags to inode flags
Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to 4.5 --- fs/btrfs/ioctl.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 48aee98..b474e32 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -123,10 +123,10 @@ static unsigned int btrfs_flags_to_ioctl(unsigned int flags) if (flags & BTRFS_INODE_NODATACOW) iflags |= FS_NOCOW_FL; - if ((flags & BTRFS_INODE_COMPRESS) && !(flags & BTRFS_INODE_NOCOMPRESS)) - iflags |= FS_COMPR_FL; - else if (flags & BTRFS_INODE_NOCOMPRESS) + if (flags & BTRFS_INODE_NOCOMPRESS) iflags |= FS_NOCOMP_FL; + else if (flags & BTRFS_INODE_COMPRESS) + iflags |= FS_COMPR_FL; return iflags; } -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4] btrfs-progs: "device ready" accepts just one device
* actual result === # ./btrfs device ready /dev/sdb foo # === * expecting result === # ./btrfs device ready /dev/sdb foo btrfs device ready: too many arguments usage: btrfs device ready Check device to see if it has all of its devices in cache for mounting # === Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-device.c b/cmds-device.c index 33da2ce..23656c3 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -326,7 +326,7 @@ static int cmd_device_ready(int argc, char **argv) clean_args_no_options(argc, argv, cmd_device_ready_usage); - if (check_argc_min(argc - optind, 1)) + if (check_argc_exact(argc - optind, 1)) usage(cmd_device_ready_usage); fd = open("/dev/btrfs-control", O_RDWR); -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4] btrfs-progs: fix a reression that "property" with -t option doesn't work
"property" is considered as working without any options from the following commit. commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") However, we can pass -t option to this command. * actual result == # ./btrfs prop list -t f /btrfs btrfs property list: invalid option -- 't' usage: btrfs property list [-t ] Lists available properties with their descriptions for the given object. Please see the help of 'btrfs property get' for a description of objects and object types. == * expected result == # btrfs prop list -t f /btrfs label Set/get label of device ====== Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- cmds-property.c | 12 +++- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/cmds-property.c b/cmds-property.c index 5b4da26..4c2beb7 100644 --- a/cmds-property.c +++ b/cmds-property.c @@ -379,9 +379,7 @@ static int cmd_property_get(int argc, char **argv) char *name = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_get_usage); - - if (check_argc_min(argc, 2) || check_argc_max(argc, 5)) + if (check_argc_min(argc, 2)) usage(cmd_property_get_usage); parse_args(argc, argv, cmd_property_get_usage, , , , @@ -415,9 +413,7 @@ static int cmd_property_set(int argc, char **argv) char *value = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_set_usage); - - if (check_argc_min(argc, 4) || check_argc_max(argc, 6)) + if (check_argc_min(argc, 4)) usage(cmd_property_set_usage); parse_args(argc, argv, cmd_property_set_usage, , @@ -446,9 +442,7 @@ static int cmd_property_list(int argc, char **argv) char *object = NULL; int types = 0; - clean_args_no_options(argc, argv, cmd_property_list_usage); - - if (check_argc_min(argc, 2) || check_argc_max(argc, 4)) + if (check_argc_min(argc, 2)) usage(cmd_property_list_usage); parse_args(argc, argv, cmd_property_list_usage, -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4] btrfs-progs: Avoid interpreting options after "--" when getting unit mode
* actual result == # ./btrfs device usage -- -m /btrfs /dev/sdf1, ID: 1 Device size: 95367.41MiB Data,single: 2056.00MiB Metadata,DUP: 2048.00MiB System,DUP: 16.00MiB Unallocated: 91247.41MiB == * expected result == # ./btrfs device usage -- -m /btrfs ERROR: can't access '-m': No such file or directory == Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to devel branch (commit: 31266d0ef811). --- utils.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/utils.c b/utils.c index d7ceaa8..77f0f68 100644 --- a/utils.c +++ b/utils.c @@ -3024,6 +3024,9 @@ unsigned int get_unit_mode_from_arg(int *argc, char *argv[], int df_mode) int arg_end; for (arg_i = 0; arg_i < *argc; arg_i++) { + if (!strcmp(argv[arg_i], "--")) + break; + if (!strcmp(argv[arg_i], "--raw")) { unit_mode = UNITS_RAW; argv[arg_i] = NULL; -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to move a btrfs volume to a smaller disk
On 2016/03/10 6:46, Hendrik Friedel wrote: Hello, I intend to move this subvolume to a new device. btrfs fi show /mnt2/Data_Store/ Label: 'Data_Store' uuid: 0ccc1e24-090d-42e2-9e61-d0a1b3101f93 Total devices 1 FS bytes used 47.93GiB devid1 size 102.94GiB used 76.03GiB path /dev/sdb4 (fi usage at the bottom of this message) The new device (sda4) is 8G smaller unfortunately. sda 8:00 111.8G 0 disk └─sda48:40 103.5G 0 part sdb 8:16 0 119.2G 0 disk └─sdb48:20 0 111G 0 part /mnt2/Data_Store Thus, btrfs replace does not work What would you suggest now to move the FS (it does contain many subvolumes)? I tried btrfs send /mnt2/Data_Store/read_only_snapshot/ | btrfs receive /mnt/sda4/ but this only created an empty subvolume /mnt/sda4/read_only_snapshot/ I suspect that you captured snapshot as == btrfs sub snap -r /mnt2/Data_Store /mnt2/Data_Store/read_only_snapshot == and /mnt2/Data_Store only contains subvolumes instead of having files/directries directly. Then send/receive work as you said. Probably if you run "ls -li /mnt2/Data_Store", all files have inode 256 (it means subvolume). In addition, all the directories under /mnt2/Data_Store/read_only_snapshot are empty. It's because "sub snap" only handles one subvolume and doesn't handle subvolumes inside it. So, if you'd like to copy all the data by send/receive, you should capture and send/receive snapshot for each subvolume. So, then btrfs device add /dev/sda4 /mnt/Data_Store btrfs balance start /mnt/Data_Store btrfs device remove /dev/sdb4 /mnt/Data_Store ? balance is not necessary. When you run device remove /dev/sdb4, then all the contents of /dev/sdb4 migrate to /dev/sda4 properly. Thanks, Satoru Or is there a better option? Regards, Hendrik btrfs fi usage /mnt2/Data_Store/ Overall: Device size: 102.94GiB Device allocated: 74.03GiB Device unallocated: 28.91GiB Device missing: 0.00B Used: 47.96GiB Free (estimated): 53.24GiB (min: 53.24GiB) Data ratio: 1.00 Metadata ratio: 1.00 Global reserve: 512.00MiB (used: 0.00B) Data,single: Size:69.00GiB, Used:44.67GiB /dev/sdb4 69.00GiB Metadata,single: Size:5.00GiB, Used:3.29GiB /dev/sdb4 5.00GiB System,single: Size:32.00MiB, Used:16.00KiB /dev/sdb4 32.00MiB Unallocated: /dev/sdb4 28.91GiB --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. https://www.avast.com/antivirus -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: Fix device scan to interpret its argument properly
Fix the following bug. # btrfs device scan -- /dev/sdb ERROR: not a block device: -- It should work as follow. # ./btrfs device scan -- /dev/sdb Scanning for Btrfs filesystems in '/dev/sdb' Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to devel (commit: 5d038a7ed212) --- cmds-device.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cmds-device.c b/cmds-device.c index 94ffdc5..e500638 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -246,7 +246,7 @@ static const char * const cmd_device_scan_usage[] = { static int cmd_device_scan(int argc, char **argv) { int i; - int devstart = 1; + int devstart; int all = 0; int ret = 0; @@ -269,6 +269,7 @@ static int cmd_device_scan(int argc, char **argv) usage(cmd_device_scan_usage); } } + devstart = optind; if (all && check_argc_max(argc - optind, 1)) usage(cmd_device_scan_usage); -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] btrfs-progs: Remove unnecessary variable devstart from cmd_device_scan()
On 2016/03/10 22:27, David Sterba wrote: On Thu, Mar 10, 2016 at 08:04:59AM +0900, Satoru Takeuchi wrote: It's unnecessary since it's always 1. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to devel (commit b2bdd0da8969). --- cmds-device.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/cmds-device.c b/cmds-device.c index cb470af..78d6535 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -246,7 +246,6 @@ static const char * const cmd_device_scan_usage[] = { static int cmd_device_scan(int argc, char **argv) { int i; - int devstart = 1; int all = 0; int ret = 0; @@ -282,7 +281,7 @@ static int cmd_device_scan(int argc, char **argv) goto out; } - for( i = devstart ; i < argc ; i++ ){ + for (i = 1; i < argc; i++) { Actually it needs to be set to optind before the for cycle, otherwise: # btrfs device scan -- /dev/sda ERROR: not a block device: -- Thank you for your comment. I'll fix it. Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] btrfs-progs: Describe device scan -d is a deprecated option in manpage
It's already marked as deprecated in cmd_device_scan_usage(). commit 5444864e5605 ("btrfs-progs: remove BTRFS_SCAN_PROC scan method") Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- Documentation/btrfs-device.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/btrfs-device.asciidoc b/Documentation/btrfs-device.asciidoc index bd878f4..bf16a94 100644 --- a/Documentation/btrfs-device.asciidoc +++ b/Documentation/btrfs-device.asciidoc @@ -89,8 +89,8 @@ Scan devices for a btrfs filesystem. If one or more devices are passed, these are scanned for a btrfs filesystem. If no devices are passed, btrfs uses block devices containing btrfs filesystem as listed by blkid. -Finally, if '--all-devices' or '-d' is passed, all the devices under /dev are -scanned. +Finally, '--all-devices' or '-d' is the deprecated option. If it is passed, +its behavior is the same as if no devices are passed. *stats* [-z] |:: Read and print the device IO stats for all mounted devices of the filesystem -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] btrfs-progs: Remove unnecessary variable devstart from cmd_device_scan()
It's unnecessary since it's always 1. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to devel (commit b2bdd0da8969). --- cmds-device.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/cmds-device.c b/cmds-device.c index cb470af..78d6535 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -246,7 +246,6 @@ static const char * const cmd_device_scan_usage[] = { static int cmd_device_scan(int argc, char **argv) { int i; - int devstart = 1; int all = 0; int ret = 0; @@ -282,7 +281,7 @@ static int cmd_device_scan(int argc, char **argv) goto out; } - for( i = devstart ; i < argc ; i++ ){ + for (i = 1; i < argc; i++) { char *path; if (is_block_device(argv[i]) != 1) { -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Fix broken 'device scan' arguments parsing
On 2016/03/09 10:19, Yauhen Kharuzhy wrote: > commit 52179e4fea41e55f31c92cd033a0b53a5107b4f4 'btrfs-progs: unify argc > min/max checking' brokes 'btrfs device scan' command when no argument > was given. Fix this. > > Signed-off-by: Yauhen Kharuzhy> --- > cmds-device.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/cmds-device.c b/cmds-device.c > index cb470af..94ffdc5 100644 > --- a/cmds-device.c > +++ b/cmds-device.c > @@ -273,7 +273,7 @@ static int cmd_device_scan(int argc, char **argv) > if (all && check_argc_max(argc - optind, 1)) It's better to the above line as follows. if (all && check_argc_exact(argc - optind, 0)) > usage(cmd_device_scan_usage); > > - if (all || argc - optind == 1) { > + if (all || argc - optind == 0) { > printf("Scanning for Btrfs filesystems\n"); > ret = btrfs_scan_lblkid(); > error_on(ret, "error %d while scanning", ret); > Here is the test result. * v4.4.1 == # ./btrfs device scan Scanning for Btrfs filesystems # ./btrfs device scan -d Scanning for Btrfs filesystems # ./btrfs device scan -d foo btrfs device scan: too many arguments usage: btrfs device scan [(-d|--all-devices)| [...]] Scan devices for a btrfs filesystem -d|--all-devices (deprecated) == * devel branch with this patch. == # ./btrfs device scan Scanning for Btrfs filesystems # ./btrfs device scan -d Scanning for Btrfs filesystems # ./btrfs device scan -d foo Scanning for Btrfs filesystems === * devel branch without this patch. == # ./btrfs device scan # ./btrfs device scan -d Scanning for Btrfs filesystems # ./btrfs device scan -d foo Scanning for Btrfs filesystems === * devel branch with this patch and my patch. = # ./btrfs device scan Scanning for Btrfs filesystems # ./btrfs device scan -d Scanning for Btrfs filesystems # ./btrfs device scan -d foo btrfs device scan: too many arguments usage: btrfs device scan [(-d|--all-devices)| [...]] Scan devices for a btrfs filesystem -d|--all-devices (deprecated) === Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3] Btrfs: Show a warning message if one of objectid reaches its highest value
It's better to show a warning message for the exceptional case that one of objectid (in most case, inode number) reaches its highest value. For example, if inode cache is off and this event happens, we can't create any file even if there are not so many files. This message ease detecting such problem. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to 4.5-rc7 V3: - Show this message every time when hitting this problem for simplicity (thanks to the comment from Goffredo). - Reflect Naota's comment - Keep the error code as is. V2: - Reflect Filipe's comment - Use btrfs_warn() instead of WARN_ONCE() - Print the id of the tree --- fs/btrfs/inode-map.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c index e50316c..5d97ee7 100644 --- a/fs/btrfs/inode-map.c +++ b/fs/btrfs/inode-map.c @@ -556,6 +556,9 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 *objectid) mutex_lock(>objectid_mutex); if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) { + btrfs_warn(root->fs_info, + "The objectid of root %llu reaches its highest value.\n", + root->root_key.objectid); ret = -ENOSPC; goto out; } -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value
On 2016/03/09 11:32, Naohiro Aota wrote: 2016-03-07 12:05 GMT+09:00 Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>: - It's better to show a warning message for the exceptional case that one of objectid (in most case, inode number) reaches its highest value. Show this message only once to avoid filling dmesg with it. - EOVERFLOW is more proper return value for this case. ENOSPC is for "No space left on device" case and objectid isn't related to any device. I have concern about EOVERFLOW. The value returned here will go through to the user space via btrfs_find_free_ino() and btrfs_create(). It means that "creat" and "mkdir" can now return EOVERFLOW when it failed to assign new inode number. Such behavior would disagree with other file systems, which result in user space programs to be confused. Also, I don't think EOVERFLOW described in "creat(2)" (or open(2)) suits for this case. As far as I read the following man page from creat(2), giving ENOSPC is better option here. I consider, as I read man, ENOSPC is also doesn't explain this case. It's not related to pathname. man 2 creat: ENOSPC pathname was to be created but the device containing pathname has no room for the new file. However, I agree with the ENOSPC is better than EOVERFLOW because existing code has worked with the former value in such case. Next patch will keep the error code as is. Thank you for your comment, Naota. Satoru ENOSPC: pathname was to be created but the device containing pathname has no room for the new file. EOVERFLOW: pathname refers to a regular file that is too large to be opened. (snip) Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to 4.5-rc7 --- fs/btrfs/inode-map.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c index e50316c..f5e3228 100644 --- a/fs/btrfs/inode-map.c +++ b/fs/btrfs/inode-map.c @@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 *objectid) mutex_lock(>objectid_mutex); if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) { - ret = -ENOSPC; + static bool __warned = false; + + if (unlikely(!__warned)) { + btrfs_warn(root->fs_info, + "The objectid of root %llu reaches its highest value.\n", + root->root_key.objectid); + __warned = true; + } + ret = -EOVERFLOW; goto out; } -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value
On 2016/03/09 4:24, Goffredo Baroncelli wrote: > On 2016-03-07 04:05, Satoru Takeuchi wrote: >> - It's better to show a warning message for the exceptional case >> that one of objectid (in most case, inode number) reaches its >> highest value. Show this message only once to avoid filling >> dmesg with it. >> - EOVERFLOW is more proper return value for this case. >> ENOSPC is for "No space left on device" case and objectid isn't >> related to any device. >> >> Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> >> --- >> This patch can be applied to 4.5-rc7 >> --- >>fs/btrfs/inode-map.c | 10 +- >>1 file changed, 9 insertions(+), 1 deletion(-) >> >> diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c >> index e50316c..f5e3228 100644 >> --- a/fs/btrfs/inode-map.c >> +++ b/fs/btrfs/inode-map.c >> @@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, >> u64 *objectid) >> mutex_lock(>objectid_mutex); >> >> if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) { >> -ret = -ENOSPC; >> +static bool __warned = false; > > > Please, don't use a static GLOBAL variable. I suggest to move to a "per > filesystem" variables for two main reasons: > > 1) if in the (very unlikely) case where two different filesystems reach > BTRFS_LAST_FREE_OBJECTID, the first error will hide the second one. > > 2) if you umount and remount the filesystem the error is not shown anymore. A > module unload/load or a reboot is required. > If something strange happens, one of the first thing that the user does, is > to umount/remount the filesystem. But the error is not show anymore. This > could complicate the diagnosis of the problem. OK, I see. I rethink what should this patch be since it's overkill to prepare per-filesystem variable just for this warning message. I'll resend patch which shows this message every time when hitting this condition instead of does it just once. Thanks, Satoru > > > > >> + >> +if (unlikely(!__warned)) { >> +btrfs_warn(root->fs_info, >> + "The objectid of root %llu reaches its >> highest value.\n", >> + root->root_key.objectid); >> +__warned = true; >> +} >> +ret = -EOVERFLOW; >> goto out; >> } >> > > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Documentation: btrfs: remove usage specific information
On 2016/03/09 2:04, David Sterba wrote: > The document in the kernel sources is yet another palce where the > documentation would need to be updated, while it is not the primary > source. We actively maintain the wiki pages. > > Signed-off-by: David Sterba> --- > Documentation/filesystems/btrfs.txt | 261 > ++-- > 1 file changed, 11 insertions(+), 250 deletions(-) > > diff --git a/Documentation/filesystems/btrfs.txt > b/Documentation/filesystems/btrfs.txt > index c772b47e7ef0..f9dad22d95ce 100644 > --- a/Documentation/filesystems/btrfs.txt > +++ b/Documentation/filesystems/btrfs.txt > @@ -1,20 +1,10 @@ > - > BTRFS > = > ... > -* btrfs-convert: in-place conversion from ext2/3/4 filesystems > + https://btrfs.wiki.kernel.org > > -* btrfs-image: dump filesystem metadata for debugging > +that maintains information about administration tasks, frequently asked > +questions, use cases, mount options, comprehensible changelogs, features, > +manual pages, source code repositories, contacts etc. About mount options, we also have "man 5 btrfs" and it's newer than wiki page's one. https://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg34545.html commit 26341f734d6d ("btrfs-progs: add mount options to btrfs-mount.5") Last update: - Btrfs wiki -> Mount options: Sep 19 2015 - btrfs-man5.asciidoc: Jan 11 2016 So, how about refer to "man 5 btrfs" from "Btrfs wiki -> Mount options -> List of options" and/or this document? Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfsck memory usage reduce idea
On 2016/03/08 17:46, Qu Wenruo wrote: Satoru Takeuchi wrote on 2016/03/08 17:28 +0900: Hi Qu, On 2016/03/07 14:42, Qu Wenruo wrote: Hi, As many have already known, "btrfs check" is a memory eater. The problem is, btrfsck checks extent tree in a very comprehensive method. 1) Create extent_ref for each extent item with backref 2) Iterate all other trees to add extent ref 3) If one extent_ref with all ref/backref matches, it's deleted. The method is good, can found any extent mismatch problem when checking extent tree. (Although it has already iterated the whole fs) For a large enough filesystem, it may have tegas of extents, and memory is easy eaten up. We hope to fix it in the following method: 1) Only check extent backref when iterating extent tree Unlike current implement, we check one extent item and its backref only. If one backref can't be reached, then it's an error and output (or try to fix). After iterating all backref of an extent item, all related memory is freed and we won't bother recording anything for later use. That's to say, we only care backref mismatch case when checking extent tree. Case like missing EXTENT_ITEM for some extent is not checked here. 2) Check extent ref while iterating other trees We only check forward-ref while iterating one tree. In this step, we only check forward-ref, so we can find the remaining problem like missing EXTENT_ITEM for given extent. Any further advice/suggestion? Or is there anyone already doing such work? Thank you for your effort. I have basic questions. 1. Could you tell me what you'd like to do? a) Provide completely the same function with current implementation by other, more efficient way. Same function, but less efficient. It may takes longer time, more IO, but less memory. I see. And some error message will be output at different time. E.g, error message for missing backref may be output at fs tree checking time, instead of extent tree checking time. It's OK if, finally, all error messages the same as the current implementation are shown. b) Replace the current implementation with the quicker one that provides the limited function. c) Any other 2. Do you have the estimation that how long does the new algorithm take compare with the current one? Depends on the fs hierarchy. But in all case, IO will be more than original implement. The most efficient case would be, one subvolume and no dedup file. (which means one file extent refer to one extent on data, no in-band or out-band dedup). In that case, old implement will iterate the whole metadata twice, and new implement will iterate the whole metadata twice + extra. For worst case, like inband dedup with multiple almost identical snapshot, > things will be much much slower, more IO, more tree search, maybe O(n^2) > or more. But memory usage should not be much different though. In short, use more IO to trade for memory. Anyway, for a large fs, it won't be possible to take a short time for a comprehensive fsck. Got it. Thanks, Satoru Thanks, Qu # Of course, "currently not sure" is OK at this stage. I'm interested in it because there is the trade-off between speed and memory consumption in many case, and btrfsck takes very long time with a large filesystem. Thanks, Satoru Thanks, Qu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfsck memory usage reduce idea
Hi Qu, On 2016/03/07 14:42, Qu Wenruo wrote: Hi, As many have already known, "btrfs check" is a memory eater. The problem is, btrfsck checks extent tree in a very comprehensive method. 1) Create extent_ref for each extent item with backref 2) Iterate all other trees to add extent ref 3) If one extent_ref with all ref/backref matches, it's deleted. The method is good, can found any extent mismatch problem when checking extent tree. (Although it has already iterated the whole fs) For a large enough filesystem, it may have tegas of extents, and memory is easy eaten up. We hope to fix it in the following method: 1) Only check extent backref when iterating extent tree Unlike current implement, we check one extent item and its backref only. If one backref can't be reached, then it's an error and output (or try to fix). After iterating all backref of an extent item, all related memory is freed and we won't bother recording anything for later use. That's to say, we only care backref mismatch case when checking extent tree. Case like missing EXTENT_ITEM for some extent is not checked here. 2) Check extent ref while iterating other trees We only check forward-ref while iterating one tree. In this step, we only check forward-ref, so we can find the remaining problem like missing EXTENT_ITEM for given extent. Any further advice/suggestion? Or is there anyone already doing such work? Thank you for your effort. I have basic questions. 1. Could you tell me what you'd like to do? a) Provide completely the same function with current implementation by other, more efficient way. b) Replace the current implementation with the quicker one that provides the limited function. c) Any other 2. Do you have the estimation that how long does the new algorithm take compare with the current one? # Of course, "currently not sure" is OK at this stage. I'm interested in it because there is the trade-off between speed and memory consumption in many case, and btrfsck takes very long time with a large filesystem. Thanks, Satoru Thanks, Qu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value
On 2016/03/07 12:05, Satoru Takeuchi wrote: > - It's better to show a warning message for the exceptional case >that one of objectid (in most case, inode number) reaches its >highest value. Show this message only once to avoid filling >dmesg with it. > - EOVERFLOW is more proper return value for this case. >ENOSPC is for "No space left on device" case and objectid isn't >related to any device. > > Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> > --- > This patch can be applied to 4.5-rc7 Sorry, I forgot to add changelog. === V2: - Reflect Filipe's comment - Use btrfs_warn() instead of WARN_ONCE() - Print the id of the tree === Thanks, Satoru > --- > fs/btrfs/inode-map.c | 10 +- > 1 file changed, 9 insertions(+), 1 deletion(-) > > diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c > index e50316c..f5e3228 100644 > --- a/fs/btrfs/inode-map.c > +++ b/fs/btrfs/inode-map.c > @@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, > u64 *objectid) > mutex_lock(>objectid_mutex); > > if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) { > - ret = -ENOSPC; > + static bool __warned = false; > + > + if (unlikely(!__warned)) { > + btrfs_warn(root->fs_info, > +"The objectid of root %llu reaches its > highest value.\n", > +root->root_key.objectid); > + __warned = true; > + } > + ret = -EOVERFLOW; > goto out; > } > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value
- It's better to show a warning message for the exceptional case that one of objectid (in most case, inode number) reaches its highest value. Show this message only once to avoid filling dmesg with it. - EOVERFLOW is more proper return value for this case. ENOSPC is for "No space left on device" case and objectid isn't related to any device. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to 4.5-rc7 --- fs/btrfs/inode-map.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c index e50316c..f5e3228 100644 --- a/fs/btrfs/inode-map.c +++ b/fs/btrfs/inode-map.c @@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 *objectid) mutex_lock(>objectid_mutex); if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) { - ret = -ENOSPC; + static bool __warned = false; + + if (unlikely(!__warned)) { + btrfs_warn(root->fs_info, + "The objectid of root %llu reaches its highest value.\n", + root->root_key.objectid); + __warned = true; + } + ret = -EOVERFLOW; goto out; } -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Show a warning message if one of highest objectid reaches its max value
Hi Filipe, On 2016/03/04 18:28, Filipe Manana wrote: On Fri, Mar 4, 2016 at 2:37 AM, Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> wrote: - It's better to show a warning message for the exceptional case that one of highest objectid (in most case, inode number) reaches its max value, BTRFS_LAST_FREE_OBJECTID. Show this message only once to avoid filling dmesg with it. - EOVERFLOW is more proper return value for this case. ENOSPC is for "No space left on device" case and objectid isn't related to any device. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to v4.5-rc6 --- fs/btrfs/inode-map.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c index e50316c..a4860fd 100644 --- a/fs/btrfs/inode-map.c +++ b/fs/btrfs/inode-map.c @@ -555,8 +555,10 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 *objectid) int ret; mutex_lock(>objectid_mutex); - if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) { - ret = -ENOSPC; + if (WARN_ONCE(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID, + "BTRFS: The highest objectid reaches its max value %llu.\n", + BTRFS_LAST_FREE_OBJECTID)) { Please, use btrfs_warn() to print messages... It avoids hardcoding prefixes and having to copy the style of the btrfs_[info|warn|error|...) functions. Also include the id of the tree (root->root_key.objectid), it's much more useful than printing root->highes_objectid Also add the prefix "Btrfs: " to the subject. Thank you for your comment. I'll fix it. + ret = -EOVERFLOW; goto out; } -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Show a warning message if one of highest objectid reaches its max value
- It's better to show a warning message for the exceptional case that one of highest objectid (in most case, inode number) reaches its max value, BTRFS_LAST_FREE_OBJECTID. Show this message only once to avoid filling dmesg with it. - EOVERFLOW is more proper return value for this case. ENOSPC is for "No space left on device" case and objectid isn't related to any device. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- This patch can be applied to v4.5-rc6 --- fs/btrfs/inode-map.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c index e50316c..a4860fd 100644 --- a/fs/btrfs/inode-map.c +++ b/fs/btrfs/inode-map.c @@ -555,8 +555,10 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 *objectid) int ret; mutex_lock(>objectid_mutex); - if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) { - ret = -ENOSPC; + if (WARN_ONCE(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID, + "BTRFS: The highest objectid reaches its max value %llu.\n", + BTRFS_LAST_FREE_OBJECTID)) { + ret = -EOVERFLOW; goto out; } -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: fix listxattrs not listing all xattrs packed in the same item
On 2016/02/23 2:52, fdman...@kernel.org wrote: From: Filipe Manana <fdman...@suse.com> In the listxattrs handler, we were not listing all the xattrs that are packed in the same btree item, which happens when multiple xattrs have a name that when crc32c hashed produce the same checksum value. Fix this by processing them all. The following test case for xfstests reproduces the issue: seq=`basename $0` seqres=$RESULT_DIR/$seq echo "QA output created by $seq" tmp=/tmp/$$ status=1 # failure is the default! trap "_cleanup; exit \$status" 0 1 2 3 15 _cleanup() { cd / rm -f $tmp.* } # get standard environment, filters and checks . ./common/rc . ./common/filter . ./common/attr # real QA test starts here _supported_fs generic _supported_os Linux _require_scratch _require_attrs rm -f $seqres.full _scratch_mkfs >>$seqres.full 2>&1 _scratch_mount # Create our test file with a few xattrs. The first 3 xattrs have a name # that when given as input to a crc32c function result in the same checksum. # This made btrfs list only one of the xattrs through listxattrs system call # (because it packs xattrs with the same name checksum into the same btree # item). touch $SCRATCH_MNT/testfile $SETFATTR_PROG -n user.foobar -v 123 $SCRATCH_MNT/testfile $SETFATTR_PROG -n user.WvG1c1Td -v qwerty $SCRATCH_MNT/testfile $SETFATTR_PROG -n user.J3__T_Km3dVsW_ -v hello $SCRATCH_MNT/testfile $SETFATTR_PROG -n user.something -v pizza $SCRATCH_MNT/testfile $SETFATTR_PROG -n user.ping -v pong $SCRATCH_MNT/testfile # Now call getfattr with --dump, which calls the listxattrs system call. # It should list all the xattrs we have set before. $GETFATTR_PROG --absolute-names --dump $SCRATCH_MNT/testfile | _filter_scratch status=0 exit Tested-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> * chris/integration-4.6(HEAD is 790dd8b) === # ./check generic/337 FSTYP -- btrfs PLATFORM -- Linux/x86_64 fedora23 4.2.7-300.fc23.x86_64 MKFS_OPTIONS -- /dev/vdc MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/vdc /scratch_mnt generic/337 - output mismatch (see /root/xfstests/results//generic/337.out.bad) --- tests/generic/337.out 2016-02-24 07:26:33.0 +0900 +++ /root/xfstests/results//generic/337.out.bad 2016-02-24 07:43:02.47100 +0900 @@ -1,7 +1,5 @@ QA output created by 337 # file: SCRATCH_MNT/testfile -user.J3__T_Km3dVsW_="hello" -user.WvG1c1Td="qwerty" user.foobar="123" user.ping="pong" user.something="pizza" ... (Run 'diff -u tests/generic/337.out /root/xfstests/results//generic/337.out.bad' to see the entire diff) Ran: generic/337 Failures: generic/337 Failed 1 of 1 tests * the above source + your patch # ./check generic/337 FSTYP -- btrfs PLATFORM -- Linux/x86_64 fedora23 4.5.0-rc3-ktest+ MKFS_OPTIONS -- /dev/vdc MOUNT_OPTIONS -- /dev/vdc /scratch_mnt generic/337 0s Ran: generic/337 Passed all 1 tests Thanks, Satoru Signed-off-by: Filipe Manana <fdman...@suse.com> --- fs/btrfs/xattr.c | 62 +++- 1 file changed, 39 insertions(+), 23 deletions(-) diff --git a/fs/btrfs/xattr.c b/fs/btrfs/xattr.c index f2a20d5..caf643d 100644 --- a/fs/btrfs/xattr.c +++ b/fs/btrfs/xattr.c @@ -260,16 +260,12 @@ out: ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size) { - struct btrfs_key key, found_key; + struct btrfs_key key; struct inode *inode = d_inode(dentry); struct btrfs_root *root = BTRFS_I(inode)->root; struct btrfs_path *path; - struct extent_buffer *leaf; - struct btrfs_dir_item *di; - int ret = 0, slot; + int ret = 0; size_t total_size = 0, size_left = size; - unsigned long name_ptr; - size_t name_len; /* * ok we want all objects associated with this id. @@ -291,6 +287,13 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size) goto err; while (1) { + struct extent_buffer *leaf; + int slot; + struct btrfs_dir_item *di; + struct btrfs_key found_key; + u32 item_size; + u32 cur; + leaf = path->nodes[0]; slot = path->slots[0]; @@ -319,28 +322,41 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size) goto next; di = btrfs_item_ptr(leaf, slot,
Re: [PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure
On 2016/02/18 0:11, David Sterba wrote: On Wed, Feb 17, 2016 at 02:54:23PM +0900, Satoru Takeuchi wrote: On 2016/02/16 2:53, David Sterba wrote: On Mon, Feb 15, 2016 at 02:38:09PM +0900, Satoru Takeuchi wrote: There are some BUG_ON()'s after kmalloc() as follows. = foo = kmalloc(); BUG_ON(!foo); /* -ENOMEM case */ = A Docker + memory cgroup user hit these BUG_ON()s. https://bugzilla.kernel.org/show_bug.cgi?id=112101 Since it's very hard to handle these ENOMEMs properly, preventing these kmalloc() failures to avoid these BUG_ON()s for now, are a bit better than the current implementation anyway. Beware that the NOFAIL semantics is can cause deadlocks if it's on the critical writeback path or and can be reentered from itself through the reclaim. Unless you're sure that this is not the case, please do not add them just because it would seemingly fix the allocation failures. About the all cases I changed, kmalloc()s can block since gfp_flags_allow_blocking() are true. Then no locks are acquired here and deadlocks don't happen. Am I missing something? Waiting as in GFP_WAIT is not the same as GFP_NOFAIL that can wait indefinetelly. The locking of NOFAIL is implied. The kmalloc callsite will block until there's memory available. If another thread of btrfs waits for this code to progress, it will block as well. In the docker example, the memory is limited by cgroups so the NOFAIL mode can exhaust all reserves and just loop endlessly waiting for the OOM killer to get some memory or just waiting without any chance to progress. I consider triggering OOM killer and killing processes in a cgroup are better than killing whole system. The action of OOM killer is not a problem. The cgroup memory limit can be low or all the memory is unreclaimable. At this point btrfs code will block. About the possibility of endless loop, there are many such problems in the whole kernel. Of course it can be said to Btrfs. Many? Examples? In this context we're talking about endless loops caused by non-failing allocations. == $ grep -rnH __GFP_NOFAIL fs/btrfs/ fs/btrfs/extent-tree.c:5970: GFP_NOFS | __GFP_NOFAIL); fs/btrfs/extent-tree.c:6043: bytenr + num_bytes - 1, GFP_NOFS | __GFP_NOFAIL); fs/btrfs/extent_io.c:4643: eb = kmem_cache_zalloc(extent_buffer_cache, GFP_NOFS|__GFP_NOFAIL); fs/btrfs/extent_io.c:4909: p = find_or_create_page(mapping, index, GFP_NOFS|__GFP_NOFAIL); == I'm aware of the existing NOFAIL allocations. There are two at extent_buffer allocation time, added recently and provoked by the intended changes to memory allocator that would fail the GFP_NOFS allocations. The other two are from year 2010, set_extent_bit, IMHO added so the error handling does not get complicated and expressing "we don't want to fail here". But there are many other calls to set_extent_bit that could fail. This is inconsistent and should be unified. In a way that we're sure that we're not introducing potential hangs. I understand fixing these problems cooperate with memory cgroup guys is the best in the long run. It's not about cgroups, btrfs needs to ideally handle all memory allocation failures in a way that uses some sort of fallbacks but still can lead to a transaction abort if there's simply no memory left. However, I consider bypassing this problem for now is better than the current implementation. It's more like replacing one problem with another. With every new NOFAIL, one has to think about the runtime interactions with the others. I'd rather see this fixed with a particular call path in mind or class, eg. the extent_map bit settings, than throwing NOFAIL at places that somebody accidentally hit. As a temporary fix we can add __GFP_HIGH to the interesting sites so we can get access to the emergency reserves, and this is on my list of things to do after the NOFS -> KERNEL updates are done. OK, got it. I'll reconsider how to fix there problem. Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure
On 2016/02/16 2:53, David Sterba wrote: On Mon, Feb 15, 2016 at 02:38:09PM +0900, Satoru Takeuchi wrote: There are some BUG_ON()'s after kmalloc() as follows. = foo = kmalloc(); BUG_ON(!foo); /* -ENOMEM case */ = A Docker + memory cgroup user hit these BUG_ON()s. https://bugzilla.kernel.org/show_bug.cgi?id=112101 Since it's very hard to handle these ENOMEMs properly, preventing these kmalloc() failures to avoid these BUG_ON()s for now, are a bit better than the current implementation anyway. Beware that the NOFAIL semantics is can cause deadlocks if it's on the critical writeback path or and can be reentered from itself through the reclaim. Unless you're sure that this is not the case, please do not add them just because it would seemingly fix the allocation failures. About the all cases I changed, kmalloc()s can block since gfp_flags_allow_blocking() are true. Then no locks are acquired here and deadlocks don't happen. Am I missing something? In the docker example, the memory is limited by cgroups so the NOFAIL mode can exhaust all reserves and just loop endlessly waiting for the OOM killer to get some memory or just waiting without any chance to progress. I consider triggering OOM killer and killing processes in a cgroup are better than killing whole system. About the possibility of endless loop, there are many such problems in the whole kernel. Of course it can be said to Btrfs. == $ grep -rnH __GFP_NOFAIL fs/btrfs/ fs/btrfs/extent-tree.c:5970: GFP_NOFS | __GFP_NOFAIL); fs/btrfs/extent-tree.c:6043: bytenr + num_bytes - 1, GFP_NOFS | __GFP_NOFAIL); fs/btrfs/extent_io.c:4643: eb = kmem_cache_zalloc(extent_buffer_cache, GFP_NOFS|__GFP_NOFAIL); fs/btrfs/extent_io.c:4909: p = find_or_create_page(mapping, index, GFP_NOFS|__GFP_NOFAIL); == I understand fixing these problems cooperate with memory cgroup guys is the best in the long run. However, I consider bypassing this problem for now is better than the current implementation. Thanks, Satoru -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure
There are some BUG_ON()'s after kmalloc() as follows. = foo = kmalloc(); BUG_ON(!foo); /* -ENOMEM case */ = A Docker + memory cgroup user hit these BUG_ON()s. https://bugzilla.kernel.org/show_bug.cgi?id=112101 Since it's very hard to handle these ENOMEMs properly, preventing these kmalloc() failures to avoid these BUG_ON()s for now, are a bit better than the current implementation anyway. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- fs/btrfs/extent_io.c | 6 ++ fs/btrfs/inode.c | 6 ++ fs/btrfs/relocation.c | 3 +-- 3 files changed, 5 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 2e7c97a..5f92450 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -874,10 +874,8 @@ __set_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, bits |= EXTENT_FIRST_DELALLOC; again: - if (!prealloc && gfpflags_allow_blocking(mask)) { - prealloc = alloc_extent_state(mask); - BUG_ON(!prealloc); - } + if (!prealloc && gfpflags_allow_blocking(mask)) + prealloc = alloc_extent_state(mask|__GFP_NOFAIL); spin_lock(>lock); if (cached_state && *cached_state) { diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 85afe66..d20e5c5 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -357,8 +357,7 @@ static noinline int add_async_extent(struct async_cow *cow, { struct async_extent *async_extent; - async_extent = kmalloc(sizeof(*async_extent), GFP_NOFS); - BUG_ON(!async_extent); /* -ENOMEM */ + async_extent = kmalloc(sizeof(*async_extent), GFP_NOFS|__GFP_NOFAIL); async_extent->start = start; async_extent->ram_size = ram_size; async_extent->compressed_size = compressed_size; @@ -1143,8 +1142,7 @@ static int cow_file_range_async(struct inode *inode, struct page *locked_page, clear_extent_bit(_I(inode)->io_tree, start, end, EXTENT_LOCKED, 1, 0, NULL, GFP_NOFS); while (start < end) { - async_cow = kmalloc(sizeof(*async_cow), GFP_NOFS); - BUG_ON(!async_cow); /* -ENOMEM */ + async_cow = kmalloc(sizeof(*async_cow), GFP_NOFS|__GFP_NOFAIL); async_cow->inode = igrab(inode); async_cow->root = root; async_cow->locked_page = locked_page; diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index ef6d8fc..6b9f718 100644 --- a/fs/btrfs/relocation.c +++ b/fs/btrfs/relocation.c @@ -1373,8 +1373,7 @@ static struct btrfs_root *create_reloc_root(struct btrfs_trans_handle *trans, u64 last_snap = 0; int ret; - root_item = kmalloc(sizeof(*root_item), GFP_NOFS); - BUG_ON(!root_item); + root_item = kmalloc(sizeof(*root_item), GFP_NOFS|__GFP_NOFAIL); root_key.objectid = BTRFS_TREE_RELOC_OBJECTID; root_key.type = BTRFS_ROOT_ITEM_KEY; -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: write down the meaning of BTRFS_ARG_BLKDEV
Although BTRFS_ARG_BLKDEV can be returned from check_arg_type(), it's not explained the meaning. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- utils.c | 1 + 1 file changed, 1 insertion(+) diff --git a/utils.c b/utils.c index 3df8b42..eabc36d 100644 --- a/utils.c +++ b/utils.c @@ -1007,6 +1007,7 @@ static int is_reg_file(const char *path) * return BTRFS_ARG_UUID: given input is uuid * return BTRFS_ARG_MNTPOINT: given input is path * return BTRFS_ARG_REG: given input is regular file + * return BTRFS_ARG_BLKDEV:given input is block device */ int check_arg_type(const char *input) { -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: Fix self-reference of man btrfs-subvolume
btrfs-subvolume(8) is mentioned at "SEE ALSO" section of itself. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- Documentation/btrfs-subvolume.asciidoc | 1 - 1 file changed, 1 deletion(-) diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc index c187fd8..96cfe4a 100644 --- a/Documentation/btrfs-subvolume.asciidoc +++ b/Documentation/btrfs-subvolume.asciidoc @@ -178,6 +178,5 @@ further details. SEE ALSO `mkfs.btrfs`(8), -`btrfs-subvolume`(8), `btrfs-quota`(8), `btrfs-qgroup`(8), -- 2.7.0.rc3 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: describe btrfs-send requires read-only subvolume
Both man btrfs-send(8) and usage message don't describe btrfs-send needs read-only snapshot as its argument. Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com> --- Documentation/btrfs-send.asciidoc | 1 + cmds-send.c | 1 + 2 files changed, 2 insertions(+) diff --git a/Documentation/btrfs-send.asciidoc b/Documentation/btrfs-send.asciidoc index 1dba8a3..e05342f 100644 --- a/Documentation/btrfs-send.asciidoc +++ b/Documentation/btrfs-send.asciidoc @@ -12,6 +12,7 @@ SYNOPSIS DESCRIPTION --- Sends the subvolume(s) specified by to stdout. + should be read-only here. By default, this will send the whole subvolume. To do an incremental send, use '-p '. diff --git a/cmds-send.c b/cmds-send.c index 478ace1..3e34d75 100644 --- a/cmds-send.c +++ b/cmds-send.c @@ -711,6 +711,7 @@ const char * const cmd_send_usage[] = { "btrfs send [-ve] [-p ] [-c ] [-f ] [...]", "Send the subvolume(s) to stdout.", "Sends the subvolume(s) specified by to stdout.", + " should be read-only here.", "By default, this will send the whole subvolume. To do an incremental", "send, use '-p '. If you want to allow btrfs to clone from", "any additional local snapshots, use '-c ' (multiple times", -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug in 3.19-rc4
Hi, On 2015/01/16 10:05, Tomasz Chmielewski wrote: I just started some btrfs stress testing on latest linux kernel 3.19-rc4: A few hours later, filesystem stopped working - the kernel bug report can be found below. Hi, your kernel BUG at fs/btrfs/inode.c:3142! from 3.19-rc4 corresponds to http://marc.info/?l=linux-btrfsm=141903172106342w=2 - it was kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123! in 3.18.1, and is exactly the same code in both cases: /* grab metadata reservation from transaction handle */ if (reserve) { ret = btrfs_orphan_reserve_metadata(trans, inode); BUG_ON(ret); /* -ENOSPC in reservation; Logic error? JDM */ } Year, it's the same. BTW, I've tried to reproduce this problem by using the way you told me. However, it hasn't reproduced yet. Thanks, Satoru -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug in 3.19-rc4
Hi Marcel, On 2015/01/16 4:46, Marcel Ritter wrote: Hi, I just started some btrfs stress testing on latest linux kernel 3.19-rc4: A few hours later, filesystem stopped working - the kernel bug report can be found below. The test consists of one massive IO thread (writing 100GB files with dd), and 2 tar instances extracting kernel sources and deleting them afterwards (I can provide the simple bash script doing this, if needed). Could you give me this script? Thanks, Satoru System information (Ubuntu 14.04.1, latest kernel): root@thunder # uname -a Linux thunder 3.19.0-rc4-custom #1 SMP Mon Jan 12 16:13:44 CET 2015 x86_64 x86_64 x86_64 GNU/Linux root@thunder # /root/btrfs-progs/btrfs --version Btrfs v3.18-36-g0173148 Tests are done on 14 SCSI disks, using raid6 for data and metadata: root@thunder # /root/btrfs-progs/btrfs fi show Label: 'raid6' uuid: cbe34d2b-5f75-46cf-9263-9813028ebc19 Total devices 14 FS bytes used 674.62GiB devid1 size 279.39GiB used 59.24GiB path /dev/cciss/c1d0 devid2 size 279.39GiB used 59.22GiB path /dev/cciss/c1d1 devid3 size 279.39GiB used 59.22GiB path /dev/cciss/c1d10 devid4 size 279.39GiB used 59.22GiB path /dev/cciss/c1d11 devid5 size 279.39GiB used 59.22GiB path /dev/cciss/c1d12 devid6 size 279.39GiB used 59.22GiB path /dev/cciss/c1d13 devid7 size 279.39GiB used 59.22GiB path /dev/cciss/c1d2 devid8 size 279.39GiB used 59.22GiB path /dev/cciss/c1d3 devid9 size 279.39GiB used 59.22GiB path /dev/cciss/c1d4 devid 10 size 279.39GiB used 59.22GiB path /dev/cciss/c1d5 devid 11 size 279.39GiB used 59.22GiB path /dev/cciss/c1d6 devid 12 size 279.39GiB used 59.22GiB path /dev/cciss/c1d7 devid 13 size 279.39GiB used 59.22GiB path /dev/cciss/c1d8 devid 14 size 279.39GiB used 59.22GiB path /dev/cciss/c1d9 Btrfs v3.18-36-g0173148 # This is provided for completeness only, and is taken # somewhen *before* the kernel crash occured, so basic # setup is the same, but allocated/free sizes won't match root@thunder # /root/btrfs-progs/btrfs fi df /tmp/m Data, single: total=8.00MiB, used=0.00B Data, RAID6: total=727.45GiB, used=697.84GiB System, single: total=4.00MiB, used=0.00B System, RAID6: total=13.50MiB, used=64.00KiB Metadata, single: total=8.00MiB, used=0.00B Metadata, RAID6: total=3.43GiB, used=805.91MiB GlobalReserve, single: total=272.00MiB, used=0.00B Here's what happens after some hours of stress testing: [85162.472989] [ cut here ] [85162.473071] kernel BUG at fs/btrfs/inode.c:3142! [85162.473139] invalid opcode: [#1] SMP [85162.473212] Modules linked in: btrfs(E) xor(E) raid6_pq(E) radeon(E) ttm(E) drm_kms_helper(E) drm(E) hpwdt(E) amd64_edac_mod(E) kvm(E) edac_core(E) shpchp(E) k8temp(E) serio_raw(E) hpilo(E) edac_mce_amd(E) mac_hid(E) i2c_algo_bit(E) ipmi_si(E) nfsd(E) auth_rpcgss(E) nfs_acl(E) nfs(E) lockd(E) grace(E) sunrpc(E) lp(E) fscache(E) parport(E) hid_generic(E) usbhid(E) hid(E) hpsa(E) psmouse(E) bnx2(E) cciss(E) pata_acpi(E) pata_amd(E) [85162.473911] CPU: 4 PID: 3039 Comm: btrfs-cleaner Tainted: G E 3.19.0-rc4-custom #1 [85162.474028] Hardware name: HP ProLiant DL585 G2 , BIOS A07 05/02/2011 [85162.474122] task: 88085b054aa0 ti: 88205ad4c000 task.ti: 88205ad4c000 [85162.474230] RIP: 0010:[a06a8182] [a06a8182] btrfs_orphan_add+0x1d2/0x1e0 [btrfs] [85162.474422] RSP: 0018:88205ad4fc48 EFLAGS: 00010286 [85162.474497] RAX: ffe4 RBX: 8810a35d42f8 RCX: 88185b896000 [85162.474595] RDX: 6a54 RSI: 0004 RDI: 88185b896138 [85162.474694] RBP: 88205ad4fc88 R08: 0001e670 R09: 88016194b240 [85162.474793] R10: a06bd797 R11: ea0004f71800 R12: 88185baa2000 [85162.474892] R13: 88085f6d7630 R14: 88185baa2458 R15: 0001 [85162.474992] FS: 7fb3f27fb740() GS:88085fd0() knlGS: [85162.475105] CS: 0010 DS: ES: CR0: 8005003b [85162.475184] CR2: 7f896c02c220 CR3: 00085b328000 CR4: 07e0 [85162.475286] Stack: [85162.475318] 88205ad4fc88 a06e6a14 88185b896b04 88105b03e800 [85162.475442] 88016194b240 8810a35d42f8 881e8ffe9a00 88133dc48ea0 [85162.475561] 88205ad4fd18 a0691a57 88016194b244 88016194b240 [85162.475680] Call Trace: [85162.475738] [a06e6a14] ? lookup_free_space_inode+0x44/0x100 [btrfs] [85162.475849] [a0691a57] btrfs_remove_block_group+0x137/0x740 [btrfs] [85162.475964] [a06ca8d2] btrfs_remove_chunk+0x672/0x780 [btrfs] [85162.476065] [a06922bf] btrfs_delete_unused_bgs+0x25f/0x280 [btrfs] [85162.476172] [a0699e0c] cleaner_kthread+0x12c/0x190 [btrfs] [85162.476269] [a0699ce0] ? check_leaf+0x350/0x350 [btrfs] [85162.476355] [8108f8d2]
Re: [PATCH] btrfs: get the accurate value of used_bytes in btrfs_get_block_group_info().
Hi Dongsheng, On 2015/01/05 20:19, Dongsheng Yang wrote: Ping. IOCTL of BTRFS_IOC_SPACE_INFO currently does not report the data used but not synced to user. Then btrfs fi df will give user a wrong numbers before sync. This patch solve this problem. On 10/27/2014 08:38 PM, Dongsheng Yang wrote: Reproducer: # mkfs.btrfs -f -b 20G /dev/sdb # mount /dev/sdb /mnt/test # fallocate -l 17G /mnt/test/largefile # btrfs fi df /mnt/test Data, single: total=17.49GiB, used=6.00GiB - only 6G, but actually it should be 17G. System, DUP: total=8.00MiB, used=16.00KiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=1.00GiB, used=112.00KiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=16.00MiB, used=0.00B I tried to reproduce your problem with 3.19-rc1. However, this problem doesn't happen. Could you also try to reproduce with the upstream kernel? * Detail test script (named yang-test.sh here): === #!/bin/bash -x PART1=/dev/vdb MNT_PNT=./mnt mkfs.btrfs -f -b 20G ${PART1} mount ${PART1} ${MNT_PNT} fallocate -l 17G ${MNT_PNT}/largefile btrfs fi df ${MNT_PNT} sync btrfs fi df ${MNT_PNT} umount ${MNT_PNT} === Result: === # ./yang-test.sh + PART1=/dev/vdb + MNT_PNT=./mnt + mkfs.btrfs -f -b 20G /dev/vdb Btrfs v3.17 See http://btrfs.wiki.kernel.org for more information. Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 fs created label (null) on /dev/vdb nodesize 16384 leafsize 16384 sectorsize 4096 size 20.00GiB + mount /dev/vdb ./mnt + fallocate -l 17G ./mnt/largefile + btrfs fi df ./mnt Data, single: total=17.01GiB, used=17.00GiB # Used 17GiB properly System, DUP: total=8.00MiB, used=16.00KiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=1.00GiB, used=112.00KiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=16.00MiB, used=0.00B + sync + btrfs fi df ./mnt Data, single: total=17.01GiB, used=17.00GiB# (of course) used 17GiB too System, DUP: total=8.00MiB, used=16.00KiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=1.00GiB, used=112.00KiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=16.00MiB, used=0.00B + umount ./mnt === Although I ran this test five times, the results are the same. Thanks, Satoru # sync # btrfs fi df /mnt/test Data, single: total=17.49GiB, used=17.00GiB - After sync, it is expected. System, DUP: total=8.00MiB, used=16.00KiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=1.00GiB, used=112.00KiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=16.00MiB, used=0.00B The value of 6.00GiB is actually calculated in btrfs_get_block_group_info() by adding the @block_group-item-used for each group together. In this way, it did not consider the bytes in cache. This patch adds the value of @pinned, @reserved and @bytes_super in struct btrfs_block_group_cache to make sure we can get the accurate @used_bytes. Reported-by: Qu Wenruo quwen...@cn.fujitsu.com Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com --- fs/btrfs/ioctl.c | 4 1 file changed, 4 insertions(+) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 33c80f5..bc2aaeb 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -3892,6 +3892,10 @@ void btrfs_get_block_group_info(struct list_head *groups_list, space-total_bytes += block_group-key.offset; space-used_bytes += btrfs_block_group_used(block_group-item); +/* Add bytes-info in cache */ +space-used_bytes += block_group-pinned; +space-used_bytes += block_group-reserved; +space-used_bytes += block_group-bytes_super; } } -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs: Fix a copy-n-paste bug in btrfs_read_fs_root().
On 2015/01/07 18:23, Qu Wenruo wrote: Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com --- disk-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/disk-io.c b/disk-io.c index 2bf8586..b853f66 100644 --- a/disk-io.c +++ b/disk-io.c @@ -693,7 +693,7 @@ struct btrfs_root *btrfs_read_fs_root(struct btrfs_fs_info *fs_info, if (location-objectid == BTRFS_CSUM_TREE_OBJECTID) return fs_info-csum_root; if (location-objectid == BTRFS_QUOTA_TREE_OBJECTID) - return fs_info-csum_root; + return fs_info-quota_root; BUG_ON(location-objectid == BTRFS_TREE_RELOC_OBJECTID || location-offset != (u64)-1); -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 0/3] Btrfs: Enhancment for qgroup.
Hi Yang, On 2015/01/05 15:16, Dongsheng Yang wrote: Hi Josef and others, This patch set is about enhancing qgroup. [1/3]: fix a bug about qgroup leak when we exceed quota limit, It is reviewd by Josef. [2/3]: introduce a new accounter in qgroup to close a window where user will exceed the limit by qgroup. It looks good to Josef. [3/3]: a new patch to fix a bug reported by Satoru. I tested your the patchset v3. Although it's far better than the patchset v2, there is still one problem in this patchset. When I wrote 1.5GiB to a subvolume with 1.0 GiB limit, 1.0GiB - 139 block (in this case, 1KiB/block) was written. I consider user should be able to write just 1.0GiB in this case. * Test result === + mkfs.btrfs -f /dev/vdb Btrfs v3.17 See http://btrfs.wiki.kernel.org for more information. Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 fs created label (null) on /dev/vdb nodesize 16384 leafsize 16384 sectorsize 4096 size 30.00GiB + mount /dev/vdb /root/btrfs-auto-test/ + ret=0 + btrfs quota enable /root/btrfs-auto-test/ + btrfs subvolume create /root/btrfs-auto-test//sub Create subvolume '/root/btrfs-auto-test/sub' + btrfs qgroup limit 1G /root/btrfs-auto-test//sub + dd if=/dev/zero of=/root/btrfs-auto-test//sub/file bs=1024 count=150 dd: error writing '/root/btrfs-auto-test//sub/file': Disk quota exceeded 1048438+0 records in# Tried to write 1GiB - 138 KiB 1048437+0 records out # Succeeded to write 1GiB - 139 KiB 1073599488 bytes (1.1 GB) copied, 19.0247 s, 56.4 MB/s === * note I tried to run the reproducer five times and the result is a bit different for each time. = # Written - 1 1GiB - 139 KiB 2 1GiB - 139 KiB 3 1GiB - 145 KiB 4 1GiB - 135 KiB 5 1GiB - 135 KiB == So I consider it's a problem comes from timing. If I changed the block size from 1KiB to 1 MiB, the difference in bytes got larger. # Written 1 1GiB - 1 MiB 2 1GiB - 1 MiB 3 1GiB - 1 MiB 4 1GiB - 1 MiB 5 1GiB - 1 MiB Thanks, Satoru BTW, I have some other plan about qgroup in my TODO list: Kernel: a). adjust the accounters in parent qgroup when we move the child qgroup. Currently, when we move a qgroup, the parent qgroup will not updated at the same time. This will cause some wrong numbers in qgroup. b). add a ioctl to show the qgroup info. Command btrfs qgroup show is showing the qgroup info read from qgroup tree. But there is some information in memory which is not synced into device. Then it will show some outdate number. c). limit and account size in 3 modes, data, metadata and both. qgroup is accounting the size both of data and metadata togather, but to a user, the data size is the most useful to them. d). remove a subvolume related qgroup when subvolume is deleted and there is no other reference to it. user-tool: a). Add the unit of B/K/M/G to btrfs qgroup show. b). get the information via ioctl rather than reading it from btree. Will keep the old way as a fallback for compatiblity. Any comment and sugguestion is welcome. :) Yang Dongsheng Yang (3): Btrfs: qgroup: free reserved in exceeding quota. Btrfs: qgroup: Introduce a may_use to account space_info-bytes_may_use. Btrfs: qgroup, Account data space in more proper timings. fs/btrfs/extent-tree.c | 41 +++--- fs/btrfs/file.c| 9 --- fs/btrfs/inode.c | 18 - fs/btrfs/qgroup.c | 68 +++--- fs/btrfs/qgroup.h | 4 +++ 5 files changed, 117 insertions(+), 23 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: clear bio reference after submit_one_bio()
Hi Naota, On 2015/01/06 1:01, Naohiro Aota wrote: After submit_one_bio(), `bio' can go away. However submit_extent_page() leave `bio' referable if submit_one_bio() failed (e.g. -ENOMEM on OOM). It will cause invalid paging request when submit_extent_page() is called next time. I reproduced ENOMEM case with the following script (need CONFIG_FAIL_PAGE_ALLOC, and CONFIG_FAULT_INJECTION_DEBUG_FS). I confirmed that this problem reproduce with 3.19-rc3 and not reproduce with 3.19-rc3 with your patch. Tested-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com Thank you for reporting this problem with the reproducer and fixing it too. NOTE: I used v3.19-rc3's tools/testing/fault-injection/failcmd.sh for the following ./failcmd.sh. ./failcmd.sh -p $percent -t $times -i $interval \ --ignore-gfp-highmem=N --ignore-gfp-wait=N --min-order=0 \ -- \ cat $directory/file /dev/null * 3.19-rc1 + your patch === # ./run 512+0 records in 512+0 records out # === * 3.19-rc3 === # ./run 512+0 records in 512+0 records out [ 188.433726] run (776): drop_caches: 1 [ 188.682372] FAULT_INJECTION: forcing a failure. name fail_page_alloc, interval 100, probability 111000, space 0, times 3 [ 188.689986] CPU: 0 PID: 954 Comm: cat Not tainted 3.19.0-rc3-ktest #1 [ 188.693834] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 [ 188.698466] 0064 88007b343618 816e5563 88007fc0fc78 [ 188.702730] 81c655c0 88007b343638 813851b5 0010 [ 188.707043] 0002 88007b343768 81188126 88007b3435a8 [ 188.711283] Call Trace: [ 188.712620] [816e5563] dump_stack+0x45/0x57 [ 188.715330] [813851b5] should_fail+0x135/0x140 [ 188.718218] [81188126] __alloc_pages_nodemask+0xd6/0xb30 [ 188.721567] [81339075] ? blk_rq_map_sg+0x35/0x170 [ 188.724558] [a0010705] ? virtio_queue_rq+0x145/0x2b0 [virtio_blk] [ 188.728191] [a01bd00f] ? btrfs_submit_compressed_read+0xcf/0x4d0 [btrfs] [ 188.732079] [811d99fb] ? kmem_cache_alloc+0x1cb/0x230 [ 188.735153] [81181265] ? mempool_alloc_slab+0x15/0x20 [ 188.738188] [811cee1a] alloc_pages_current+0x9a/0x120 [ 188.741153] [a01bd0e9] btrfs_submit_compressed_read+0x1a9/0x4d0 [btrfs] [ 188.744835] [a0178621] btrfs_submit_bio_hook+0x1c1/0x1d0 [btrfs] [ 188.748225] [a018b7b3] ? lookup_extent_mapping+0x13/0x20 [btrfs] [ 188.751547] [a0179c08] ? btrfs_get_extent+0x98/0xad0 [btrfs] [ 188.754656] [a01901d7] submit_one_bio+0x67/0xa0 [btrfs] [ 188.757554] [a0193f27] submit_extent_page.isra.35+0xd7/0x1c0 [btrfs] [ 188.760981] [a019509d] __do_readpage+0x31d/0x7b0 [btrfs] [ 188.763920] [a0195f10] ? btrfs_create_repair_bio+0x110/0x110 [btrfs] [ 188.767382] [a0179b70] ? btrfs_submit_direct+0x7b0/0x7b0 [btrfs] [ 188.770671] [a018f88d] ? btrfs_lookup_ordered_range+0x13d/0x180 [btrfs] [ 188.774366] [a01958ca] __extent_readpages.constprop.42+0x2ba/0x2d0 [btrfs] [ 188.778031] [a0179b70] ? btrfs_submit_direct+0x7b0/0x7b0 [btrfs] [ 188.781241] [a01969b9] extent_readpages+0x169/0x1b0 [btrfs] [ 188.784322] [a0179b70] ? btrfs_submit_direct+0x7b0/0x7b0 [btrfs] [ 188.789014] [a0176b0f] btrfs_readpages+0x1f/0x30 [btrfs] [ 188.792028] [8118bf5c] __do_page_cache_readahead+0x18c/0x1f0 [ 188.795078] [8118c09f] ondemand_readahead+0xdf/0x260 [ 188.797702] [a016c5df] ? btrfs_congested_fn+0x5f/0xa0 [btrfs] [ 188.800718] [8118c291] page_cache_async_readahead+0x71/0xa0 [ 188.803650] [8118017f] generic_file_read_iter+0x40f/0x5e0 [ 188.806480] [811f43be] new_sync_read+0x7e/0xb0 [ 188.808832] [811f55d8] __vfs_read+0x18/0x50 [ 188.811068] [811f569a] vfs_read+0x8a/0x140 [ 188.813298] [811f5796] SyS_read+0x46/0xb0 [ 188.815486] [81125806] ? __audit_syscall_exit+0x1f6/0x2a0 [ 188.818293] [816eb8e9] system_call_fastpath+0x12/0x17 [ 188.821005] BUG: unable to handle kernel paging request at 0001000c [ 188.821984] IP: [a01901b3] submit_one_bio+0x43/0xa0 [btrfs] [ 188.821984] PGD 7bad3067 PUD 0 [ 188.821984] Oops: [#1] SMP [ 188.821984] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables bnep bluetooth rfkill btrfs xor raid6_pq microcode 8139too serio_raw virtio_balloon 8139cp mii nfsd auth_rpcgss nfs_acl lockd grace sunrpc virtio_blk ata_generic pata_acpi [ 188.821984] CPU: 1 PID: 954 Comm: cat Not tainted 3.19.0-rc3-ktest #1
Re: [PATCH] btrfs: qgroup: move WARN_ON() to the correct location.
On 2015/01/06 21:54, Dongsheng Yang wrote: In function qgroup_excl_accounting(), we need to WARN when qg-excl is less than what we want to free, same to child and parents. But currently, for parent qgroup, the WARN_ON() is located after freeing qg-excl. It will WARN out even we free it normally. This patch move this WARN_ON() before freeing qg-excl. Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com --- fs/btrfs/qgroup.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 48b60db..97159a8 100644 --- a/fs/btrfs/qgroup.c +++ b/fs/btrfs/qgroup.c @@ -1431,9 +1431,8 @@ static int qgroup_excl_accounting(struct btrfs_fs_info *fs_info, qgroup = u64_to_ptr(unode-aux); qgroup-rfer += sign * oper-num_bytes; qgroup-rfer_cmpr += sign * oper-num_bytes; + WARN_ON(sign 0 qgroup-excl oper-num_bytes); qgroup-excl += sign * oper-num_bytes; - if (sign 0) - WARN_ON(qgroup-excl oper-num_bytes); qgroup-excl_cmpr += sign * oper-num_bytes; qgroup_dirty(fs_info, qgroup); -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123!
Hi Tomasz, On 2014/12/20 8:28, Tomasz Chmielewski wrote: Get this BUG with 3.18.1 (pasted at the bottom of the email). Below all actions from creating the fs to BUG. I did not attempt to reproduce. I tried to reproduce this problem and have some questions. # mkfs.btrfs /dev/vdb Btrfs v3.17.3 See http://btrfs.wiki.kernel.org for more information. Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 fs created label (null) on /dev/vdb nodesize 16384 leafsize 16384 sectorsize 4096 size 256.00GiB # mount -o noatime /dev/vdb /mnt/test/ # cd /mnt/test # btrfs sub cre subvolume Create subvolume './subvolume' # dd if=/dev/urandom of=bigfile.img bs=64k Does it really this command? I consider it will fill up whole /dev/vdb. And is it not subvolume/bigfile.img but bigfile.img? ^C91758+0 records in 91757+0 records out 6013386752 bytes (6.0 GB) copied, 374.777 s, 16.0 MB/s # btrfs sub list /mnt/test/ ID 257 gen 16 top level 5 path subvolume # btrfs quota enable /mnt/test # btrfs qgroup show /mnt/test qgroupid rfer excl 0/5 16384 16384 0/2576013403136 6013403136 # dd if=/dev/urandom of=bigfile2.img bs=64k ^C47721+0 records in 47720+0 records out 3127377920 bytes (3.1 GB) copied, 194.641 s, 16.1 MB/s If bigfile.img is just under /mnt/test, I can't understand why this command succeeded to write more 3 GiB. # btrfs qgroup show /mnt/test qgroupid rfer excl 0/5 16384 16384 0/2578704049152 8704049152 root@srv2:/mnt/test/subvolume# sync root@srv2:/mnt/test/subvolume# btrfs qgroup show /mnt/test qgroupid rfer excl 0/5 16384 16384 0/2579140781056 9140781056 # dd if=/dev/urandom of=bigfile3.img bs=64k ^C3617580+0 records in 3617579+0 records out 237081657344 bytes (237 GB) copied, 14796 s, 16.0 MB/s It's too. Thanks, Satoru # df -h Filesystem Size Used Avail Use% Mounted on (...) /dev/vdb256G 230G 25G 91% /mnt/test # btrfs qgroup show /mnt/test qgroupid rfer excl 0/5 1638416384 0/257245960245248 245960245248 # ls -l total 240451584 -rw-r--r-- 1 root root 3127377920 Dec 19 20:06 bigfile2.img -rw-r--r-- 1 root root 237081657344 Dec 20 00:15 bigfile3.img -rw-r--r-- 1 root root 6013386752 Dec 19 20:02 bigfile.img # rm bigfile3.img # sync # dmesg (...) [ 95.055420] BTRFS: device fsid 97f98279-21e7-4822-89be-3aed9dc05f2c devid 1 transid 3 /dev/vdb [ 118.446509] BTRFS info (device vdb): disk space caching is enabled [ 118.446518] BTRFS: flagging fs with big metadata feature [ 118.452176] BTRFS: creating UUID tree [ 575.189412] BTRFS info (device vdb): qgroup scan completed [15948.234826] [ cut here ] [15948.234883] kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123! [15948.234906] invalid opcode: [#1] SMP [15948.234925] Modules linked in: nf_log_ipv6 ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables nf_log_ipv4 nf_log_common xt_LOG ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables dm_crypt btrfs xor crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel ppdev aes_x86_64 lrw raid6_pq gf128mul glue_helper ablk_helper cryptd serio_raw mac_hid pvpanic 8250_fintek parport_pc i2c_piix4 lp parport psmouse qxl ttm floppy drm_kms_helper drm [15948.235172] CPU: 0 PID: 3274 Comm: btrfs-cleaner Not tainted 3.18.1-031801-generic #201412170637 [15948.235193] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 [15948.235222] task: 880036708a00 ti: 88007b97c000 task.ti: 88007b97c000 [15948.235240] RIP: 0010:[c0458ec9] [c0458ec9] btrfs_orphan_add+0x1a9/0x1c0 [btrfs] [15948.235305] RSP: 0018:88007b97fc98 EFLAGS: 00010286 [15948.235318] RAX: ffe4 RBX: 88007b80a800 RCX: [15948.235333] RDX: 219e RSI: 0004 RDI: 880079418138 [15948.235349] RBP: 88007b97fcd8 R08: 88007fc1cae0 R09: 88007ad272d0 [15948.235366] R10: R11: 0010 R12: 88007a2d9500 [15948.235381] R13: 8800027d60e0 R14: 88007b80ac58 R15: 0001 [15948.235401] FS: () GS:88007fc0() knlGS: [15948.235418] CS: 0010 DS: ES: CR0: 80050033 [15948.235432] CR2: 7f0489ff CR3: 7a5e CR4: 001407f0 [15948.235464] Stack: [15948.235473] 88007b97fcd8 c0497acf 88007b809800 88003c207400 [15948.235498] 88007b809800 88007ad272d0 88007a2d9500 0001 [15948.235521] 88007b97fd58 c04412e0 880079418000 0004c0427fea [15948.235551] Call Trace: [15948.235601] [c0497acf] ?
Re: [PATCH] btrfs: suppress a build warning on building 32bit kernel
Hi David, On 2014/12/30 0:09, David Sterba wrote: On Thu, Dec 25, 2014 at 06:21:41PM +0900, Satoru Takeuchi wrote: From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -2190,7 +2190,7 @@ void btrfs_free_io_failure_record(struct inode *inode, u64 start, u64 end) next = next_state(state); - failrec = (struct io_failure_record *)state-private; + failrec = (struct io_failure_record *)(unsigned long)state-private; We're always using the 'private' data to store a pointer to 'struct io_failure_record *', please change the defintion in 'struct extent_state' instead of the typecasting. Current definition is as follow. === struct extent_state { ... /* for use by the FS */ u64 private; }; === It it OK to changing u64 private to struct io_failure_record *failrec and change {set,get}_state_private() to {set,get}_state_failrec()? Or is it better to keep the name private as is and just change its type to unsigned long or (void *)? Thanks, Satoru free_extent_state(state); kfree(failrec); -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs: Fix a extent buffer leak in count_csum_range().
On 2015/01/05 16:56, Qu Wenruo wrote: The commit f495a2ac6611 (btrfs-progs: fsck: remove unfriendly BUG_ON() for searching tree failure) is causing tons of extent buffer leak if some csum mismatches in btrfsck. This is caused by a misplaced btrfs_release_path(), fix it. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com --- cmds-check.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-check.c b/cmds-check.c index d2d218a..5b644cf 100644 --- a/cmds-check.c +++ b/cmds-check.c @@ -1186,9 +1186,9 @@ static int count_csum_range(struct btrfs_root *root, u64 start, path.slots[0]++; } out: + btrfs_release_path(path); if (ret 0) return ret; - btrfs_release_path(path); return 0; } -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Debian/Jessie 3.16.7-ckt2-1 kernel error
Hi Petr, On 2014/12/28 0:36, Petr Janecek wrote: Hello Satoru and all, that Oct. report was the only time I've experienced the error, so I don't have much to add. I can try to answer your questions: Here are my questions. 1. Is your system btrfs scrub clean? yes, 2. Is this message shown every boot time? no, I have seen them only during one boot 3. Is this message shown only in boot? As in my Oct. email http://permalink.gmane.org/gmane.comp.file-systems.btrfs/39721 I've seen a similar one after creating a subvolume on a new fs. But it was during the same boot. 4. When this message is started to be shown? 5. Do you have any trouble, change your operation or configuration just before the answer of Q4 ? a disk was added to the fs and balance has been run. The balance crashed, as in https://bugzilla.kernel.org/show_bug.cgi?id=64961 (probably unrelated). After reboot, I've seen the messages. Additional questions. Q5. Could you give me your kernel configuration? At least, could you tell me whether your kernel enabled CONFIG_PREEMPT or not? CONFIG_PREEMPT_VOLUNTARY=y Q6. If the answer of Q1 is correct, please give me the file system image which can be captured by the following command. Sorry, the fs's are long gone. I continued to run similar workloads on that test box, but these errors never appeared again. Thank you for giving me information. So, further investigation of this problem seems to be hard. Please give us the above-mentioned information if this problem happens again. Thanks, Satoru Regards, Petr -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html