Re: Send/receive snapshot from/between backup

2016-11-17 Thread Piotr Pawłow
On 16.11.2016 21:32, René Bühlmann wrote: > >> 1. Change received uuid of S1 on SSH to match S1 uuid on USB. >> [...] > 2. Full transfer S1 from SSH to SSH' > > 3. Incremental transfer S2 from USB to SSH' (S1 as parent) > > 4. Incremental transfer S3 from Origin to SSH' (S2 as parent) > > [..] > St

Re: [PATCH] fstests: Introduce check for explicit SHARED extent flag reporting

2016-11-17 Thread Eryu Guan
On Thu, Nov 17, 2016 at 10:06:48AM +0800, Qu Wenruo wrote: > For fs support reflink, some of them (OK, btrfs again) doesn't split > SHARED flag for extent fiemap reporting. > > For example: > 0 4K 8K >/ File1: Extent 0 \ > /\ > |<- On disk Extent-->|

Re: [PATCH 2/3] btrfs-progs: test: expand size of test device of fsck-tests 013

2016-11-17 Thread Qu Wenruo
On 11/18/2016 01:47 PM, Tsutomu Itoh wrote: In my test environment, size of /lib/modules/`uname -r`/* is larger than 1GB, so test data can not copy to testdev. Therefore we need expand the size of testdev. IMHO the correct fix is to enhance populate_fs() to fill the fs into a given size, oth

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Qu Wenruo
On 11/18/2016 12:43 PM, Zygo Blaxell wrote: On Fri, Nov 18, 2016 at 10:42:23AM +0800, Qu Wenruo wrote: At 11/18/2016 09:56 AM, Hugo Mills wrote: On Fri, Nov 18, 2016 at 09:19:11AM +0800, Qu Wenruo wrote: At 11/18/2016 07:13 AM, Zygo Blaxell wrote: On Tue, Nov 15, 2016 at 10:50:22AM +080

Re: [PATCH] fstests: Introduce check for explicit SHARED extent flag reporting

2016-11-17 Thread Qu Wenruo
On 11/18/2016 01:54 PM, Darrick J. Wong wrote: On Thu, Nov 17, 2016 at 10:06:48AM +0800, Qu Wenruo wrote: For fs support reflink, some of them (OK, btrfs again) doesn't split SHARED flag for extent fiemap reporting. For example: 0 4K 8K / File1: Extent 0 \ /

Re: [PATCH] fstests: Introduce check for explicit SHARED extent flag reporting

2016-11-17 Thread Darrick J. Wong
On Thu, Nov 17, 2016 at 10:06:48AM +0800, Qu Wenruo wrote: > For fs support reflink, some of them (OK, btrfs again) doesn't split > SHARED flag for extent fiemap reporting. > > For example: > 0 4K 8K >/ File1: Extent 0 \ > /\ > |<- On disk Extent-->|

[PATCH 3/3] btrfs-progs: test: fix convert-tests 004 failure

2016-11-17 Thread Tsutomu Itoh
convert-tests 004 failed because the argument to check_all_images is not specified. # make test-convert [TEST] convert-tests.sh [TEST/conv] 004-ext2-backup-superblock-ranges find: '': No such file or directory # Signed-off-by: Tsutomu Itoh --- tests/convert-tests/004-ext2-backup-sup

[PATCH 2/3] btrfs-progs: test: expand size of test device of fsck-tests 013

2016-11-17 Thread Tsutomu Itoh
In my test environment, size of /lib/modules/`uname -r`/* is larger than 1GB, so test data can not copy to testdev. Therefore we need expand the size of testdev. # make test-fsck [TEST] fsck-tests.sh [TEST/fsck] 013-extent-tree-rebuild failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs

[PATCH 1/3] btrfs-progs: test: fix error of test target of Makefile

2016-11-17 Thread Tsutomu Itoh
Add test-cli to test target of Makefile. Signed-off-by: Tsutomu Itoh --- Makefile.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.in b/Makefile.in index 4bd3e63..6c78841 100644 --- a/Makefile.in +++ b/Makefile.in @@ -298,7 +298,7 @@ test-inst: all

Re: [RFC PATCH 0/2] Btrfs: make a source length of 0 imply EOF for dedupe

2016-11-17 Thread Christoph Hellwig
On Thu, Nov 17, 2016 at 04:07:48PM -0800, Omar Sandoval wrote: > So now we have 3 options: > > a) Apply these patches as-is. > b) Fix XFS to both return the actual bytes_deduped and cap the length >for the EOF case. Do the same for Btrfs. > c) Make XFS consistent with the existing Btrfs behavi

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Zygo Blaxell
On Fri, Nov 18, 2016 at 10:42:23AM +0800, Qu Wenruo wrote: > > > At 11/18/2016 09:56 AM, Hugo Mills wrote: > >On Fri, Nov 18, 2016 at 09:19:11AM +0800, Qu Wenruo wrote: > >> > >> > >>At 11/18/2016 07:13 AM, Zygo Blaxell wrote: > >>>On Tue, Nov 15, 2016 at 10:50:22AM +0800, Qu Wenruo wrote: >

Re: [RFC PATCH 1/2] Btrfs: refactor btrfs_extent_same() slightly

2016-11-17 Thread Qu Wenruo
At 11/18/2016 08:07 AM, Omar Sandoval wrote: From: Omar Sandoval This just makes the following patch cleaner. Looks good for me. It is just a good refactoring. Making code short and easier to read. And it doesn't affect the len = 0 behavior. So no matter what the choice we do, it can be ap

Re: [PATCH] generic: test concurrent non-overlapping direct I/O on the same extents

2016-11-17 Thread Eryu Guan
On Thu, Nov 17, 2016 at 10:14:12AM -0800, Omar Sandoval wrote: > On Thu, Nov 17, 2016 at 01:26:11PM +0800, Eryu Guan wrote: > > On Wed, Nov 16, 2016 at 04:29:34PM -0800, Omar Sandoval wrote: > > > From: Omar Sandoval > > > > > > There have been a couple of logic bugs in `btrfs_get_extent()` which

From shenzhen China Freight agency company

2016-11-17 Thread Alfred
Dear Sirs: How are you! If your company have import some product from China.May I help you on the logistics field. We specialized in international express, air transport, shipping. with good price and good service, with many years experience ! We are pursuing is the integrity and

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Qu Wenruo
At 11/18/2016 09:56 AM, Hugo Mills wrote: On Fri, Nov 18, 2016 at 09:19:11AM +0800, Qu Wenruo wrote: At 11/18/2016 07:13 AM, Zygo Blaxell wrote: On Tue, Nov 15, 2016 at 10:50:22AM +0800, Qu Wenruo wrote: Fix the so-called famous RAID5/6 scrub error. Thanks Goffredo Baroncelli for reportin

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-17 Thread Qu Wenruo
At 11/18/2016 03:27 AM, Austin S. Hemmelgarn wrote: On 2016-11-17 13:51, Hans van Kranenburg wrote: Hey, On 11/17/2016 02:27 AM, Qu Wenruo wrote: At 11/17/2016 04:30 AM, Hans van Kranenburg wrote: In the last two days I've added the --blockgroup option to btrfs heatmap to let it create pic

Re: root backup-reformat-restore

2016-11-17 Thread Marcus Sundman
On 18.11.2016 02:52, Hugo Mills wrote: On Fri, Nov 18, 2016 at 02:38:25AM +0200, Marcus Sundman wrote: The FAQ says that "the best solution for small devices (under about 16 GB) is to reformat the FS with the --mixed option to mkfs.btrfs". OK. Does anyone have any good suggestions for doing tha

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Hugo Mills
On Fri, Nov 18, 2016 at 09:19:11AM +0800, Qu Wenruo wrote: > > > At 11/18/2016 07:13 AM, Zygo Blaxell wrote: > >On Tue, Nov 15, 2016 at 10:50:22AM +0800, Qu Wenruo wrote: > >>Fix the so-called famous RAID5/6 scrub error. > >> > >>Thanks Goffredo Baroncelli for reporting the bug, and make it into

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Qu Wenruo
At 11/18/2016 07:13 AM, Zygo Blaxell wrote: On Tue, Nov 15, 2016 at 10:50:22AM +0800, Qu Wenruo wrote: Fix the so-called famous RAID5/6 scrub error. Thanks Goffredo Baroncelli for reporting the bug, and make it into our sight. (Yes, without the Phoronix report on this, https://www.phoronix.co

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Qu Wenruo
At 11/18/2016 02:09 AM, Goffredo Baroncelli wrote: Hi Qu, On 2016-11-15 03:50, Qu Wenruo wrote: Fix the so-called famous RAID5/6 scrub error. Thanks Goffredo Baroncelli for reporting the bug, and make it into our sight. (Yes, without the Phoronix report on this, https://www.phoronix.com/scan

Re: root backup-reformat-restore

2016-11-17 Thread Hugo Mills
On Fri, Nov 18, 2016 at 02:38:25AM +0200, Marcus Sundman wrote: > The FAQ says that "the best solution for small devices (under about > 16 GB) is to reformat the FS with the --mixed option to mkfs.btrfs". > > OK. Does anyone have any good suggestions for doing that with an > existing / partition (

root backup-reformat-restore

2016-11-17 Thread Marcus Sundman
The FAQ says that "the best solution for small devices (under about 16 GB) is to reformat the FS with the --mixed option to mkfs.btrfs". OK. Does anyone have any good suggestions for doing that with an existing / partition (which has special files and whatnot)? I assume a backup-restore cycle

[RFC PATCH 1/2] Btrfs: refactor btrfs_extent_same() slightly

2016-11-17 Thread Omar Sandoval
From: Omar Sandoval This just makes the following patch cleaner. Signed-off-by: Omar Sandoval --- fs/btrfs/ioctl.c | 33 - 1 file changed, 12 insertions(+), 21 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 7acbd2c..e23f945 100644 --- a/fs/

[RFC PATCH 0/2] Btrfs: make a source length of 0 imply EOF for dedupe

2016-11-17 Thread Omar Sandoval
From: Omar Sandoval This is the follow-up to the discussions here [1] and here [2] that makes Btrfs treat a src_length of 0 to dedupe as "until EOF". The implementation is straightforward, but there are a few catches that convinced me to post this as an RFC: 1. We still don't know for sure wheth

[RFC PATCH 2/2] Btrfs: make a source length of 0 imply EOF for dedupe

2016-11-17 Thread Omar Sandoval
From: Omar Sandoval This makes us consistent with both reflink and the XFS implementation of dedupe. Signed-off-by: Omar Sandoval --- fs/btrfs/ioctl.c | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index e23f945..320fb35 1

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Zygo Blaxell
On Tue, Nov 15, 2016 at 10:50:22AM +0800, Qu Wenruo wrote: > Fix the so-called famous RAID5/6 scrub error. > > Thanks Goffredo Baroncelli for reporting the bug, and make it into our > sight. > (Yes, without the Phoronix report on this, > https://www.phoronix.com/scan.php?page=news_item&px=Btrfs-RA

btrfs raid1 degraded needs chunk root rebuild

2016-11-17 Thread Vladi Gergov
Any ideas if currently I can still recover data from this volume? I have tried the new btrfs tools with btrfs rescue chunk-recover without much success. Any ideas if i can recover any of the data? - Forwarded message from Chris Mason - From: Chris Mason Subject: Re: btrfs raid1 degraded

Re: [Bug 186671] New: OOM on system with just rsync running 32GB of ram 30GB of pagecache

2016-11-17 Thread Vlastimil Babka
On 11/16/2016 02:39 PM, E V wrote: > System panic'd overnight running 4.9rc5 & rsync. Attached a photo of > the stack trace, and the 38 call traces in a 2 minute window shortly > before, to the bugzilla case for those not on it's e-mail list: > > https://bugzilla.kernel.org/show_bug.cgi?id=186671

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-17 Thread Hans van Kranenburg
On 11/17/2016 08:27 PM, Austin S. Hemmelgarn wrote: > On 2016-11-17 13:51, Hans van Kranenburg wrote: >> >> When generating a picture of a file system with multiple devices, >> boundaries between the separate devices are not visible now. >> >> If someone has a brilliant idea about how to do this wi

Re: degraded BTRFS RAID 1 not mountable: open_ctree failed, unable to find block group for 0

2016-11-17 Thread Martin Steigerwald
Am Donnerstag, 17. November 2016, 12:05:31 CET schrieb Chris Murphy: > I think the wiki should be updated to reflect that raid1 and raid10 > are mostly OK. I think it's grossly misleading to consider either as > green/OK when a single degraded read write mount creates single chunks > that will then

Re: degraded BTRFS RAID 1 not mountable: open_ctree failed, unable to find block group for 0

2016-11-17 Thread Austin S. Hemmelgarn
On 2016-11-17 15:05, Chris Murphy wrote: I think the wiki should be updated to reflect that raid1 and raid10 are mostly OK. I think it's grossly misleading to consider either as green/OK when a single degraded read write mount creates single chunks that will then prevent a subsequent degraded rea

Re: degraded BTRFS RAID 1 not mountable: open_ctree failed, unable to find block group for 0

2016-11-17 Thread Chris Murphy
I think the wiki should be updated to reflect that raid1 and raid10 are mostly OK. I think it's grossly misleading to consider either as green/OK when a single degraded read write mount creates single chunks that will then prevent a subsequent degraded read write mount. And also the lack of various

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-17 Thread Austin S. Hemmelgarn
On 2016-11-17 13:51, Hans van Kranenburg wrote: Hey, On 11/17/2016 02:27 AM, Qu Wenruo wrote: At 11/17/2016 04:30 AM, Hans van Kranenburg wrote: In the last two days I've added the --blockgroup option to btrfs heatmap to let it create pictures of block group internals. Examples and more inst

Re: [PATCH] btrfs-progs: send: fix failure of xfstests btrfs/038

2016-11-17 Thread Omar Sandoval
On Tue, Nov 15, 2016 at 05:16:27PM +0900, Tsutomu Itoh wrote: > The following patch was imperfect, so xfstests btrfs/038 was failed. > > 6d4fb3d btrfs-progs: send: fix handling of multiple snapshots (-p option) > > [before] > | # ./check btrfs/038 > | FSTYP -- btrfs > | PLATFORM -

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-17 Thread Hans van Kranenburg
Hey, On 11/17/2016 02:27 AM, Qu Wenruo wrote: > > At 11/17/2016 04:30 AM, Hans van Kranenburg wrote: >> In the last two days I've added the --blockgroup option to btrfs heatmap >> to let it create pictures of block group internals. >> >> Examples and more instructions are to be found in the READM

Re: [PATCH 0/2] RAID5/6 scrub race fix

2016-11-17 Thread Goffredo Baroncelli
Hi Qu, On 2016-11-15 03:50, Qu Wenruo wrote: > Fix the so-called famous RAID5/6 scrub error. > > Thanks Goffredo Baroncelli for reporting the bug, and make it into our > sight. > (Yes, without the Phoronix report on this, > https://www.phoronix.com/scan.php?page=news_item&px=Btrfs-RAID-56-Is-Bad,

Re: [PATCH] generic: test concurrent non-overlapping direct I/O on the same extents

2016-11-17 Thread Omar Sandoval
On Thu, Nov 17, 2016 at 01:26:11PM +0800, Eryu Guan wrote: > On Wed, Nov 16, 2016 at 04:29:34PM -0800, Omar Sandoval wrote: > > From: Omar Sandoval > > > > There have been a couple of logic bugs in `btrfs_get_extent()` which > > could lead to spurious -EEXIST errors from read or write. This test

Re: btrfs btree_ctree_super fault

2016-11-17 Thread Chris Mason
On 11/17/2016 12:39 AM, Chris Cui wrote: We have just encountered the same bug on 4.9.0-rc2. Any solution now? kernel BUG at fs/btrfs/ctree.c:3172! invalid opcode: [#1] PREEMPT SMP DEBUG_PAGEALLOC CPU: 0 PID: 22702 Comm: trinity-c40 Not tainted 4.9.0-rc4-think+ #1 task: 8804ffde37c0 t