On Thu, Oct 27, 2016 at 09:13:44AM -0400, Josef Bacik wrote:
> On 10/26/2016 08:30 PM, Dave Chinner wrote:
> >On Wed, Oct 26, 2016 at 11:11:35AM -0400, Josef Bacik wrote:
> >>On 10/25/2016 06:01 PM, Dave Chinner wrote:
> >>>On Tue, Oct 25, 2016 at 02:41:44PM -0400, Josef Bacik wrote:
> With any
On 10/13/16 12:36 PM, Eric Sandeen wrote:
> This was reported when docker failed to build against
> btrfs-progs v4.8.1.
>
> It includes ioctl.h which now calls BUILD_ASSERT(), which
> is defined in kerncompat.h, which was not included in the
> ioctl.h header file.
Ping?
-Eric
> Signed-off-by:
From: Robbie Ko
We found a fsync deadlock in log_new_dir_dentries, because
btrfs_search_forward get path lock, then call btrfs_iget will
get another extent_buffer lock, maybe occur deadlock.
Fix this by release path before call btrfs_iget, avoid deadlock occur.
Example:
Pid waiting: 32021->
Introduce a new function, btrfs_read_one_data_csum(), to read just one
data csum for check usage.
Unlike original implement in cmds-check.c which checks csum by one
CSUM_EXTENT, this just read out one csum(4 bytes).
It is not fast but makes code easier to read.
And will be used in later fsck scru
Introduce a new function, scrub_one_full_stripe(), to check a full
stripe.
It handles the full stripe scrub in the following steps:
0) Check if we need to check full stripe
If full stripe contains no extent, why waste our CPU and IO?
1) Read out full stripe
Then we know how many devices are
Mainly copied from kernel code, with minor modification to follow the
new naming schema.
Signed-off-by: Qu Wenruo
---
raid56.c | 63 +++
raid56.h | 4
2 files changed, 67 insertions(+)
diff --git a/raid56.c b/raid56.c
index 8c79c
Introduce new function, scrub_one_data_stripe(), to check all data and
tree blocks inside the data stripe.
Signed-off-by: Qu Wenruo
---
check/scrub.c | 117 ++
1 file changed, 117 insertions(+)
diff --git a/check/scrub.c b/check/scrub.c
in
For READ, caller normally hopes to get what they request, other than
full stripe map.
In this case, we should remove unrelated stripe map, just like the
following case:
32K 96K
|<-request range->|
0 64k 128K
RAID0: |
Copied from kernel raid6 lib, with some naming modification.
Signed-off-by: Qu Wenruo
---
raid56.c | 36
raid56.h | 3 +++
2 files changed, 39 insertions(+)
diff --git a/raid56.c b/raid56.c
index 599533e..3e471d6 100644
--- a/raid56.c
+++ b/raid56.c
@@ -233
Introduce a new function, scrub_tree_mirror(), to scrub mirror based
tree blocks (Single/DUP/RAID0/1/10)
This function can be used on in-memory tree blocks using @data parameter
for RAID5/6 full stripe, or just by @bytenr, for other profiles.
Signed-off-by: Qu Wenruo
---
check/scrub.c | 59
Signed-off-by: Qu Wenruo
---
raid56.c | 76
raid56.h | 13 +++
2 files changed, 89 insertions(+)
diff --git a/raid56.c b/raid56.c
index 3e471d6..8d54e55 100644
--- a/raid56.c
+++ b/raid56.c
@@ -269,3 +269,79 @@ int raid6_re
Introuduce new local structures, scrub_full_stripe and scrub_stripe, for
incoming offline scrub support.
Signed-off-by: Qu Wenruo
---
Makefile.in | 2 +-
check/scrub.c | 110 ++
2 files changed, 111 insertions(+), 1 deletion(-)
create
For any one who wants to try it, it can be get from my repo:
https://github.com/adam900710/btrfs-progs/tree/fsck_scrub
Currently, I only tested it on SINGLE/DUP/RAID1/RAID5 filesystems, with
mirror or parity or data corrupted.
The tool are all able to detect them and give recoverbility report.
Se
Introduce new function, scrub_one_block_group(), to scrub a block group.
For Single/DUP/RAID0/RAID1/RAID10, we use old mirror number based
map_block, and check extent by extent.
For parity based profile (RAID5/6), we use new map_block_v2() and check
full stripe by full stripe.
Signed-off-by: Qu
From: Robbie Ko
The hole punching can result in adding new leafs (and as a consequence
new nodes) to the tree because when we find file extent items that span
beyond the hole range we may end up not deleting them (just adjusting them)
and add new file extent items representing holes.
That after
Introduce a new function, scrub_one_extent(), as a wrapper to check one
extent.
Signed-off-by: Qu Wenruo
---
check/scrub.c | 76 +++
1 file changed, 76 insertions(+)
diff --git a/check/scrub.c b/check/scrub.c
index b4bb081..ab78a0d 100644
Copied from kernel code with minor modification.
Signed-off-by: Qu Wenruo
---
.gitignore | 2 +
Makefile.in | 12 -
mktables.c | 142
raid56.h| 7 +++
4 files changed, 161 insertions(+), 2 deletions(-)
create mode 10064
Introduce function, recover_from_parities(), to recover data stripes.
However this function only support RAID5 yet, but should be good enough
for the scrub framework.
Signed-off-by: Qu Wenruo
---
check/scrub.c | 49 +
1 file changed, 49 insertions
Will be used for later scrub usage.
Signed-off-by: Qu Wenruo
---
ctree.h | 2 ++
extent-tree.c | 52
2 files changed, 54 insertions(+)
diff --git a/ctree.h b/ctree.h
index e0dd2e4..b511ff2 100644
--- a/ctree.h
+++ b/ctree.h
@@ -2363,6
Although there is only 2 function in raid56 header, later recovery will
introduce a lot of tables, so split it from disk-io.h.
Signed-off-by: Qu Wenruo
---
Makefile.in | 2 +-
disk-io.h | 4
raid56.h| 20
volumes.c | 1 +
4 files changed, 22 insertions(+), 5
Introduce a new function, scrub_data_mirror(), to check mirror based
data blocks.
Signed-off-by: Qu Wenruo
---
check/scrub.c | 69 +++
1 file changed, 69 insertions(+)
diff --git a/check/scrub.c b/check/scrub.c
index ef94193..b4bb081 10064
Introduce new function, verify_parities(), to check if parities matches
for full stripe which all data stripes matches with their csum.
Signed-off-by: Qu Wenruo
---
check/scrub.c | 66 +++
1 file changed, 66 insertions(+)
diff --git a/chec
Now, btrfs check has a kernel scrub equivalent.
And even more, it's has stronger csum check against reconstructed data
and existing data stripes.
It will avoid any possible silent data corruption in kernel scrub.
Now it only supports to do read-only check, but is already able to
provide info on t
Introduce a new function, __btrfs_map_block_v2().
Unlike old btrfs_map_block(), which needs different parameter to handle
different RAID profile, this new function uses unified btrfs_map_block
structure to handle all RAID profile in a more meaningful method:
Return physical address along with log
Am Thu, 27 Oct 2016 19:17:52 -0600
schrieb Bearcat Şándor :
> Folks,
>
> I have a btrfs raid 10 spread out among 4 ssds. The main drive is
> sda. When i reboot my system after the kernel loads i see "a start job
> is running for dev-sda1.device. I've let this sit for 3 hours with no
> progress
From: Robbie Ko
Test that an incremental send operation doesn't work for rmdir
because it uses the wrong name to delete.
This test exercises scenarios used to fail in btrfs and are fixed by
the following patch for the linux kernel:
"Btrfs: incremental send, skip check overwritten if parents' ge
From: Robbie Ko
Patch for test btrfs incremental send.
v2:
* Improve the change log
Robbie Ko (4):
xfstests: btrfs/131: add test for an incremental send with name
collision
xfstests: btrfs/132: add test for invaild update time by an
incremental send
xfstests: btrfs/133: add test f
From: Robbie Ko
Test that an incremental send operation dosen't work because
it tries to rename a directory which is already deleted.
This test exercises scenarios used to fail in btrfs and are fixed by
the following patch for the linux kernel:
"Btrfs: incremental send, add generation check for
From: Robbie Ko
Test that an incremental send operation doesn't work because
there's a name collision in the destination and it's not checked
corretly before the rename operation applies.
This test exercises scenarios used to fail in btrfs and are fixed by
the following patch for the linux kerne
From: Robbie Ko
Test that an incremental send operation dosen't' work because
it tries to update the time to a deleted directory after it finishes
a move operation.
The other one is that an operation is applied to a file using the old
name not the new name.
This test exercises scenarios used to
From: Robbie Ko
Patch for fix btrfs incremental send.
These patches base on v4.8.0-rc8
v2:
* Improve the change log
* Add a new patch "add generation check in existence demtermination for
the parent directory"
Robbie Ko (6):
Btrfs: incremental send, do not skip generation inconsistency chec
From: Robbie Ko
Example scenario:
Parent snapshot:
| dir258/ (ino 258, gen 15, dir)
|--- dir257/ (ino 257, gen 15, dir)
| dir259/ (ino 259, gen 15, dir)
Send snapshot:
| file258 (ino 258, gen 21, file)
| new_dir259/ (ino 259, gen 21, dir)
|--- dir257/ (ino 257, gen 15, di
From: Robbie Ko
Example scenario:
Parent snapshot:
| d1/ (ino 257, gen 44)
| d4/ (ino 258, gen 44)
| d3/ (ino 259, gen 44)
Send snapshot:
| d1/ (ino 258, gen 47)
| d4/ (ino 260, gen 47)
| d3/ (i
From: Robbie Ko
Example scenario:
Parent snapshot:
| d259_old/ (ino 259, gen 96)
| d1/ (ino 258, gen 96)
| f (ino 257, gen 96)
Send snapshot:
| d258/ (ino 258, gen 98)
| d259/ (ino 259, gen 98)
| d1/
From: Robbie Ko
Exampla scenario:
Parent snapshot:
| dir258/ (ino 258, gen 7, dir)
|--- dir257/ (ino 257, gen 7, dir)
| dir259/ (ino 259, gen 7, dir)
Send snapshot:
| file258 (ino 258, gen 10, file)
| new_dir259/ (ino 259, gen 10, dir)
|--
From: Robbie Ko
Example scenario:
Parent snapshot:
| dir258/ (ino 258, gen 27)
| dir257/ (ino 257, gen 27)
| dir259/ (ino 259, gen 27)
Send snapshot:
| new_dir259/ (ino 259, gen 38)
| new_dir258/ (ino 258, gen 38)
| new_dir257/ (ino 257, gen 38)
rmdir dir
From: Robbie Ko
Example scenario:
Parent snapshot:
|.(ino 256, gen 5)
| a1/ (ino 257, gen 5)
| a2/ (ino 258, gen 5)
Send snapshot:
|.(ino 256, gen 7)
| a2 (ino 257, gen 7)
rmdir a1
mkfile o257-7-0
rename o257-7-0 -> a2
ERROR:
Folks,
I have a btrfs raid 10 spread out among 4 ssds. The main drive is
sda. When i reboot my system after the kernel loads i see "a start job
is running for dev-sda1.device. I've let this sit for 3 hours with no
progress into the boot, and it seems to be a race condition of some
sort.
If i re
Any comment?
Especially the final patch will fix a long standing bug.
Thanks,
Qu
At 10/18/2016 09:31 AM, Qu Wenruo wrote:
The patchset does the following things:
1) Enhance comment for qgroup, rename 2 functions
Explain the how qgroup works, so new developers won't waste too much
time di
btrfs_remove_all_log_ctxs takes a shortcut where it avoids walking the
list because it knows all of the waiters are patiently waiting for the
commit to finish.
But, there's a small race where btrfs_sync_log can remove itself from
the list if it finds a log commit is already done. Also, it uses
li
While deleting pending extents, we insert extents to the
extent_tree. However, because of corruption, this might already
be freed. We trickle the EEXISTS error and let the caller
decide what needs to be done with it. In this case, ignore, so
the repair may proceed.
The primary motivation of this i
On Thu, Oct 27, 2016 at 10:32:48AM +0800, Qu Wenruo wrote:
> While most guys are using ctags and cscope with vim, new completion tool
> like vim-clang_completion is gaining its popularity, due to its compiler
> level accuracy simpleness to use.
>
> Since ctags and cscope are already in .gitignore,
On Mon, Oct 17, 2016 at 10:07:17AM +0530, Prasanth K S R wrote:
> From: Prasanth K S R
>
> This commit fixes coverity defect CID 1328695.
The patch does more than that. Please split it into two, one that
handles the memory allocations (the error message is not needed) and
anohter that switches t
On Thu, Oct 27, 2016 at 04:41:33PM +1100, Dave Chinner wrote:
> And that's indicative of a delalloc metadata reservation being
> being too small and so we're allocating unreserved blocks.
>
> Different symptoms, same underlying cause, I think.
>
> I see the latter assert from time to time
On Wed, Oct 26, 2016 at 05:52:11PM +0800, Wang Xiaoguang wrote:
> When enabling btrfs compression, original codes can not fill fs
> correctly, here we introduce _fill_fs() in common/rc, which'll keep
> creating and writing files until enospc error occurs. Note _fill_fs
> is copied from tests/generi
On 10/27/2016 10:34 AM, Linus Torvalds wrote:
On Wed, Oct 26, 2016 at 11:33 PM, Christoph Hellwig wrote:
Dave, can you hit the warnings with this? Totally untested...
Can we just kill off the unhelpful blk_map_ctx structure, e.g.:
Yeah, I found that hard to read too. The difference between
On Wed, Oct 26, 2016 at 11:33 PM, Christoph Hellwig wrote:
>> Dave, can you hit the warnings with this? Totally untested...
>
> Can we just kill off the unhelpful blk_map_ctx structure, e.g.:
Yeah, I found that hard to read too. The difference between
blk_map_ctx and blk_mq_alloc_data is minimal,
Hi,
Le 27/10/2016 à 02:50, Lionel Bouton a écrit :
> [...]
> I'll stop for tonight and see what happens during the day. I'd like to
> try a device add / delete next but I'm worried I could end up with a
> completely unusable filesystem if the device delete hits the same
> problem than replace.
> I
On 2016-10-24 19:05 +0200, David Sterba wrote:
>
> Ok that's better, just the signed-off-by line is missing. I would add it
> myself on behalf of contributors I know, but as a matter of practice,
> fix it please and resend the patch. Using the "git commit -s" adds the
> line for you.
>
I added s
csum member of struct btrfs_super_block has array type of u8. It makes sense
that function btrfs_csum_final should be also declared to accept u8 *. I
changed the declaration of method void btrfs_csum_final(u32 crc, char *result);
to void btrfs_csum_final(u32 crc, u8 *result);
Signed-off-by: Domago
On Wed, Oct 26, 2016 at 05:52:11PM +0800, Wang Xiaoguang wrote:
> When enabling btrfs compression, original codes can not fill fs
> correctly, here we introduce _fill_fs() in common/rc, which'll keep
> creating and writing files until enospc error occurs. Note _fill_fs
> is copied from tests/generi
On 10/26/2016 08:30 PM, Dave Chinner wrote:
On Wed, Oct 26, 2016 at 11:11:35AM -0400, Josef Bacik wrote:
On 10/25/2016 06:01 PM, Dave Chinner wrote:
On Tue, Oct 25, 2016 at 02:41:44PM -0400, Josef Bacik wrote:
With anything that populates the inode/dentry cache with a lot of one time use
inode
Hello,
Following the previous discussion:
https://www.spinics.net/lists/linux-btrfs/msg19075.html
I would be interested in finding a way to reliably identify reflink /
CoW files in order to use deduplication programs (like fdupes, jdupes,
rmlint) efficiently.
Using FIEMAP doesn't seem to be reli
On 10/26/2016 08:00 PM, Jens Axboe wrote:
> On 10/26/2016 05:47 PM, Dave Jones wrote:
>> On Wed, Oct 26, 2016 at 07:38:08PM -0400, Chris Mason wrote:
>>
>> > >-hctx->queued++;
>> > >-data->hctx = hctx;
>> > >-data->ctx = ctx;
>> > >+data->hctx = alloc_data.hctx;
>> > >+da
btrfs_map_block supports different types of mappings, which to a large
extent resemble block layer operations. But they don't always do, and
currently btrfs dangerously overlays it's own flag over the block layer
flags. This is just asking for a conflict, so introduce a different
map flags enum i
55 matches
Mail list logo
