[PATCH v3 3/5] btrfs: tree-checker: Enhance output for btrfs_check_leaf

2017-09-28 Thread Qu Wenruo
Enhance the output to print: 1) Reason 2) Bad value If reason can't explain enough 3) Good value (range) Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 27 +-- 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tre

[PATCH v3 5/5] btrfs: tree-checker: Enhance output for check_extent_data_item

2017-09-28 Thread Qu Wenruo
Output the invalid member name and its bad value, along with its expected value range or alignment. Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 98 +++-- 1 file changed, 70 insertions(+), 28 deletions(-) diff --git a/fs/btrfs/tree-checker.c

[PATCH v3 0/5] Enhance tree block validation checker

2017-09-28 Thread Qu Wenruo
The patchset can be fetched from github: https://github.com/adam900710/linux/tree/checker_enhance It's based on David's misc-next branch, with following commit as base: a5e50b4b444c ("btrfs: Add checker for EXTENT_CSUM") According to David's suggestion, enhance the output format of tree block val

[PATCH v3 4/5] btrfs: tree-checker: Enhance output for check_csum_item

2017-09-28 Thread Qu Wenruo
Output the bad value and expected good value (or its alignment). Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c index 183ff7faa218..c0fd192f8140 100644 --- a/f

[PATCH v3 1/5] btrfs: Move leaf and node validation checker to tree-checker.c

2017-09-28 Thread Qu Wenruo
It's no doubt the comprehensive tree block checker will become larger and larger, so move them into their own file is quite reasonable. Signed-off-by: Qu Wenruo --- fs/btrfs/Makefile | 2 +- fs/btrfs/ctree.h| 4 + fs/btrfs/disk-io.c | 284 +-

[PATCH v3 2/5] btrfs: tree-checker: Enhance btrfs_check_node output

2017-09-28 Thread Qu Wenruo
Use inline function to replace macro since we don't need stringification. (Macro still exist until all caller get updated) And add more info about the error, and replace EIO with EUCLEAN. For nr_items error, report if it's too large or too small, and output valid value range. For blk pointer, ad

WARNING: CPU: 1 PID: 13825 at fs/btrfs/backref.c:1255 find_parent_nodes+0xb5c/0x1310

2017-09-28 Thread Paul Jones
Hi, Just ran into this warning while running deduplication. There were 10's of thousands of them over a 24hr period. No other problems were reported. Filesystem is raid1, freshly converted from single. Zstd compression.  4.14.0-rc2 kernel Sep 28 14:57:06 home kernel: [ cut here ]--

Re: [PATCH v2 2/5] btrfs: tree-checker: Enhance btrfs_check_node output

2017-09-28 Thread Qu Wenruo
On 2017年09月29日 14:05, Nikolay Borisov wrote: On 29.09.2017 04:36, Qu Wenruo wrote: Use inline function to replace macro since we don't need stringification. (Macro still exist until all caller get updated) And add more info about the error. For nr_items error, report if it's too large or t

Re: [PATCH v2 2/5] btrfs: tree-checker: Enhance btrfs_check_node output

2017-09-28 Thread Nikolay Borisov
On 29.09.2017 04:36, Qu Wenruo wrote: > Use inline function to replace macro since we don't need > stringification. > (Macro still exist until all caller get updated) > > And add more info about the error. > > For nr_items error, report if it's too large or too small, and output > valid value r

Re: Mount error on 32 bits, ok on 64 bits

2017-09-28 Thread Jean-Denis Girard
Le 28/09/2017 à 15:29, Jean-Denis Girard a écrit : > Hi list, > > I have an Alix motherboard with a SD card using btrfs running fine since > 2010. Today, I wanted to upgrade to kernel 4.13.4 from 4.9.52 (i586). As > always, I cross-compiled from my main system, installed on Alix, but > boot failed

[PATCH v2 0/5] Enhance tree block validation checker

2017-09-28 Thread Qu Wenruo
The patchset can be fetched from github: https://github.com/adam900710/linux/tree/checker_enhance It's based on David's misc-next branch, with following commit as base: a5e50b4b444c ("btrfs: Add checker for EXTENT_CSUM") According to David's suggestion, enhance the output format of tree block val

[PATCH v2 4/5] btrfs: tree-checker: Enhance output for check_csum_item

2017-09-28 Thread Qu Wenruo
Output the bad value and expected good value (or its alignment). Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c index 94027f4215e9..a5b743763362 100644 --- a/f

[PATCH v2 5/5] btrfs: tree-checker: Enhance output for check_extent_data_item

2017-09-28 Thread Qu Wenruo
Output the invalid member name and its bad value, along with its expected value range or alignment. Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 98 +++-- 1 file changed, 70 insertions(+), 28 deletions(-) diff --git a/fs/btrfs/tree-checker.c

[PATCH v2 3/5] btrfs: tree-checker: Enhance output for btrfs_check_leaf

2017-09-28 Thread Qu Wenruo
Enhance the output to print: 1) Reason 2) Bad value If reason can't explain enough 3) Good value (range) Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 27 +-- 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tre

[PATCH v2 1/5] btrfs-progs: Move leaf and node validation checker to tree-checker.c

2017-09-28 Thread Qu Wenruo
It's no doubt the comprehensive tree block checker will become larger and larger, so move them into their own file is quite reasonable. Signed-off-by: Qu Wenruo --- fs/btrfs/Makefile | 2 +- fs/btrfs/ctree.h| 4 + fs/btrfs/disk-io.c | 284 +-

[PATCH v2 2/5] btrfs: tree-checker: Enhance btrfs_check_node output

2017-09-28 Thread Qu Wenruo
Use inline function to replace macro since we don't need stringification. (Macro still exist until all caller get updated) And add more info about the error. For nr_items error, report if it's too large or too small, and output valid value range. For blk pointer, added a new alignment checker.

Mount error on 32 bits, ok on 64 bits

2017-09-28 Thread Jean-Denis Girard
Hi list, I have an Alix motherboard with a SD card using btrfs running fine since 2010. Today, I wanted to upgrade to kernel 4.13.4 from 4.9.52 (i586). As always, I cross-compiled from my main system, installed on Alix, but boot failed while trying to mount root: "BTRFS critical (device sda1): una

[PATCH] btrfs-progs: fix invalid assert in backref.c

2017-09-28 Thread Josef Bacik
This should be verify'ing that we have an empty key, not that we have a filled out key. Signed-off-by: Josef Bacik --- Dave this is on top of your ext/jeffm/extent-cache branch and fixes the segfault you reported. backref.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bac

Re : [Xen-devel] Re : task btrfs-transacti:651 blocked for more than 120 seconds

2017-09-28 Thread Olivier Bonvalet
Le jeudi 28 septembre 2017 à 16:28 +0200, Olivier Bonvalet a écrit : > [ 3263.452023] INFO: task systemd:1 blocked for more than 120 > seconds. > [ 3263.452040] Tainted: GW 4.9-dae-xen #2 > [ 3263.452044] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message

[PATCH] btrfs: Use DIV_ROUND_UP rathen than opencoding it

2017-09-28 Thread Nikolay Borisov
Signed-off-by: Nikolay Borisov --- fs/btrfs/extent-tree.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index e2d7e86b51d1..9e67616892cd 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2896,9 +2896,8 @

[PATCH v8 5/6] Btrfs: heuristic add byte set calculation

2017-09-28 Thread Timofey Titovets
Calculate byte set size for data sample: Calculate how many unique bytes has been in sample By count all bytes in bucket with count > 0 If byte set low (~25%), data are easily compressible Otherwise need additional analize Signed-off-by: Timofey Titovets --- fs/btrfs/compression.c | 48 +

[PATCH v8 1/6] Btrfs: compression.c separated heuristic/compression workspaces

2017-09-28 Thread Timofey Titovets
Compression heuristic itself is not a compression type, as current infrastructure supposed to provide workspaces for several compression types, it's difficult to just add heuristic workspace. Just refactor the code to support compression/heuristic workspaces with maximum code sharing and minimum c

[PATCH v8 3/6] Btrfs: implement heuristic sampling logic

2017-09-28 Thread Timofey Titovets
Copy sample data from input data range to sample buffer then calculate byte type count for that sample into bucket. Signed-off-by: Timofey Titovets --- fs/btrfs/compression.c | 71 +++--- 1 file changed, 61 insertions(+), 10 deletions(-) diff --git a/

[PATCH v8 4/6] Btrfs: heuristic add detection of repeated data patterns

2017-09-28 Thread Timofey Titovets
Walk over data sample and use memcmp to detect repeated data (like zeroed) Signed-off-by: Timofey Titovets --- fs/btrfs/compression.c | 17 - 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index e2419639ae7f..1cb4df02

[PATCH v8 2/6] Btrfs: heuristic workspace add bucket and sample items, macros

2017-09-28 Thread Timofey Titovets
Added macros: - For future sampling algo - For bucket size Heuristic workspace: - Add bucket for storing byte type counters - Add sample array for storing partial copy of input data range - Add counter for store current sample size to workspace Signed-off-by: Timofey Titovets --- fs/btr

[PATCH v8 0/6] Btrfs: populate heuristic with code

2017-09-28 Thread Timofey Titovets
Based on linux master 4.14-rc2 Duplicated to github: https://github.com/Nefelim4ag/linux/tree/heuristic_v8 Compile tested, hand tested on live system Patches short: 1. Implement workspaces for heuristic Separate heuristic/compression workspaces Main target for that patch: Maximum code sh

[PATCH v8 6/6] Btrfs: heuristic add byte core set calculation

2017-09-28 Thread Timofey Titovets
Calculate byte core set for data sample: Sort bucket's numbers in decreasing order Count how many numbers use 90% of sample If core set are low (<=25%), data are easily compressible If core set high (>=80%), data are not compressible Signed-off-by: Timofey Titovets --- fs/btrfs/compression.c | 6

Re : task btrfs-transacti:651 blocked for more than 120 seconds

2017-09-28 Thread Olivier Bonvalet
Le jeudi 28 septembre 2017 à 14:18 +0300, Nikolay Borisov a écrit : > So what this stack trace means is that transaction commit has hung. > And > judging by the called functions (assuming they are correct, though > the ? > aren't very encouraging). Concretely, it means that an io has been > started

Re: [PATCH 2/2] Remove misleading BCP 78 boilerplate

2017-09-28 Thread Nicholas D Steeves
Hi David, On 18 September 2017 at 10:40, David Sterba wrote: > On Sun, Sep 17, 2017 at 07:52:27PM -0400, Nicholas D Steeves wrote: >> BCP 78 applies to RFC 6234, but sha224-256.c is Simplified BSD. >> >> This causes the following lintian error when building on Debian and >> Debian derivatives: >>

Re: task btrfs-transacti:651 blocked for more than 120 seconds

2017-09-28 Thread Nikolay Borisov
On 28.09.2017 13:16, Olivier Bonvalet wrote: > Hi ! > > I have a virtual server (Xen) which very frequently hangs with only > this error in logs : > > [ 1330.144124] INFO: task btrfs-transacti:651 blocked for more than 120 > seconds. > [ 1330.144141] Not tainted 4.9-dae-xen #2 > [ 1330.1

task btrfs-transacti:651 blocked for more than 120 seconds

2017-09-28 Thread Olivier Bonvalet
Hi ! I have a virtual server (Xen) which very frequently hangs with only this error in logs : [ 1330.144124] INFO: task btrfs-transacti:651 blocked for more than 120 seconds. [ 1330.144141] Not tainted 4.9-dae-xen #2 [ 1330.144146] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables

[PATCH v2 2/2] btrfs: Fix transaction abort during failure in btrfs_rm_dev_item

2017-09-28 Thread Nikolay Borisov
btrfs_rm_dev_item calls several function under an activa transaction, however it fails to abort it if an error happens. Fix this by adding explicit btrfs_abort_transaction/btrfs_end_transaction calls Signed-off-by: Nikolay Borisov --- v2: * Explicitly handle every failure case w.r.t transactio

[PATCH v2 1/2] btrfs: Refactor transaction handling

2017-09-28 Thread Nikolay Borisov
If btrfs_transaction_commit fails it will proceed to call cleanup_transaction, which in turn already does btrfs_abort_transaction. So let's remove the unnecessary code duplication. Also let's be explicit about handling failure of btrfs_uuid_tree_add by calling btrfs_end_transaction. Signed-off-by:

Re: [PATCH 5/5] btrfs: tree-checker: Enhance output for check_extent_data_item

2017-09-28 Thread Qu Wenruo
On 2017年09月28日 14:09, Nikolay Borisov wrote: On 28.09.2017 06:36, Qu Wenruo wrote: Output the invalid member name and its bad value, along with its expected value range or alignment. Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 92 +--

[PATCH 1/2] btrfs: Explicitly handle btrfs_update_root failure

2017-09-28 Thread Nikolay Borisov
btrfs_udpate_root can fail and it aborts the transaction, the correct way to handle an aborted transaction is to explicitly end with btrfs_end_transaction. Even now the code is correct since btrfs_commit_transaction would handle an aborted transaction but this is more of an implementation detail. S

[PATCH v3 2/2] btrfs: Remove received_uuid during received snapshot ro->rw switch

2017-09-28 Thread Nikolay Borisov
Currently when a read-only snapshot is received and subsequently its ro property is set to false i.e. switched to rw-mode the received_uuid of that subvol remains intact. However, once the received volume is switched to RW mode we cannot guaranteee that it contains the same data, so it makes sense

[PATCH] btrfs-progs: tests: arg override in command line

2017-09-28 Thread Su Yue
Lowmem mode only repairs few cases which has a beacon file ".lowmem_repairable" in the case' directory. However, defining TEST_ENABLE_OVERRIDE=true in command line does work in above strategy. Because _skip_spec() in tests/common.local isn't interpreted by shell in that case. Solve it by making _

Re: [PATCH] btrfs: take the error path out if btrfs_attach_transaction() fails

2017-09-28 Thread Anand Jain
On 09/27/2017 10:17 PM, David Sterba wrote: On Wed, Sep 27, 2017 at 05:50:52PM +0800, Anand Jain wrote: btrfs_init_new_device() calls btrfs_attach_transaction() to commit sys chunks, however take the error path out if it fails. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 3 ++- 1 f