From: Qu Wenruo
Add disable subcommand for dedupe command group.
Signed-off-by: Qu Wenruo
Signed-off-by: Lu Fengqi
---
Documentation/btrfs-dedupe-inband.asciidoc | 5 +++
btrfs-completion | 2 +-
cmds-dedupe-ib.c | 41
From: Qu Wenruo
Add basic ioctl header and command group framework for later use.
Alone with basic man page doc.
Signed-off-by: Qu Wenruo
Signed-off-by: Lu Fengqi
---
Documentation/Makefile.in | 1 +
Documentation/btrfs-dedupe-inband.asciidoc | 40 ++
From: Qu Wenruo
Introduce reconfigure subcommand to co-operate with new kernel ioctl
modification.
Signed-off-by: Qu Wenruo
Signed-off-by: Lu Fengqi
---
Documentation/btrfs-dedupe-inband.asciidoc | 7 +++
btrfs-completion | 2 +-
cmds-dedupe-ib.c
From: Qu Wenruo
Add enable subcommand for dedupe commmand group.
Signed-off-by: Qu Wenruo
Signed-off-by: Lu Fengqi
---
Documentation/btrfs-dedupe-inband.asciidoc | 114 +-
btrfs-completion | 6 +-
cmds-dedupe-ib.c | 238
Patchset can be fetched from github:
https://github.com/littleroad/btrfs-progs.git dedupe_latest
Inband dedupe(in-memory backend only) ioctl support for btrfs-progs.
v7 changes:
Update ctree.h to follow kernel structure change
Update print-tree to follow kernel structure change
V8 changes:
From: Qu Wenruo
Add status subcommand for dedupe command group.
Signed-off-by: Qu Wenruo
Signed-off-by: Lu Fengqi
---
Documentation/btrfs-dedupe-inband.asciidoc | 3 +
btrfs-completion | 2 +-
cmds-dedupe-ib.c | 80 ++
From: Wang Xiaoguang
Introduce type based metadata reserve parameter for delalloc space
reservation/freeing function.
The problem we are going to solve is, btrfs use different max extent
size for different mount options.
For de-duplication, the max extent size can be set by the dedupe ioctl,
From: Qu Wenruo
Core implement for inband de-duplication.
It reuses the async_cow_start() facility to do the calculate dedupe hash.
And use dedupe hash to do inband de-duplication at extent level.
The workflow is as below:
1) Run delalloc range for an inode
2) Calculate hash for the delalloc
From: Wang Xiaoguang
Add generic function to initialize dedupe info.
Signed-off-by: Qu Wenruo
Signed-off-by: Wang Xiaoguang
Reviewed-by: Josef Bacik
Signed-off-by: Lu Fengqi
---
fs/btrfs/Makefile | 2 +-
fs/btrfs/dedupe.c | 169 +
From: Wang Xiaoguang
Add ioctl interface for inband deduplication, which includes:
1) enable
2) disable
3) status
And a pseudo RO compat flag, to imply that btrfs now supports inband
dedup.
However we don't add any ondisk format change, it's just a pseudo RO
compat flag.
All these ioctl
From: Wang Xiaoguang
Introduce static function inmem_search() to handle the job for in-memory
hash tree.
The trick is, we must ensure the delayed ref head is not being run at
the time we search the for the hash.
With inmem_search(), we can implement the btrfs_dedupe_search()
interface.
From: Wang Xiaoguang
Introduce the header for btrfs in-band(write time) de-duplication
framework and needed header.
The new de-duplication framework is going to support 2 different dedupe
methods and 1 dedupe hash.
Signed-off-by: Qu Wenruo
Signed-off-by: Wang Xiaoguang
Signed-off-by: Lu
From: Wang Xiaoguang
Introduce static function inmem_del() to remove hash from in-memory
dedupe tree.
And implement btrfs_dedupe_del() and btrfs_dedup_disable() interfaces.
Also for btrfs_dedupe_disable(), add new functions to wait existing
writer and block incoming writers to eliminate all
From: Qu Wenruo
Introduce new reconfigure ioctl and new FORCE flag for in-band dedupe
ioctls.
Now dedupe enable and reconfigure ioctl are stateful.
| Current state | Ioctl| Next state |
| Disabled
From: Qu Wenruo
Since the introduction of btrfs dedupe tree, it's possible that balance can
race with dedupe disabling.
When this happens, dedupe_enabled will make btrfs_get_fs_root() return
PTR_ERR(-ENOENT).
But due to a bug in error handling branch, when this happens
backref_cache->nr_nodes
This patchset can be fetched from github:
https://github.com/littleroad/linux.git dedupe_latest
Now the new base is v4.20-rc1.
Normal test cases from auto group exposes no regression, and ib-dedupe
group can pass without problem.
xfstests ib-dedupe group can be fetched from github:
From: Wang Xiaoguang
Introduce static function inmem_add() to add hash into in-memory tree.
And now we can implement the btrfs_dedupe_add() interface.
Signed-off-by: Qu Wenruo
Signed-off-by: Wang Xiaoguang
Reviewed-by: Josef Bacik
Signed-off-by: Lu Fengqi
---
fs/btrfs/dedupe.c | 150
From: Qu Wenruo
For in-band dedupe, btrfs needs to increase data ref with delayed_ref
locked, so add a new function btrfs_add_delayed_data_ref_lock() to
increase extent ref with delayed_refs already locked. Export
init_delayed_ref_head and init_delayed_ref_common for inband dedupe.
From: Wang Xiaoguang
Add ordered-extent support for dedupe.
Note, current ordered-extent support only supports non-compressed source
extent.
Support for compressed source extent will be added later.
Signed-off-by: Qu Wenruo
Signed-off-by: Wang Xiaoguang
Reviewed-by: Josef Bacik
---
From: Wang Xiaoguang
Unlike in-memory or on-disk dedupe method, only SHA256 hash method is
supported yet, so implement btrfs_dedupe_calc_hash() interface using
SHA256.
Signed-off-by: Qu Wenruo
Signed-off-by: Wang Xiaoguang
Reviewed-by: Josef Bacik
Signed-off-by: Lu Fengqi
---
On 5.11.18 г. 2:23 ч., Qu Wenruo wrote:
> Great, then it's completely free space cache causing the problem.
>
> You could use -o nospace_cache mount option to avoid the problem as a
> workaround.
>
> Free space cache only speed up free extent search, it doesn't has extra
> effect on the fs
On 5.11.18 г. 23:42 ч., Rosen Penev wrote:
> On Mon, Nov 5, 2018 at 1:31 PM Nikolay Borisov wrote:
>>
>>
>>
>> On 5.11.18 г. 21:06 ч., Rosen Penev wrote:
>>> Replaced bswap with _ variants. While it's a glibc extension, all of the
>>> popular libc implementations (glibc, uClibc, musl, BIONIC)
On 2018/11/6 上午2:01, Attila Vangel wrote:
> Hi,
>
> TL;DR: I want to save data from my unmountable btrfs partition.
> I saw some commands in another thread "Salvage files from broken btrfs".
> I use the most recent Manjaro live (kernel: 4.19.0-3-MANJARO,
> btrfs-progs 4.17.1-1) to execute these
Hi Qu,
On 2018/11/5 22:42, Qu Wenruo wrote:
>
>
> On 2018/11/5 下午7:33, Zhangshaokun wrote:
>> Hi Qu,
>>
>> On 2018/11/5 19:03, Qu Wenruo wrote:
>>>
>>>
>>> On 2018/11/5 下午6:49, Shaokun Zhang wrote:
block_group_err shows the group system as a decimal value with a '0x'
prefix, which is
On Mon, Nov 5, 2018 at 1:31 PM Nikolay Borisov wrote:
>
>
>
> On 5.11.18 г. 21:06 ч., Rosen Penev wrote:
> > Replaced bswap with _ variants. While it's a glibc extension, all of the
> > popular libc implementations (glibc, uClibc, musl, BIONIC) seem to support
> > it.
> >
> > Added static inline
On 5.11.18 г. 21:06 ч., Rosen Penev wrote:
> pthread_t is a pointer type, not an integer one. The > 0 makes no sense
> and throws a warning.
Code-wise the patch is ok, however, technically pthread_t is an opaque
type. I guess David could fix it on the way in so no need to resend.
Reviewed-by:
On 5.11.18 г. 21:06 ч., Rosen Penev wrote:
> Replaced bswap with _ variants. While it's a glibc extension, all of the
> popular libc implementations (glibc, uClibc, musl, BIONIC) seem to support
> it.
>
> Added static inline to two functions to match little endian variants. This
> fixes a
On 5.11.18 г. 21:06 ч., Rosen Penev wrote:
> Found using -Wmissing-prototypes in GCC.
>
> This should improve LTO behavior.
>
> Note that set_free_space_tree_thresholds is an unused function. Adding
> inline seems to remove the unused function warning.
>
> Signed-off-by: Rosen Penev
I had
Replaced bswap with _ variants. While it's a glibc extension, all of the
popular libc implementations (glibc, uClibc, musl, BIONIC) seem to support
it.
Added static inline to two functions to match little endian variants. This
fixes a linking error experienced when compiling.
Signed-off-by:
pthread_t is a pointer type, not an integer one. The > 0 makes no sense
and throws a warning.
Signed-off-by: Rosen Penev
---
task-utils.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/task-utils.c b/task-utils.c
index a9bee8f..e4dcd36 100644
--- a/task-utils.c
+++
Found using -Wmissing-prototypes in GCC.
This should improve LTO behavior.
Note that set_free_space_tree_thresholds is an unused function. Adding
inline seems to remove the unused function warning.
Signed-off-by: Rosen Penev
---
btrfs.c | 2 +-
check/mode-lowmem.c | 2 +-
On Sat, Nov 03, 2018 at 05:24:52PM +0800, Qu Wenruo wrote:
> In extent-io self test, we need 2 ordered extents at its maximum size to
> do the test.
>
> Instead of using the intermediate numbers, use BTRFS_MAX_EXTENT_SIZE for
> @max_bytes, and twice @max_bytes for @total_dirty.
> This should
On Mon, Nov 5, 2018 at 6:27 AM, Austin S. Hemmelgarn
wrote:
> On 11/4/2018 11:44 AM, waxhead wrote:
>>
>> Sterling Windmill wrote:
>>>
>>> Out of curiosity, what led to you choosing RAID1 for data but RAID10
>>> for metadata?
>>>
>>> I've flip flipped between these two modes myself after finding
Hi,
Stupid gmail has put my email (or Qu's reply? ) to spam, so I just saw
the reply after I sent my reply (gmail asked me whether to remove it
from spam).
Anyway here is the requested output. Thanks for the help!
$ sudo btrfs check /dev/nvme0n1p2
Opening filesystem to check...
checksum verify
On Mon, Oct 22, 2018 at 02:13:27PM -0700, Omar Sandoval wrote:
> > > Omar Sandoval (6):
> > > mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> > > mm: export add_swap_extent()
> > > vfs: update swap_{,de}activate documentation
> > > Btrfs: prevent ioctls from interfering with a swap
Hi,
TL;DR: I want to save data from my unmountable btrfs partition.
I saw some commands in another thread "Salvage files from broken btrfs".
I use the most recent Manjaro live (kernel: 4.19.0-3-MANJARO,
btrfs-progs 4.17.1-1) to execute these commands.
$ sudo mount -o ro,nologreplay
On Mon, Nov 05, 2018 at 06:49:09PM +0800, Shaokun Zhang wrote:
> block_group_err shows the group system as a decimal value with a '0x'
> prefix, which is somewhat misleading.
Thanks. As this is user visible I'll add stable tag and get it to 4.20
in the next rc.
On Mon, Nov 5, 2018 at 4:34 PM David Sterba wrote:
>
> On Mon, Nov 05, 2018 at 04:30:35PM +, Filipe Manana wrote:
> > On Mon, Nov 5, 2018 at 4:29 PM David Sterba wrote:
> > >
> > > On Wed, Oct 24, 2018 at 01:48:40PM +0100, Filipe Manana wrote:
> > > > > Ah ok makes sense. Well in that case
On Mon, Nov 05, 2018 at 04:30:35PM +, Filipe Manana wrote:
> On Mon, Nov 5, 2018 at 4:29 PM David Sterba wrote:
> >
> > On Wed, Oct 24, 2018 at 01:48:40PM +0100, Filipe Manana wrote:
> > > > Ah ok makes sense. Well in that case lets just make
> > > > btrfs_read_locked_inode()
> > > > take a
On Mon, Nov 5, 2018 at 4:29 PM David Sterba wrote:
>
> On Wed, Oct 24, 2018 at 01:48:40PM +0100, Filipe Manana wrote:
> > > Ah ok makes sense. Well in that case lets just make
> > > btrfs_read_locked_inode()
> > > take a path, and allocate it in btrfs_iget, that'll remove the ugly
> > >
> > >
On Wed, Oct 24, 2018 at 01:48:40PM +0100, Filipe Manana wrote:
> > Ah ok makes sense. Well in that case lets just make
> > btrfs_read_locked_inode()
> > take a path, and allocate it in btrfs_iget, that'll remove the ugly
> >
> > if (path != in_path)
>
> You mean the following on top of v4:
>
>
On Thu, Nov 01, 2018 at 02:09:46PM +0200, Nikolay Borisov wrote:
> This callback is called only from writepage_delalloc which in turn
> is guaranteed to be called from the data page writeout path. In the end
> there is no reason to have the call to this function to be indrected
> via the
On Thu, Nov 01, 2018 at 02:09:45PM +0200, Nikolay Borisov wrote:
> extent_io_ops has a set of 8 optional hooks which are set only for data and
> freespace inodes. The majority of them actually deal with delallocs in one
> way
> or another. Inspecting the code it transpired that there is
On Fri, Oct 05, 2018 at 05:45:53PM +0800, Qu Wenruo wrote:
> Inspired by Hans' possible flawed DUP chunk allocator, add the following
> dev extents checker:
>
> 1) Dev extent overlap check
>Dev extents don't use extent_cache so it can't report dev extents
>overlap.
>So manually check
On Fri, Nov 02, 2018 at 09:39:46AM +0800, Qu Wenruo wrote:
> Can be fetched from github:
> https://github.com/adam900710/linux/tree/refactor_find_free_extent
>
> Which is based on david's misc-4.20 branch.
> v5:
Now moved from for-next topic branch to misc-next, from now on please
send updates
On 2018/11/5 下午7:33, Zhangshaokun wrote:
> Hi Qu,
>
> On 2018/11/5 19:03, Qu Wenruo wrote:
>>
>>
>> On 2018/11/5 下午6:49, Shaokun Zhang wrote:
>>> block_group_err shows the group system as a decimal value with a '0x'
>>> prefix, which is somewhat misleading.
>>>
>>> Fix it to print hexadecimal,
On 2018/11/5 下午7:11, Filipe Manana wrote:
> On Mon, Nov 5, 2018 at 4:10 AM robbieko wrote:
>>
>> Filipe Manana 於 2018-10-30 19:36 寫到:
>>> On Tue, Oct 30, 2018 at 7:00 AM robbieko wrote:
From: Robbie Ko
In apply_children_dir_moves, we first create an empty list (stack),
On 11/4/2018 11:44 AM, waxhead wrote:
Sterling Windmill wrote:
Out of curiosity, what led to you choosing RAID1 for data but RAID10
for metadata?
I've flip flipped between these two modes myself after finding out
that BTRFS RAID10 doesn't work how I would've expected.
Wondering what made you
On 03.11.2018 at 02:05 Qu Wenruo wrote:
> On 2018/11/3 上午1:18, M. Klingmann wrote:
>> On 02.11.2018 at 15:45 Qu Wenruo wrote:
>>> On 2018/11/2 下午10:30, M. Klingmann wrote:
On 31.10.2018 at 01:03 Qu Wenruo wrote:
> My plan for such recovery is:
>
> 1) btrfs ins dump-super to make
Hi,
btrfs-progs version 4.19 have been released.
The version 4.18 was skipped to keep the time of release close to kernel. The
sort-of promise that 'progs version X supports features from kernel X' does not
hold for the user accessible ioctls to list subvolumes. As this is not a
critical feature
On 5.11.18 г. 13:15 ч., fdman...@kernel.org wrote:
> From: Filipe Manana
>
> A bug in file cloning/reflinking was recently found that afftected both
> Btrfs and XFS, which was caused by allowing the cloning of an eof block
> into the middle of a file when the eof is not aligned to the
On 5.11.18 г. 13:14 ч., fdman...@kernel.org wrote:
> From: Filipe Manana
>
> If we attempt to deduplicate the last block of a file A into the middle of
> a file B, and file A's size is not a multiple of the block size, we end
> rounding the deduplication length to 0 bytes, to avoid the data
Hi Qu,
On 2018/11/5 19:03, Qu Wenruo wrote:
>
>
> On 2018/11/5 下午6:49, Shaokun Zhang wrote:
>> block_group_err shows the group system as a decimal value with a '0x'
>> prefix, which is somewhat misleading.
>>
>> Fix it to print hexadecimal, as was intended.
>>
>> Cc: David Sterba
>> Cc: Chris
From: Filipe Manana
A bug in file cloning/reflinking was recently found that afftected both
Btrfs and XFS, which was caused by allowing the cloning of an eof block
into the middle of a file when the eof is not aligned to the filesystem's
block size.
The fix consists of returning the errno
From: Filipe Manana
Test that we can not clone a range from a file A into the middle of a file B
when the range includes the last block of file A and file A's size is not
aligned with the filesystem's block size. Allowing such case would lead to
data corruption since the data between EOF and the
From: Filipe Manana
Test that deduplication of an entire file that has a size that is not
aligned to the filesystem's block size into the middle of a different
file does not corrupt the destination's file data by reflinking the last
(eof) block.
This test is motivated by a bug recently found
From: Filipe Manana
We currently allow cloning a range from a file which includes the last
block of the file even if the file's size is not aligned to the block
size. This is fine and useful when the destination file has the same size,
but when it does not and the range ends somewhere in the
From: Filipe Manana
If we attempt to deduplicate the last block of a file A into the middle of
a file B, and file A's size is not a multiple of the block size, we end
rounding the deduplication length to 0 bytes, to avoid the data corruption
issue fixed by commit de02b9f6bb65 ("Btrfs: fix data
On Mon, Nov 5, 2018 at 4:10 AM robbieko wrote:
>
> Filipe Manana 於 2018-10-30 19:36 寫到:
> > On Tue, Oct 30, 2018 at 7:00 AM robbieko wrote:
> >>
> >> From: Robbie Ko
> >>
> >> In apply_children_dir_moves, we first create an empty list (stack),
> >> then we get an entry from pending_dir_moves
On 2018/11/5 下午6:49, Shaokun Zhang wrote:
> block_group_err shows the group system as a decimal value with a '0x'
> prefix, which is somewhat misleading.
>
> Fix it to print hexadecimal, as was intended.
>
> Cc: David Sterba
> Cc: Chris Mason
> Cc: Josef Bacik
> Signed-off-by: Shaokun
On 5.11.18 г. 12:49 ч., Shaokun Zhang wrote:
> block_group_err shows the group system as a decimal value with a '0x'
> prefix, which is somewhat misleading.
>
> Fix it to print hexadecimal, as was intended.
>
> Cc: David Sterba
> Cc: Chris Mason
> Cc: Josef Bacik
> Signed-off-by: Shaokun
block_group_err shows the group system as a decimal value with a '0x'
prefix, which is somewhat misleading.
Fix it to print hexadecimal, as was intended.
Cc: David Sterba
Cc: Chris Mason
Cc: Josef Bacik
Signed-off-by: Shaokun Zhang
---
fs/btrfs/tree-checker.c | 2 +-
1 file changed, 1
62 matches
Mail list logo
