Re: [patch] btrfs: array overflow in btrfs_ioctl_rm_dev_v2()

Thanks Dan. Chris pointed out as well. We are working on it.. Just one concern when device is added the max device length is BTRFS_PATH_NAME_MAX. However below fix is proper from the vol_args perspective. Thanks, Anand On 02/18/2016 01:01 PM, Dan Carpenter wrote: We were putting the

Re: [PATCH v4 12/13] btrfs: introduce device delete by devid

On 02/17/2016 06:49 PM, David Sterba wrote: On Sat, Feb 13, 2016 at 10:01:39AM +0800, Anand Jain wrote: + if (vol_args->flags & BTRFS_DEVICE_BY_ID) { + ret = btrfs_rm_device(root, NULL, vol_args->devid); + } else { + vol_args->name[BTRFS_PATH_NAME_MAX]

[RFC PATCH v2 5/5] fstests: btrfs: Test inband dedup with balance.

Btrfs balance will reloate date extent, but its hash is removed too late at run_delayed_ref() time, which will cause extent ref increased increased during balance, cause either find_data_references() gives WARN_ON() or even run_delayed_refs() fails and cause transaction abort. Add such

[RFC PATCH v2 2/5] fstests: btrfs: Add basic test for btrfs in-band de-duplication

Add basic test for btrfs in-band de-duplication, including: 1) Enable 2) Re-enable 3) On disk extents are refering to same bytenr 4) Disable Signed-off-by: Qu Wenruo --- common/defrag | 8 tests/btrfs/200 | 112

[RFC PATCH v2 4/5] fstests: btrfs: Add per inode dedup flag test

This test will check per inode dedup flag. Signed-off-by: Qu Wenruo --- tests/btrfs/202 | 117 tests/btrfs/202.out | 15 +++ tests/btrfs/group | 1 + 3 files changed, 133 insertions(+) create mode 100755

[RFC PATCH v2 0/5] Btrfs in-band de-duplication tests cases

Btrfs in-band de-duplication test cases for btrfs internal use only. Sequence number is not modified and uses number from 200 to avoid possible conflicts. Will modify sequence number and send to xfstests mail list after kernel and btrfs-progs patches get merged. Qu Wenruo (5): fstests: Add

[RFC PATCH v2 1/5] fstests: Add support to check btrfs sysfs features

Btrfs has its sysfs interface showing what features current kernel/btrfs module support. Add _require_btrfs_kernel_feature() to check such interface. Also rename _require_btrfs() to _require_btrfs_subcommand() to avoid confusion. Signed-off-by: Qu Wenruo --- common/rc

[RFC PATCH v2 3/5] fstests: btrfs: Add testcase for btrfs dedup enable disable race test

Add test case to check btrfs dedup enable/disable race. Signed-off-by: Qu Wenruo --- tests/btrfs/201 | 101 tests/btrfs/201.out | 1 + tests/btrfs/group | 1 + 3 files changed, 103 insertions(+) create mode

[PATCH 6/8] btrfs-progs: Add show-super support for new DEDUP flag

Now btrfs-show-super can handle DEDUP ro compat flag. Signed-off-by: Qu Wenruo --- btrfs-show-super.c | 17 + 1 file changed, 17 insertions(+) diff --git a/btrfs-show-super.c b/btrfs-show-super.c index 051bd11..0bc0b1f 100644 --- a/btrfs-show-super.c

[PATCH 3/8] btrfs-progs: dedup: Add disable support for inband deduplication

Add disable subcommand for dedup command group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedup.asciidoc | 5 + cmds-dedup.c | 42 ++ 2 files changed, 47 insertions(+) diff --git

[PATCH 8/8] btrfs-progs: property: add a dedup property

From: Wang Xiaoguang Normally if we enable online dedup for a fs, it's filesystem wide de-duplication. With this property, we can explicitly disable data de-duplication for specified files. Signed-off-by: Wang Xiaoguang ---

[PATCH 5/8] btrfs-progs: Add dedup feature for mkfs and convert

Add new DEDUP ro compat flag and corresponding mkfs/convert flag 'dedup'. Since dedup tree is completely isolated from fs tree, so even old kernel could do read mount. So add it to RO compat flag instead of common incompat flags Signed-off-by: Qu Wenruo ---

[PATCH 2/8] btrfs-progs: dedup: Add enable command for dedup command group

Add enable subcommand for dedup commmand group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedup.asciidoc | 101 ++- cmds-dedup.c | 138 + ioctl.h| 2 +

[PATCH 7/8] btrfs-progs: debug-tree: Add dedup tree support

Add dedup tree support for btrfs-debug-tree. Signed-off-by: Qu Wenruo --- v2: Add support to print hex objectid/offset for dedup hash. Add support to print hex hash. --- cmds-inspect-dump-tree.c | 4 +++ ctree.h | 7 print-tree.c

[PATCH 1/8] btrfs-progs: Basic framework for dedup command group

Add basic ioctl header and command group framework for later use. Alone with basic man page doc. Signed-off-by: Qu Wenruo --- Documentation/Makefile.in | 1 + Documentation/btrfs-dedup.asciidoc | 39 +++ Documentation/btrfs.asciidoc

[PATCH 4/8] btrfs-progs: dedup: Add status subcommand

Add status subcommand for dedup command group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedup.asciidoc | 3 ++ cmds-dedup.c | 76 ++ 2 files changed, 79 insertions(+) diff --git

[PATCH v7 02/20] btrfs: dedup: Introduce function to initialize dedup info

From: Wang Xiaoguang Add generic function to initialize dedup info. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/Makefile | 2 +- fs/btrfs/dedup.c | 97

[PATCH v7 03/20] btrfs: dedup: Introduce function to add hash into in-memory tree

From: Wang Xiaoguang Introduce static function inmem_add() to add hash into in-memory tree. And now we can implement the btrfs_dedup_add() interface. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang ---

[PATCH v7 04/20] btrfs: dedup: Introduce function to remove hash from in-memory tree

From: Wang Xiaoguang Introduce static function inmem_del() to remove hash from in-memory dedup tree. And implement btrfs_dedup_del() and btrfs_dedup_destroy() interfaces. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v7 19/20] btrfs: try more times to alloc metadata reserve space

From: Wang Xiaoguang In btrfs_delalloc_reserve_metadata(), the number of metadata bytes we try to reserve is calculated by the difference between outstanding_extents and reserved_extents. When reserve_metadata_bytes() fails to reserve desited metadata space, it has

[PATCH v7 18/20] btrfs: dedup: add per-file online dedup control

From: Wang Xiaoguang Introduce inode_need_dedup() to implement per-file online dedup control. Signed-off-by: Wang Xiaoguang --- fs/btrfs/inode.c | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git

[PATCH v7 07/20] btrfs: dedup: Implement btrfs_dedup_calc_hash interface

From: Wang Xiaoguang Unlike in-memory or on-disk dedup method, only SHA256 hash method is supported yet, so implement btrfs_dedup_calc_hash() interface using SHA256. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v7 10/20] btrfs: dedup: Add basic tree structure for on-disk dedup method

Introduce a new tree, dedup tree to record on-disk dedup hash. As a persist hash storage instead of in-memeory only implement. Unlike Liu Bo's implement, in this version we won't do hack for bytenr -> hash search, but add a new type, DEDUP_BYTENR_ITEM for such search case, just like in-memory

[PATCH v7 15/20] btrfs: dedup: Add ioctl for inband deduplication

From: Wang Xiaoguang Add ioctl interface for inband deduplication, which includes: 1) enable 2) disable 3) status We will later add ioctl to disable inband dedup for given file/dir. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v7 06/20] btrfs: dedup: Introduce function to search for an existing hash

From: Wang Xiaoguang Introduce static function inmem_search() to handle the job for in-memory hash tree. The trick is, we must ensure the delayed ref head is not being run at the time we search the for the hash. With inmem_search(), we can implement the

[PATCH v7 13/20] btrfs: dedup: Add support to delete hash for on-disk backend

Now on-disk backend can delete hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 100 +++ 1 file changed, 100 insertions(+) diff --git

[PATCH v7 20/20] btrfs: dedup: Fix a bug when running inband dedup with balance

From: Wang Xiaoguang When running inband dedup with balance, it's possible that inband dedup still increase ref on extents which are in RO chunk. This may cause either find_data_references() gives warning, or make run_delayed_refs() return -EIO and cause trans abort.

[PATCH v7 14/20] btrfs: dedup: Add support for adding hash for on-disk backend

Now on-disk backend can add hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 83 1 file changed, 83 insertions(+) diff --git a/fs/btrfs/dedup.c

[PATCH v7 08/20] btrfs: ordered-extent: Add support for dedup

From: Wang Xiaoguang Add ordered-extent support for dedup. Note, current ordered-extent support only supports non-compressed source extent. Support for compressed source extent will be added later. Signed-off-by: Qu Wenruo Signed-off-by:

[PATCH v7 11/20] btrfs: dedup: Introduce interfaces to resume and cleanup dedup info

Since we will introduce a new on-disk based dedup method, introduce new interfaces to resume previous dedup setup. And since we introduce a new tree for status, also add disable handler for it. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo

[PATCH v7 17/20] btrfs: dedup: add a property handler for online dedup

From: Wang Xiaoguang We use btrfs extended attribute "btrfs.dedup" to record per-file online dedup status, so add a dedup property handler. Signed-off-by: Wang Xiaoguang --- fs/btrfs/props.c | 40

[PATCH v7 09/20] btrfs: dedup: Inband in-memory only de-duplication implement

Core implement for inband de-duplication. It reuse the async_cow_start() facility to do the calculate dedup hash. And use dedup hash to do inband de-duplication at extent level. The work flow is as below: 1) Run delalloc range for an inode 2) Calculate hash for the delalloc range at the unit of

[PATCH v7 05/20] btrfs: delayed-ref: Add support for increasing data ref under spinlock

For in-band dedup, btrfs needs to increase data ref with delayed_ref locked, so add a new function btrfs_add_delayed_data_ref_lock() to increase extent ref with delayed_refs already locked. Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 1 +

[PATCH v7 16/20] btrfs: dedup: add an inode nodedup flag

From: Wang Xiaoguang Introduce BTRFS_INODE_NODEDUP flag, then we can explicitly disable online data deduplication for specified files. Signed-off-by: Wang Xiaoguang --- fs/btrfs/ctree.h | 1 + fs/btrfs/ioctl.c | 6 +- 2 files

[GIT PULL][PATCH v7 00/19][For 4.6] Btrfs: Add inband (write time) de-duplication framework

Hi Chris, This is the pull request for btrfs in-band de-duplication patchset. The patchset can also be fetched from github: https://github.com/adam900710/linux.git wang_dedup_20160218 Theis patchset went through several tests and seems quite good based on integration-4.5. We will continue test

[PATCH v7 12/20] btrfs: dedup: Add support for on-disk hash search

Now on-disk backend should be able to search hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 133 --- fs/btrfs/dedup.h | 1 + 2 files changed, 118

[PATCH v7 01/20] btrfs: dedup: Introduce dedup framework and its header

From: Wang Xiaoguang Introduce the header for btrfs online(write time) de-duplication framework and needed header. The new de-duplication framework is going to support 2 different dedup method and 1 dedup hash. Signed-off-by: Qu Wenruo

[patch] btrfs: array overflow in btrfs_ioctl_rm_dev_v2()

We were putting the NUL terminator at BTRFS_PATH_NAME_MAX (4087) bytes instead of BTRFS_SUBVOL_NAME_MAX (4039) so it corrupted memory. Fixes: 22af1a869288 ('btrfs: introduce device delete by devid') Signed-off-by: Dan Carpenter diff --git a/fs/btrfs/ioctl.c

Re: RAID 6 full, but there is still space left on some devices

Dan Blazejewski wrote on 2016/02/17 18:04 -0500: Hello, I upgraded my kernel to 4.4.2, and btrfs-progs to 4.4. I also added another 4TB disk and kicked off a full balance (currently 7x4TB RAID6). I'm interested to see what an additional drive will do to this. I'll also have to wait and see if

Re: [PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure

On 2016/02/18 0:11, David Sterba wrote: On Wed, Feb 17, 2016 at 02:54:23PM +0900, Satoru Takeuchi wrote: On 2016/02/16 2:53, David Sterba wrote: On Mon, Feb 15, 2016 at 02:38:09PM +0900, Satoru Takeuchi wrote: There are some BUG_ON()'s after kmalloc() as follows. = foo = kmalloc();

RE: btrfs: reada: simplify dev->reada_in_flight processing

Hi, Dan Carpenter > From: Dan Carpenter [mailto:dan.carpen...@oracle.com] > Sent: Thursday, February 18, 2016 3:02 AM > To: zhao...@cn.fujitsu.com > Cc: linux-btrfs@vger.kernel.org > Subject: re: btrfs: reada: simplify dev->reada_in_flight processing > > Hello Zhao Lei, > > The patch

Re: [PATCH 1/2] fstests: generic test for directory fsync after rename operation

On Mon, Feb 15, 2016 at 10:54:23AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that if we move one file between directories, fsync the parent > directory of the old directory, power fail and remount the filesystem, > the file is not lost and it's located

КЛИЕНТСКИЕ БАЗЫ! Тел\Viber\Whatsapp: +79139393506 Email: mgordee...@gmail.com Skype: prodawez389

КЛИЕНТСКИЕ БАЗЫ! Соберем для Вас по интернет базу данных потенциальных клиентов для Вашего Бизнеса! Много! Быстро! Недорого! Узнайте об этом подробнее по Тел: +79139393506 Viber: +79139393506 Whatsapp: +79139393506 Skype: prodawez389 Email: mgordee...@gmail.com -- To unsubscribe from this

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

Qu Wenruo wrote: > If you're really interesting in whether your fs has skinny metadata  > enabled, you can check btrfs-show-super output. > Like the following output indicates skinny metadata: > -- > incompat_flags0x161 > ( MIXED_BACKREF | >

Re: [PATCH][btrfs-progs] Include file verification with convert-tests

I think its a good idea to populate fs and verify them after conversion. I'm not sure about existing tools, simply we can create local script to this testsuite. Some thoughts on possible data set: -- a) regular files - empty files (touch) - smaller files dd if=/dev/zero - smaller files

re: btrfs: reada: simplify dev->reada_in_flight processing

Hello Zhao Lei, The patch 7aff519c04d2: "btrfs: reada: simplify dev->reada_in_flight processing" from Jan 12, 2016, leads to the following static checker warning: fs/btrfs/reada.c:697 reada_start_machine_dev() warn: inconsistent indenting fs/btrfs/reada.c 688

КЛИЕНТСКИЕ БАЗЫ! Тел\Viber\Whatsapp: +79139393506 Email: mgordee...@gmail.com Skype: prodawez389

КЛИЕНТСКИЕ БАЗЫ! Соберем для Вас по интернет базу данных потенциальных клиентов для Вашего Бизнеса! Много! Быстро! Недорого! Узнайте об этом подробнее по Тел: +79139393506 Viber: +79139393506 Whatsapp: +79139393506 Skype: prodawez389 Email: mgordee...@gmail.com -- To unsubscribe from this

[josef-btrfs:fdinfo 1/1] fs/notify/inotify/inotify_user.c:321:18: warning: initialization from incompatible pointer type

tree: https://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git fdinfo head: 9ecc561fd5b586bf648bb23c929750294e81c6db commit: 9ecc561fd5b586bf648bb23c929750294e81c6db [1/1] fdinfo: handle large fdinfo buffers config: x86_64-randconfig-x012-201607 (attached as .config) reproduce:

Re: [PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure

On Wed, Feb 17, 2016 at 02:54:23PM +0900, Satoru Takeuchi wrote: > On 2016/02/16 2:53, David Sterba wrote: > > On Mon, Feb 15, 2016 at 02:38:09PM +0900, Satoru Takeuchi wrote: > >> There are some BUG_ON()'s after kmalloc() as follows. > >> > >> = > >> foo = kmalloc(); > >> BUG_ON(!foo);

btrfs_orphan_cleanup

Hi all, after boot I am seeing this in dmesg [ 23.874698] BTRFS info (device sda1): enabling auto defrag [ 23.874702] BTRFS info (device sda1): disk space caching is enabled [ 26.074456] Adding 4193276k swap on /dev/sda4. Priority:-1 extents:1 across:4193276k [ 26.242940] [

Re: btrfs send error

Will do. Thank you. On Wed, Feb 17, 2016 at 3:01 PM, Filipe Manana wrote: > On Wed, Feb 17, 2016 at 1:02 PM, Pavol Cupka wrote: >> Hi all, >> >> when trying to btrfs send a snaphost I get this in dmesg: >> [8614395.539466] BTRFS error (device sda1):

Re: btrfs send error

On Wed, Feb 17, 2016 at 1:02 PM, Pavol Cupka wrote: > Hi all, > > when trying to btrfs send a snaphost I get this in dmesg: > [8614395.539466] BTRFS error (device sda1): did not find backref in > send_root. inode=673755, offset=131072, disk_byte=25730310144 found >

btrfs send error

Hi all, when trying to btrfs send a snaphost I get this in dmesg: [8614395.539466] BTRFS error (device sda1): did not find backref in send_root. inode=673755, offset=131072, disk_byte=25730310144 found extent=25730310144 scrub reveals no errors scrub status for

Re: [Docs]? Only one Subvolume with DUP (or different parameters)?

On 2016-02-16 23:49, Duncan wrote: Christian Völker posted on Tue, 16 Feb 2016 20:25:47 +0100 as excerpted: sorry for the simple question and I assume every developer here laughs about this question. Anyway: I have read loads of documents but did not find an answer for sure. Even though I

[PATCH] btrfs: fix ifnullfree.cocci warnings

fs/btrfs/volumes.c:1886:2-7: WARNING: NULL check before freeing functions like kfree, debugfs_remove, debugfs_remove_recursive or usb_free_urb is not needed. Maybe consider reorganizing relevant code to avoid passing NULL values. NULL check before some freeing functions is not needed. Based

Re: [PATCH v4 12/13] btrfs: introduce device delete by devid

On Sat, Feb 13, 2016 at 10:01:39AM +0800, Anand Jain wrote: > + if (vol_args->flags & BTRFS_DEVICE_BY_ID) { > + ret = btrfs_rm_device(root, NULL, vol_args->devid); > + } else { > + vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';

Re: [PATCH 1/3] generic/304: fix high offset

On Wed, Feb 17, 2016 at 10:04:23AM +, Filipe Manana wrote: > > What does btrfs return here? > > -EINVAL, so this makes the test pass on btrfs. Ok, looks fine to me then: Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in

Re: [PATCH 1/3] generic/304: fix high offset

On Wed, Feb 17, 2016 at 9:56 AM, Christoph Hellwig wrote: > On Mon, Feb 15, 2016 at 12:46:14PM -0800, Darrick J. Wong wrote: >> "Invalid argument" is a better response to an impossibly high offset >> dedupe request than "extents don't match", so change the test. > > What does

Re: [PATCH 3/3] xfs/24[356]: check for -c switch to xfs_io bmap command

Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/3] punch-alternating: use the block size reported by the fs for punching

Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/3] generic/304: fix high offset

On Mon, Feb 15, 2016 at 12:46:14PM -0800, Darrick J. Wong wrote: > "Invalid argument" is a better response to an impossibly high offset > dedupe request than "extents don't match", so change the test. What does btrfs return here? -- To unsubscribe from this list: send the line "unsubscribe