Re: Very quick embarrassing q: /var/lib permissions.

Satoru Takeuchi jp.fujitsu.com> writes: > Something is wrong with my setup or process. I'm still not getting progress reports (any longer) on scrub status and I wanted to make sure it's not me before I whinge again. :-D I've just, with your help, ruled out permissions. I next need to have a

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

At 06/02/2016 05:15 AM, Mark Fasheh wrote: Thanks for trying to fix this problem, comments below. On Wed, Jun 01, 2016 at 01:48:05PM +0800, Lu Fengqi wrote: Only in the case of different root_id or different object_id, check_shared identified extent as the shared. However, If a extent was

Re: [PATCH v7 0/8] Inband dedupe for btrfs-progs

On Wed, 1 Jun 2016 14:32:10 -0700 Mark Fasheh wrote: > Can we please have seperate and obvious namespaces for in-band dedupe and > out-of-band dedupe commands? I realize that there is no oob-dedupe > funcationality in btrfs-progs today but I would like to avoid confusing > users

Re: Very quick embarrassing q: /var/lib permissions.

On 2016/06/02 10:20, al wrote: Satoru Takeuchi jp.fujitsu.com> writes: Thank you, sir! I wonder if you would let me have the permissions (only) of any of the files you have inside your equivalent directory. Here it is. === # ls -l /var/lib/btrfs/ total 4 -rw---. 1 root

Re: How to map extents to files

At 06/02/2016 11:06 AM, Nikolaus Rath wrote: Hello, For one of my btrfs volumes, btrfsck reports a lot of the following warnings: [...] checking extents bad extent [138477568, 138510336), type mismatch with chunk bad extent [140091392, 140148736), type mismatch with chunk bad extent

How to map extents to files

Hello, For one of my btrfs volumes, btrfsck reports a lot of the following warnings: [...] checking extents bad extent [138477568, 138510336), type mismatch with chunk bad extent [140091392, 140148736), type mismatch with chunk bad extent [140148736, 140201984), type mismatch with chunk bad

Problems with storing Mercurial repositories

Hello, A little while ago I noticed that a btrfsck of my home directory produced a *lot* of the following errors: [...] checking fs roots root 5 inode 3149867 errors 400, nbytes wrong root 5 inode 3150237 errors 400, nbytes wrong root 5 inode 3150238 errors 400, nbytes wrong [...] I've now

Re: [PATCH v10 09/21] btrfs: dedupe: Inband in-memory only de-duplication implement

At 06/02/2016 06:08 AM, Mark Fasheh wrote: On Fri, Apr 01, 2016 at 02:35:00PM +0800, Qu Wenruo wrote: Core implement for inband de-duplication. It reuse the async_cow_start() facility to do the calculate dedupe hash. And use dedupe hash to do inband de-duplication at extent level. The work

Re: [PATCH v10 08/21] btrfs: ordered-extent: Add support for dedupe

At 06/02/2016 06:06 AM, Mark Fasheh wrote: On Fri, Apr 01, 2016 at 02:34:59PM +0800, Qu Wenruo wrote: From: Wang Xiaoguang Add ordered-extent support for dedupe. Note, current ordered-extent support only supports non-compressed source extent. Support for

Re: [PATCH v10 04/21] btrfs: dedupe: Introduce function to remove hash from in-memory tree

At 06/02/2016 03:40 AM, Mark Fasheh wrote: On Fri, Apr 01, 2016 at 02:34:55PM +0800, Qu Wenruo wrote: From: Wang Xiaoguang Introduce static function inmem_del() to remove hash from in-memory dedupe tree. And implement btrfs_dedupe_del() and btrfs_dedup_destroy()

Re: [PATCH v10 03/21] btrfs: dedupe: Introduce function to add hash into in-memory tree

At 06/02/2016 03:37 AM, Mark Fasheh wrote: On Fri, Apr 01, 2016 at 02:34:54PM +0800, Qu Wenruo wrote: From: Wang Xiaoguang Introduce static function inmem_add() to add hash into in-memory tree. And now we can implement the btrfs_dedupe_add() interface.

Re: [PATCH] btrfs-progs: convert: Insert needed holes for superblock migration

At 06/01/2016 09:49 PM, David Sterba wrote: On Wed, Jun 01, 2016 at 04:29:43PM +0800, Qu Wenruo wrote: New convert doesn't insert holes for superblock migration range. Unlike old design, which only relocate 4K(superblock size) to other places. In new design, to make sure convert can handle

raid5/6 production use status?

Hey. I've lost a bit track recently and the wiki changelog doesn't seem to contain much about how things went on at the RAID5/6 front... so how're things going? Is it already more or less "productively" usable? What's still missing? I guess there still aren't any administrative tools that e.g.

Re: "No space left on device" and balance doesn't work

On Wed, Jun 1, 2016 at 11:06 PM, MegaBrutal wrote: > Hi Peter, > > I tried. I either get "Done, had to relocate 0 out of 33 chunks" or > "ERROR: error during balancing '/': No space left on device", and > nothing changes. > > > 2016-06-01 22:29 GMT+02:00 Peter Becker

Re: [PATCH v10 10/21] btrfs: try more times to alloc metadata reserve space

On Tue, May 17, 2016 at 03:20:16PM +0200, David Sterba wrote: > On Fri, Apr 01, 2016 at 02:35:01PM +0800, Qu Wenruo wrote: > > @@ -5815,6 +5817,23 @@ out_fail: > > } > > if (delalloc_lock) > > mutex_unlock(_I(inode)->delalloc_mutex); > > + /* > > +* The number of metadata

Re: [PATCH v10 09/21] btrfs: dedupe: Inband in-memory only de-duplication implement

On Fri, Apr 01, 2016 at 02:35:00PM +0800, Qu Wenruo wrote: > Core implement for inband de-duplication. > It reuse the async_cow_start() facility to do the calculate dedupe hash. > And use dedupe hash to do inband de-duplication at extent level. > > The work flow is as below: > 1) Run delalloc

Re: [PATCH v10 08/21] btrfs: ordered-extent: Add support for dedupe

On Fri, Apr 01, 2016 at 02:34:59PM +0800, Qu Wenruo wrote: > From: Wang Xiaoguang > > Add ordered-extent support for dedupe. > > Note, current ordered-extent support only supports non-compressed source > extent. > Support for compressed source extent will be added

Re: [PATCH v7 0/8] Inband dedupe for btrfs-progs

On Fri, Apr 01, 2016 at 02:37:42PM +0800, Qu Wenruo wrote: > No much change from previous version. > 1) Rebased to latest devel branch > > 2) Update ctree.h to follow kernel structure change > > 3) Update print-tree to follow kernel structure change > > Qu Wenruo (7): > btrfs-progs: Basic

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > > +static int ref_tree_add(struct ref_root *ref_tree, u64 root_id, u64 > > object_id, > > + u64 offset, u64 parent, int count) > > +{ > > + struct ref_node *node = NULL; > > + struct rb_node **pos = NULL; > > +

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

Thanks for trying to fix this problem, comments below. On Wed, Jun 01, 2016 at 01:48:05PM +0800, Lu Fengqi wrote: > Only in the case of different root_id or different object_id, check_shared > identified extent as the shared. However, If a extent was referred by > different offset of same file,

Re: "No space left on device" and balance doesn't work

Hi Peter, I tried. I either get "Done, had to relocate 0 out of 33 chunks" or "ERROR: error during balancing '/': No space left on device", and nothing changes. 2016-06-01 22:29 GMT+02:00 Peter Becker : > try this: > > btrfs fi balance start -musage=0 / > btrfs fi balance

Re: "No space left on device" and balance doesn't work

try this: btrfs fi balance start -musage=0 / btrfs fi balance start -dusage=0 / btrfs fi balance start -musage=1 / btrfs fi balance start -dusage=1 / btrfs fi balance start -musage=5 / btrfs fi balance start -musage=10 / btrfs fi balance start -musage=20 / btrfs fi balance start -dusage=5 /

Re: [PATCH v10 04/21] btrfs: dedupe: Introduce function to remove hash from in-memory tree

On Fri, Apr 01, 2016 at 02:34:55PM +0800, Qu Wenruo wrote: > From: Wang Xiaoguang > > Introduce static function inmem_del() to remove hash from in-memory > dedupe tree. > And implement btrfs_dedupe_del() and btrfs_dedup_destroy() interfaces. > > Signed-off-by: Qu

Re: [PATCH v10 03/21] btrfs: dedupe: Introduce function to add hash into in-memory tree

On Fri, Apr 01, 2016 at 02:34:54PM +0800, Qu Wenruo wrote: > From: Wang Xiaoguang > > Introduce static function inmem_add() to add hash into in-memory tree. > And now we can implement the btrfs_dedupe_add() interface. > > Signed-off-by: Qu Wenruo

"No space left on device" and balance doesn't work

Hi all, I have a 20 GB file system and df says I have about 2,6 GB free space, yet I can't do anything on the file system because I get "No space left on device" errors. I read that balance may help to remedy the situation, but it actually doesn't. Some data about the FS:

Re: [PATCH V2 0/8] Btrfs: self-tests: Support non-4k page size

On 06/01/2016 11:51 AM, Feifei Xu wrote: Hi, Btrfs self-test module assumed that both sectorsize and PAGE_SIZE are 4K. Thus many self-tests fail on non-4K page size systems, like ppc64. This patchset enables self-tests to be executed on non-4k page size systems. This patchset enables us to

Re: [PATCH V2 0/8] Btrfs: self-tests: Support non-4k page size

On 2016/6/1 23:51, Feifei Xu wrote: Hi, Btrfs self-test module assumed that both sectorsize and PAGE_SIZE are 4K. Thus many self-tests fail on non-4K page size systems, like ppc64. This patchset enables self-tests to be executed on non-4k page size systems. This patchset enables us to easily

[PATCH V2 5/8] Btrfs: self-tests: Support testing all possible sectorsizes and nodesizes

To test all possible sectorsizes, this commit adds a sectorsize array. This commit executes the tests for all possible sectorsizes and nodesizes. Signed-off-by: Chandan Rajendra Signed-off-by: Feifei Xu --- fs/btrfs/super.c | 54

[PATCH V2 2/8] Btrfs: Fix integer overflow when calculating bytes_per_bitmap

On ppc64, bytes_per_bitmap will be (65536*8*65536). Hence append UL to fix integer overflow. Reviewed-by: Chandan Rajendra Signed-off-by: Feifei Xu --- fs/btrfs/free-space-cache.c | 12 ++--

[PATCH V2 6/8] Btrfs: self-tests: Use macros instead of constants and add missing newline

This commit replaces numerical constants with appropriate preprocessor macros. Signed-off-by: Chandan Rajendra Signed-off-by: Feifei Xu --- fs/btrfs/disk-io.c | 3 ++- fs/btrfs/tests/extent-buffer-tests.c | 2 +-

[PATCH V2 8/8] Btrfs: self-tests: Fix extent buffer bitmap test fail on BE system

In __test_eb_bitmaps(), we write random data to a bitmap. Then copy the bitmap to another bitmap that resides inside an extent buffer. Later we verify the values of corresponding bits in the bitmap and the bitmap inside the extent buffer. However, extent_buffer_test_bit() reads in byte granularity

[PATCH V2 1/8] Btrfs: test_check_exists: Fix infinite loop when searching for free space entries

On a ppc64 machine using 64K as the block size, assume that the RB tree at btrfs_free_space_ctl->free_space_offset contains following two entries: 1. A bitmap entry having an offset value of 0 and having the bits corresponding to the address range [128M+512K, 128M+768K] set. 2. An extent entry

[PATCH V2 3/8] Btrfs: self-tests: Support non-4k page size

self-tests code assumes 4k as the sectorsize and nodesize. This commit fix hardcoded 4K. Enables the self-tests code to be executed on non-4k page sized systems (e.g. ppc64). Signed-off-by: Feifei Xu Signed-off-by: Chandan Rajendra ---

[PATCH V2 0/8] Btrfs: self-tests: Support non-4k page size

Hi, Btrfs self-test module assumed that both sectorsize and PAGE_SIZE are 4K. Thus many self-tests fail on non-4K page size systems, like ppc64. This patchset enables self-tests to be executed on non-4k page size systems. This patchset enables us to easily add support for possible sectorsizes

[PATCH V2 7/8] Btrfs: self-tests: Fix test_bitmaps fail on 64k sectorsize

With 64K sectorsize, 1G sized block group cannot span across bitmaps. To execute test_bitmaps() function, this commit allocates "BITS_PER_BITMAP * sectorsize + PAGE_SIZE" sized block group. Reviewed-by: Chandan Rajendra Signed-off-by: Feifei Xu

[PATCH V2 4/8] Btrfs: self-tests: Execute page straddling test only when nodesize < PAGE_SIZE

On ppc64, PAGE_SIZE is 64k which is same as BTRFS_MAX_METADATA_BLOCKSIZE. In such a scenario, we will never be able to have an extent buffer containing more than one page. Hence in such cases this commit does not execute the page straddling tests. Signed-off-by: Feifei Xu

Re: [PATCH] btrfs: fix check_shared for fiemap ioctl

On Wed, Jun 01, 2016 at 09:23:57AM +0800, luke wrote: > > > At 06/01/2016 12:15 AM, David Sterba wrote: > > On Tue, May 31, 2016 at 11:08:39AM +0800, luke wrote: > +}; > + > +/* dynamically allocate and initialize a ref_root */ > +static struct ref_root *ref_root_alloc(gfp_t

Re: [PATCH v2 1/2] btrfs-progs: utils: Introduce new pseudo random API

On Thu, May 26, 2016 at 10:56:50AM +0800, Qu Wenruo wrote: > David has reported some quite chaos usage of pseudo random numbers. > Like using static srand seed, or even calling rand() without setting > seed correctly. > > The new pseudo random API will initialize the random seed on its first >

Re: [PATCH] btrfs-progs: corrupt-block: Add support to corrupt extent for skinny metadata

On Thu, May 26, 2016 at 10:36:02AM +0800, Qu Wenruo wrote: > For skinny metadata case, btrfs-corrupt-block can't corrupt a metadata > extent due to the type judgment doesn't include BTRFS_METADATA_ITEM_KEY. > > Fix it. > > Signed-off-by: Qu Wenruo Applied, thanks. --

Re: [PATCH] btrfs-progs: fsck: Add more explanation on low memory mode

On Fri, May 20, 2016 at 10:21:07AM +0800, Qu Wenruo wrote: > It seems the initial support for low memory mode will come in recent > release, it's better to add more explanation on the unsupported > functions, to avoid user reporting known bugs. > > Signed-off-by: Qu Wenruo

Re: [PATCH] btrfs-progs: convert: Insert needed holes for superblock migration

On Wed, Jun 01, 2016 at 04:29:43PM +0800, Qu Wenruo wrote: > New convert doesn't insert holes for superblock migration range. > > Unlike old design, which only relocate 4K(superblock size) to other > places. > In new design, to make sure convert can handle different page size and > align chunk

Re: [PATCH] btrfs-progs: convert: Fix bugs in backup superblock migration

On Wed, Jun 01, 2016 at 04:22:26PM +0800, Qu Wenruo wrote: > New convert has several bugs with backup superblock migration > > 1) Backup superblocks are not migrated due to bad judgement >Two wrong judgement cause backup superblocks are not migrated at all > > 2) Converted ext* image doesn't

Re: [PATCH] btrfs-progs: convert-tests: Add test for backup superblock migration

On Wed, Jun 01, 2016 at 09:51:21AM +0800, Qu Wenruo wrote: > New convert framework uses new and simpler chunk layout, while the cost > is the more complex superblock range migration logical, compared to old > convert. > > Enhance the convert test script to create file which will takes up 2nd >

Re: [GIT PULL] Device replace fixes

On Wed, Jun 01, 2016 at 12:44:13PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Hi Chris, > > Please consider the following device replace related fixes for inclusion > in a future 4.7 release candidate kernel. These were all previously sent > to the mailing

Re: [PATCH V2 5/8] Btrfs: self-tests: Support testing all possible sectorsizes and nodesizes

On Wed, Jun 01, 2016 at 02:21:18PM +0200, David Sterba wrote: > > static int btrfs_run_sanity_tests(void) > > { > > - int ret; > > + int ret, i; > > u32 sectorsize, nodesize; > > - > > - sectorsize = PAGE_SIZE; > > - nodesize = PAGE_SIZE; > > This does not seem to apply to any

[PATCH V2 4/8] Btrfs: self-tests: Execute page straddling test only when nodesize < PAGE_SIZE

On ppc64, PAGE_SIZE is 64k which is same as BTRFS_MAX_METADATA_BLOCKSIZE. In such a scenario, we will never be able to have an extent buffer containing more than one page. Hence in such cases this commit does not execute the page straddling tests. Signed-off-by: Feifei Xu

[PATCH V2 3/8] Btrfs: self-tests: Support non-4k page size

self-tests code assumes 4k as the sectorsize and nodesize. This commit fix hardcoded 4K. Enables the self-tests code to be executed on non-4k page sized systems (e.g. ppc64). Signed-off-by: Feifei Xu Signed-off-by: Chandan Rajendra ---

Re: [PATCH V2 5/8] Btrfs: self-tests: Support testing all possible sectorsizes and nodesizes

On Wed, Jun 01, 2016 at 07:18:27PM +0800, Feifei Xu wrote: > To test all possible sectorsizes, this commit adds a sectorsize > array. This commit executes the tests for all possible sectorsizes and > nodesizes. > > Signed-off-by: Chandan Rajendra > Signed-off-by:

Re: [PATCH V2 0/8] Btrfs: self-tests: Support non-4k page size

On Wed, Jun 01, 2016 at 07:18:22PM +0800, Feifei Xu wrote: > Btrfs self-test module assumed that both sectorsize and PAGE_SIZE are 4K. > Thus many self-tests fail on non-4K page size systems, like ppc64. This > patchset enables self-tests to be executed on non-4k page size systems. > > This

[GIT PULL] Device replace fixes

From: Filipe Manana Hi Chris, Please consider the following device replace related fixes for inclusion in a future 4.7 release candidate kernel. These were all previously sent to the mailing list and reviewed by Josef (and I recently rebased them against your for-linus-4.7

Re: [PATCH] btrfs-progs: fix uninit variables in btrfs-image

On Wed, Jun 01, 2016 at 10:15:47AM +0800, Lu Fengqi wrote: > Signed-off-by: Lu Fengqi Patch folded to the original. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

[PATCH V2 5/8] Btrfs: self-tests: Support testing all possible sectorsizes and nodesizes

To test all possible sectorsizes, this commit adds a sectorsize array. This commit executes the tests for all possible sectorsizes and nodesizes. Signed-off-by: Chandan Rajendra Signed-off-by: Feifei Xu --- fs/btrfs/super.c | 54

[PATCH V2 8/8] Btrfs: self-tests: Fix extent buffer bitmap test fail on BE system

In __test_eb_bitmaps(), we write random data to a bitmap. Then copy the bitmap to another bitmap that resides inside an extent buffer. Later we verify the values of corresponding bits in the bitmap and the bitmap inside the extent buffer. However, extent_buffer_test_bit() reads in byte granularity

[PATCH V2 7/8] Btrfs: self-tests: Fix test_bitmaps fail on 64k sectorsize

With 64K sectorsize, 1G sized block group cannot span across bitmaps. To execute test_bitmaps() function, this commit allocates "BITS_PER_BITMAP * sectorsize + PAGE_SIZE" sized block group. Reviewed-by: Chandan Rajendra Signed-off-by: Feifei Xu

[PATCH V2 6/8] Btrfs: self-tests: Use macros instead of constants and add missing newline

This commit replaces numerical constants with appropriate preprocessor macros. Signed-off-by: Chandan Rajendra Signed-off-by: Feifei Xu --- fs/btrfs/disk-io.c | 3 ++- fs/btrfs/tests/extent-buffer-tests.c | 2 +-

[PATCH V2 2/8] Btrfs: Fix integer overflow when calculating bytes_per_bitmap

On ppc64, bytes_per_bitmap will be (65536*8*65536). Hence append UL to fix integer overflow. Reviewed-by: Chandan Rajendra Signed-off-by: Feifei Xu --- fs/btrfs/free-space-cache.c | 12 ++--

[PATCH V2 1/8] Btrfs: test_check_exists: Fix infinite loop when searching for free space entries

On a ppc64 machine using 64K as the block size, assume that the RB tree at btrfs_free_space_ctl->free_space_offset contains following two entries: 1. A bitmap entry having an offset value of 0 and having the bits corresponding to the address range [128M+512K, 128M+768K] set. 2. An extent entry

[PATCH V2 0/8] Btrfs: self-tests: Support non-4k page size

Hi, Btrfs self-test module assumed that both sectorsize and PAGE_SIZE are 4K. Thus many self-tests fail on non-4K page size systems, like ppc64. This patchset enables self-tests to be executed on non-4k page size systems. This patchset enables us to easily add support for possible sectorsizes

Re: Hot data tracking / hybrid storage

On 2016-05-29 22:45, Ferry Toth wrote: Op Sun, 29 May 2016 12:33:06 -0600, schreef Chris Murphy: On Sun, May 29, 2016 at 12:03 PM, Holger Hoffstätte wrote: On 05/29/16 19:53, Chris Murphy wrote: But I'm skeptical of bcache using a hidden area historically for

[PATCH] btrfs-progs: convert: Insert needed holes for superblock migration

New convert doesn't insert holes for superblock migration range. Unlike old design, which only relocate 4K(superblock size) to other places. In new design, to make sure convert can handle different page size and align chunk bytenr, we relocate the whole 64K range. And if there is only a 4K used

[PATCH] btrfs-progs: convert: Fix bugs in backup superblock migration

New convert has several bugs with backup superblock migration 1) Backup superblocks are not migrated due to bad judgement Two wrong judgement cause backup superblocks are not migrated at all 2) Converted ext* image doesn't keep hole for backup superblocks Since we are creating file extents

[PATCH v2] fstests: btrfs: add test for qgroup handle extent de-reference

Test if qgroup can handle extent de-reference during reallocation. "extent de-reference" means that reducing an extent's reference count or freeing an extent. Although current qgroup can handle it, we still need to prevent any regression which may break current qgroup. Signed-off-by: Lu Fengqi