On Thu, Sep 22, 2016 at 12:37:53PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
>
> The behavior of different combinations of space_cache mount options
> wasn't well defined, which led to a regression between my initial
> patches adding the free space tree and the 4.5 release. Add a test to
On Wed, Sep 21, 2016 at 08:32:33AM -0400, je...@suse.com wrote:
> From: Jeff Mahoney
>
> There was a bug where the btrfs snapshot/subvol creation ioctls
> would accept a regular file and then Oops when it tried to use
> the file inode operations to do a lookup.
>
> This also adds an ioctl-helper
Hi,
On 09/23/2016 12:58 PM, Eryu Guan wrote:
On Fri, Sep 23, 2016 at 11:12:07AM +0800, Wang Xiaoguang wrote:
In btrfs, there is a bug about btrfs' truncate codes, it'll leak
some fs space as the truncate operation proceeds. If this truncate
operation is very large, later metadata request in thi
Rich Freeman posted on Thu, 22 Sep 2016 07:18:35 -0500 as excerpted:
> I have been getting panics consistently after doing a btrfs replace
> operation on a raid1 and rebooting. I linked a photo of the panic; I
> haven't been able to get a text capture of it.
>
> https://ibin.co/2vx0HhDeViu3.jpg
On Fri, Sep 23, 2016 at 11:12:07AM +0800, Wang Xiaoguang wrote:
> In btrfs, there is a bug about btrfs' truncate codes, it'll leak
> some fs space as the truncate operation proceeds. If this truncate
> operation is very large, later metadata request in this truncate
> operation may fail for enospc
Fixes: ("btrfs: update btrfs_space_info's bytes_may_use timely")
Signed-off-by: Wang Xiaoguang
---
fs/btrfs/extent-tree.c | 13 +++--
1 file changed, 3 insertions(+), 10 deletions(-)
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 38c2df8..9fbcfaa 100644
--- a/fs/btrf
In btrfs, there is a bug about btrfs' truncate codes, it'll leak
some fs space as the truncate operation proceeds. If this truncate
operation is very large, later metadata request in this truncate
operation may fail for enospc error. I also have sent a kernel
patch fot btrfs to fix this issue.
Sig
1614 gen 8019 top level 5 path everything-20160805
ID 1615 gen 8019 top level 5 path everything-20160831
ID 1617 gen 8025 top level 5 path everything-20160913
ID 1629 gen 8030 top level 5 path everything-20160922
[root@f24s brick2]# btrfs sub show /brick2/everything-20160218/
/brick2/everything
On Thu, Sep 22, 2016 at 8:03 PM, Chris Murphy wrote:
> [chris@f24s first]$ sudo btrfs sub list -t /brick2
> IDgentop levelpath
> ------
> 120880195everything-20160218
[...snip...]
> [chris@f24s first]$ sudo btrfs sub show everything-20160218
>
everything-20160720
161480195everything-20160805
161580195everything-20160831
161780255everything-20160913
162980285 everything-20160922
[chris@f24s first]$ sudo btrfs sub show everything-20160218
/mnt/first/eve
At 09/23/2016 02:47 AM, Goldwyn Rodrigues wrote:
From: Goldwyn Rodrigues
While free'ing qgroup->reserved resources, we must check
if the page is already commmitted to disk or still in memory.
If not, the reserve free is doubly accounted, once while
invalidating the page, and the next time whi
From: Omar Sandoval
In convert_free_space_to_{bitmaps,extents}(), we buffer the free space
bitmaps in memory and copy them directly to/from the extent buffers with
{read,write}_extent_buffer(). The extent buffer bitmap helpers use byte
granularity, which is equivalent to a little-endian bitmap. T
From: Omar Sandoval
The in-memory bitmap code manipulates words and is therefore sensitive
to endianness, while the extent buffer bitmap code addresses bytes and
is byte-order agnostic. Because the byte addressing of the extent buffer
bitmaps is equivalent to a little-endian in-memory bitmap, the
From: Omar Sandoval
test_find_delalloc() allocates 256 MB worth of pages. That's all of the
RAM that my MIPS emulator has, so it ends up panicking after it OOM
kills everything. We don't actually need to use that much for this test.
Signed-off-by: Omar Sandoval
---
fs/btrfs/tests/extent-io-tes
From: Omar Sandoval
The free space tree format conversion functions were broken on
big-endian systems, but the sanity tests didn't catch it because all of
the operations were aligned to multiple words. This was meant to catch
any bugs in the extent buffer code's handling of high memory, but it
en
From: Omar Sandoval
There are two separate issues that can lead to corrupted free space
trees.
1. The free space tree bitmaps had an endianness issue on big-endian
systems which is fixed by an earlier patch in this series.
2. btrfs-progs before v4.7.3 modified filesystems without updating the
From: Omar Sandoval
We moved the code for creating the free space tree the first time that
it's enabled, but didn't move the clearing code along with it. This
breaks my (undocumented) intention that `mount -o
clear_cache,space_cache=v2` would clear the free space tree and then
recreate it.
Fixes
From: Omar Sandoval
This is v2 of my earlier series "Btrfs: fix free space tree
bitmaps+tests on big-endian systems" [1]. Patches 1, 4, and 5 are the
same as patches 1, 2, and 3 from the original series. I've added patch 2
to fix another bug I noticed (an xfstest went out earlier). Patch 3 is
the
Hi,
Same error when mouting with this fedora iso, with mount, mount -o
recovery and mount -o ro,recovery
#
[ 682.954511] BTRFS info (device sdc2): disk space caching is enabled
[ 682.954518] BTRFS info (device sdc2): ha
On 09/21/2016 07:14 AM, Paul Jones wrote:
-Original Message-
From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-
ow...@vger.kernel.org] On Behalf Of Zygo Blaxell
Sent: Wednesday, 21 September 2016 2:56 PM
To: linux-btrfs@vger.kernel.org
Subject: btrfs rare silent data corruption
Hi Josef,
as we talked off line, I think the idea of maintaining a byte counter
and rounding in balance_dirty_pages() is the best way to do this. And
Jan spotted all the actual bugs, so I only have a few nitpicks :)
On Tue, Sep 20, 2016 at 04:57:46PM -0400, Josef Bacik wrote:
> @@ -44,12 +44,13 @
I started using snapper (v0.2.4) on 2016-09-18 and then 2 days later
while deleting snapshots at 02:16 the system ran out of memory (2GB)
[1].
At this point there were 6 /home subvolumes with the following snapshots:
Type | # | Pre # | Date | User | Cleanup |
Descripti
On Thu, 2016-09-22 at 19:49 +0200, Kai Krakow wrote:
> I think mysql data files
> have
> their own checksumming
Last time I've checked it, none of the major DBs or VM-image formats
had this... postgresql being the only one supporting something close to
fs level csumming (but again not per default).
From: Omar Sandoval
The behavior of different combinations of space_cache mount options
wasn't well defined, which led to a regression between my initial
patches adding the free space tree and the 4.5 release. Add a test to
exercise all of the meaningful permutations of clear_cache,
nospace_cache
From: Goldwyn Rodrigues
Fixes: 7cf5b97650f2 ("btrfs: qgroup: Cleanup old inaccurate facilities")
Signed-off-by: Goldwyn Rodrigues
---
fs/btrfs/extent-tree.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 23b3b9f..a8a4931 100644
--- a/f
From: Goldwyn Rodrigues
Code cleanup. parent_start is initialized multiple times when it is
not necessary to do so.
Signed-off-by: Goldwyn Rodrigues
---
fs/btrfs/ctree.c | 18 +++---
1 file changed, 3 insertions(+), 15 deletions(-)
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
From: Goldwyn Rodrigues
Code cleanup. count is already (unsgined long)-1. That is the reason
run_all was set. Do not reassign it (unsigned long)-1.
Signed-off-by: Goldwyn Rodrigues
---
fs/btrfs/extent-tree.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/e
From: Goldwyn Rodrigues
While free'ing qgroup->reserved resources, we must check
if the page is already commmitted to disk or still in memory.
If not, the reserve free is doubly accounted, once while
invalidating the page, and the next time while free'ing
delalloc. This results is qgroup->reserve
On Thu, Sep 22, 2016 at 05:08:24PM +0200, David Sterba wrote:
> From: Josef Bacik
>
> Really there's lots of things that can go wrong here, kill all the
> BUG_ON()'s and replace the logic ones with ASSERT()'s and return EIO
> instead.
>
> Signed-off-by: Josef Bacik
> Signed-off-by: David Sterba
On 2016/09/22 17:26, Omar Sandoval wrote:
> As far as I know, space_cache=v2 isn't documented in the man page, so
> hopefully anyone who managed to find out about it follows the mailing
> list. If someone updates their kernel without updating btrfs-progs, the
> new compat_ro bit will prevent the o
Hi Josef,
Em qui, 2016-09-22 às 13:49 -0400, Josef Bacik escreveu:
> That patch fixed a problem where we would screw up the ENOSPC
> accounting, and
> would slowly leak space into one of the counters. So eventually (or
> often in
> your case) you'd hit ENOSPC, but have plenty of space available
On 09/22/2016 01:06 PM, Ronan Arraes Jardim Chagas wrote:
Hi Josef,
Em qui, 2016-09-22 às 10:39 -0400, Josef Bacik escreveu:
This is what fixed it. I thought it was in 4.7 which is why I
started paying
attention, but I guess I was wrong. Glad your problem is
resolved. Thanks,
Do you have a
On Thu, Sep 22, 2016 at 05:10:25PM +0200, David Sterba wrote:
> On Tue, Sep 13, 2016 at 07:02:27PM -0700, Liu Bo wrote:
> > The extent buffer 'next' needs to be free'd conditionally.
> >
> > Signed-off-by: Liu Bo
>
> Reviewed-by: David Sterba
>
> Josef's patch contained this fix but mixed with
On Thu, Sep 22, 2016 at 05:47:59PM +, ojab wrote:
> On 2016/09/22 17:26, Omar Sandoval wrote:
> > As far as I know, space_cache=v2 isn't documented in the man page, so
> > hopefully anyone who managed to find out about it follows the mailing
> > list. If someone updates their kernel without upd
Am Wed, 21 Sep 2016 09:02:09 -0400
schrieb Zygo Blaxell :
> On Wed, Sep 21, 2016 at 11:14:35AM +, Paul Jones wrote:
> > > -Original Message-
> > > From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-
> > > ow...@vger.kernel.org] On Behalf Of Zygo Blaxell
> > > Sent: Wednesday,
On 9/22/16 8:18 AM, Rich Freeman wrote:
> I have been getting panics consistently after doing a btrfs replace
> operation on a raid1 and rebooting. I linked a photo of the panic; I
> haven't been able to get a text capture of it.
>
> https://ibin.co/2vx0HhDeViu3.jpg
>
> I'm getting this error on
On Thu, Sep 22, 2016 at 10:46 AM, Rich Freeman
wrote:
> On Sep 22, 2016 8:18 AM, "Rich Freeman" wrote:
>>
>> I have been getting panics consistently after doing a btrfs replace
>> operation on a raid1 and rebooting. I linked a photo of the panic; I
>> haven't been able to get a text capture of i
On Thu, Sep 22, 2016 at 12:10:09PM +0200, Hans van Kranenburg wrote:
> Hi,
>
> On 09/22/2016 10:52 AM, David Sterba wrote:
> > On Wed, Sep 21, 2016 at 01:31:52PM -0700, Omar Sandoval wrote:
> I'm not sure I understand - can you explain why this is was so wrong?
> Or Omar maybe?
>
>
Hi Josef,
Em qui, 2016-09-22 às 10:39 -0400, Josef Bacik escreveu:
> This is what fixed it. I thought it was in 4.7 which is why I
> started paying
> attention, but I guess I was wrong. Glad your problem is
> resolved. Thanks,
Do you have any explanations why the problem solved by the patch w
On Thu, Sep 22, 2016 at 10:52:05AM +0200, David Sterba wrote:
> On Wed, Sep 21, 2016 at 01:31:52PM -0700, Omar Sandoval wrote:
> > > > I'm not sure I understand - can you explain why this is was so wrong?
> > > > Or Omar maybe?
> > > >
> > > > If btrfsck wants to correct something (write), it can
On Sep 22, 2016 8:18 AM, "Rich Freeman" wrote:
>
> I have been getting panics consistently after doing a btrfs replace
> operation on a raid1 and rebooting. I linked a photo of the panic; I
> haven't been able to get a text capture of it.
>
> https://ibin.co/2vx0HhDeViu3.jpg
>
> I'm getting this
On Thu, Sep 22, 2016 at 02:44:44PM +0200, Holger Hoffstätte wrote:
> On 09/22/16 14:18, Rich Freeman wrote:
> > I have been getting panics consistently after doing a btrfs replace
> > operation on a raid1 and rebooting. I linked a photo of the panic; I
> > haven't been able to get a text capture o
On 9/22/16 12:56 AM, Anand Jain wrote:
> btrfs_show_devname() is using the device_list_mutex, sometimes
> a call to blkdev_put() leads vfs calling into this func. So
> call blkdev_put() outside of device_list_mutex, as of now.
>
> [ 983.284212]
On Wed, Sep 21, 2016 at 11:15:50AM +0800, Qu Wenruo wrote:
> The branch can be fetched from my github:
> https://github.com/adam900710/btrfs-progs/tree/lowmem_fs_tree
I've put it into integration but haven't reviewed nor tested it.
> Already merged lowmem mode fsck only works for extent/chunk tre
On Thu, Sep 22, 2016 at 12:56:13PM +0800, Anand Jain wrote:
> btrfs_show_devname() is using the device_list_mutex, sometimes
> a call to blkdev_put() leads vfs calling into this func. So
> call blkdev_put() outside of device_list_mutex, as of now.
>
[...]
>
> Reported-by: Ilya Dryomov
> Signed-o
On Tue, Sep 13, 2016 at 07:02:27PM -0700, Liu Bo wrote:
> The extent buffer 'next' needs to be free'd conditionally.
>
> Signed-off-by: Liu Bo
Reviewed-by: David Sterba
Josef's patch contained this fix but mixed with other (removing BUG_ON).
I've removed it so we can merge both patches, for sa
From: Josef Bacik
Really there's lots of things that can go wrong here, kill all the
BUG_ON()'s and replace the logic ones with ASSERT()'s and return EIO
instead.
Signed-off-by: Josef Bacik
Signed-off-by: David Sterba
---
fs/btrfs/extent-tree.c | 27 +++
1 file changed
On Wed, Sep 14, 2016 at 03:19:18PM +0200, Holger Hoffstätte wrote:
> On 09/14/16 04:02, Liu Bo wrote:
> > The extent buffer 'next' needs to be free'd conditionally.
> >
> > Signed-off-by: Liu Bo
> > ---
> > fs/btrfs/extent-tree.c | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/fs
On Wed, Sep 21, 2016 at 12:17:37PM -0400, Jeff Mahoney wrote:
> We can hit unused variable warnings when btrfs_debug and friends are
> just aliases for no_printk. This is due to the fs_info not getting
> consumed by the function call, which can happen if convenenience
> variables are used. This p
On Wed, Sep 21, 2016 at 08:31:29AM -0400, je...@suse.com wrote:
> From: Jeff Mahoney
>
> If the subvol/snapshot create/destroy ioctls are passed a regular file
> with execute permissions set, we'll eventually Oops while trying to do
> inode->i_op->lookup via lookup_one_len.
>
> This patch ensure
On 09/22/2016 10:03 AM, Ronan Arraes Jardim Chagas wrote:
Em qui, 2016-09-22 às 09:41 -0400, Austin S. Hemmelgarn escreveu:
Most likely the kernel upgrade fixed things. It's possible that the
large allocation is impacting something and making it work, but I
don't
think that that is very likely.
On 09/22/2016 02:41 AM, Stefan Priebe - Profihost AG wrote:
Hi,
i always encounter btrfs deadlocks / hung tasks, when i have a lot of
cached mem and i'm doing heavy rsync --inplace operations in my system
from btrfs zlib compressed disk A to btrfs zlib compressed disk B.
The last output i see
Em qui, 2016-09-22 às 09:41 -0400, Austin S. Hemmelgarn escreveu:
> Most likely the kernel upgrade fixed things. It's possible that the
> large allocation is impacting something and making it work, but I
> don't
> think that that is very likely.
The patches related to btrfs I could find in kern
On 2016-09-22 09:20, Ronan Arraes Jardim Chagas wrote:
Guys,
Something very strange happened. I have not seen the problem since
Monday, which is pretty much the first time ever I work more than 3
days without seeing it.
Ok, it can be a coincidence. Notice that I did not change anything
related
On 9/18/16 10:38 PM, Wang Xiaoguang wrote:
> hi,
>
> On 09/14/2016 10:25 PM, Jeff Mahoney wrote:
>> On 9/13/16 10:24 PM, Josef Bacik wrote:
>>> On 09/08/2016 07:02 PM, Jeff Mahoney wrote:
On 9/8/16 2:49 PM, Jeff Mahoney wrote:
> On 9/8/16 2:24 PM, Ronan Arraes Jardim Chagas wrote:
>>
On 09/22/2016 07:48 AM, Jan Kara wrote:
On Tue 20-09-16 16:57:48, Josef Bacik wrote:
Now that we have metadata counters in the VM, we need to provide a way to kick
writeback on dirty metadata. Introduce super_operations->write_metadata. This
allows file systems to deal with writing back any di
On 09/22/2016 07:34 AM, Jan Kara wrote:
On Tue 20-09-16 16:57:47, Josef Bacik wrote:
These are counters that constantly go up in order to do bandwidth calculations.
It isn't important what the units are in, as long as they are consistent between
the two of them, so convert them to count bytes wr
On 09/22/2016 07:18 AM, Jan Kara wrote:
On Tue 20-09-16 16:57:46, Josef Bacik wrote:
Btrfs has no bounds except memory on the amount of dirty memory that we have in
use for metadata. Historically we have used a special inode so we could take
advantage of the balance_dirty_pages throttling that
Guys,
Something very strange happened. I have not seen the problem since
Monday, which is pretty much the first time ever I work more than 3
days without seeing it.
Ok, it can be a coincidence. Notice that I did not change anything
related to my work behavior. However, I did do two things:
_ Upd
Folks ... it's me again :)
Just a preliminary word of warning - I did backup a data before doing
that, so I can rebuild array back ... just want to present this
problem to you because it's a interesting problem / use case issue /
bug. So please no bitching because I don't bitch either !
So, I got
On 09/22/16 14:18, Rich Freeman wrote:
> I have been getting panics consistently after doing a btrfs replace
> operation on a raid1 and rebooting. I linked a photo of the panic; I
> haven't been able to get a text capture of it.
>
> https://ibin.co/2vx0HhDeViu3.jpg
>
> I'm getting this error on
I have been getting panics consistently after doing a btrfs replace
operation on a raid1 and rebooting. I linked a photo of the panic; I
haven't been able to get a text capture of it.
https://ibin.co/2vx0HhDeViu3.jpg
I'm getting this error on the latest 4.4, 4.1, and even on an old
3.18.26 kerne
On Tue 20-09-16 16:57:48, Josef Bacik wrote:
> Now that we have metadata counters in the VM, we need to provide a way to kick
> writeback on dirty metadata. Introduce super_operations->write_metadata.
> This
> allows file systems to deal with writing back any dirty metadata we need based
> on th
On Tue 20-09-16 16:57:47, Josef Bacik wrote:
> These are counters that constantly go up in order to do bandwidth
> calculations.
> It isn't important what the units are in, as long as they are consistent
> between
> the two of them, so convert them to count bytes written/dirtied, and allow the
>
On Tue 20-09-16 16:57:46, Josef Bacik wrote:
> Btrfs has no bounds except memory on the amount of dirty memory that we have
> in
> use for metadata. Historically we have used a special inode so we could take
> advantage of the balance_dirty_pages throttling that comes with using
> pagecache.
> H
Hi,
On 09/22/2016 10:52 AM, David Sterba wrote:
> On Wed, Sep 21, 2016 at 01:31:52PM -0700, Omar Sandoval wrote:
I'm not sure I understand - can you explain why this is was so wrong?
Or Omar maybe?
If btrfsck wants to correct something (write), it can simply always
and unc
Since commit b02441999efcc6152b87cd58e7970bb7843f76cf, we don't wait all
ordered extents, but I run into some enospc errors when doing large file
create and delete test, it's because shrink_delalloc() does not write
enough delalloc bytes and wait them finished:
From: Miao Xie
Date: Mon, 4
On Wed, Sep 21, 2016 at 01:31:52PM -0700, Omar Sandoval wrote:
> > > I'm not sure I understand - can you explain why this is was so wrong?
> > > Or Omar maybe?
> > >
> > > If btrfsck wants to correct something (write), it can simply always
> > > and unconditionally invalidate the fst instead of tr
hi,
On 09/22/2016 03:14 PM, Eryu Guan wrote:
On Thu, Sep 22, 2016 at 10:13:08AM +0800, Wang Xiaoguang wrote:
In btrfs, there is a bug about btrfs' truncate codes, it'll leak
some fs space as the truncate operation proceeds. If this truncate
operation is very large, later metadata request in thi
hi,
On 09/22/2016 02:18 PM, Darrick J. Wong wrote:
On Thu, Sep 22, 2016 at 10:13:08AM +0800, Wang Xiaoguang wrote:
In btrfs, there is a bug about btrfs' truncate codes, it'll leak
some fs space as the truncate operation proceeds. If this truncate
operation is very large, later metadata request
On Thu, Sep 22, 2016 at 10:13:08AM +0800, Wang Xiaoguang wrote:
> In btrfs, there is a bug about btrfs' truncate codes, it'll leak
> some fs space as the truncate operation proceeds. If this truncate
> operation is very large, later metadata request in this truncate
> operation may fail for enospc
71 matches
Mail list logo
