Re: No space left on device (28)
Hi,
output here:
[ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.548280] space_info 4 has 6439292928 free, is full
[ 590.548283] space_info total=25748307968, used=19308916736, pinned=0,
reserved=32768, may_use=6438354944, readonly=65536
[ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.552264] space_info 4 has 6439284736 free, is full
[ 590.552267] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.556258] space_info 4 has 6439284736 free, is full
[ 590.556261] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 591.147373] space_info 4 has 6439235584 free, is full
[ 591.147375] space_info total=25748307968, used=19308916736, pinned=0,
reserved=90112, may_use=6438354944, readonly=65536
[ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 595.562390] space_info 4 has 6439120896 free, is full
[ 595.562393] space_info total=25748307968, used=19309047808, pinned=0,
reserved=73728, may_use=6438297600, readonly=65536
Am 25.03.2013 21:14, schrieb Josef Bacik:
On Fri, Mar 22, 2013 at 02:55:07PM -0600, Stefan Priebe wrote:
Hi Jsoef,
thanks!
Ok I don't think the thing I just fixed will make any difference for you so
here's another debug patch, just apply it on top of what I've already sent you
and re-run and give me the dmesg again. Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index aabaea6..bf6433f 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4162,7 +4162,11 @@ out:
}
if (flushing) {
if (ret == -ENOSPC) {
- printk(KERN_ERR returning enospc, dumping space
info\n);
+ spin_lock(block_rsv-lock);
+ printk(KERN_ERR returning enospc, space_info %u, size %Lu
reserved %Lu,
+ flush %d, flush_state %d dumping space info\n,
block_rsv-type,
+ block_rsv-size, block_rsv-reserved, flush,
flush_state);
+ spin_unlock(block_rsv-lock);
dump_space_info(space_info, 0, 0);
}
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Kernel BUG on corrupted space cache under 3.7 and 3.8
Dear list members, Yesterday I was about to restart my computer and that's why I closed every opened applications: three Eclipse instances, two other Java applications, 2 web browsers etc. It involved a lot of IO operations and before everything could settle down my system became unresponsive. I mean X co. worked fine, but anything related to IO (opening a new screen, a new application etc.) hung. My process explorer showed that nearly all of the applications which were shutting down have very high IO, but in fact there were no HDD head movement at all. I did a hard reset. Today morning when I restarted Linux I got a kernel BUG from btrfs. I successfully started a recovery Linux (with kernel 3.7) and when I tried to mount this partition, the bug/panic appeared again. After I could rebuild the space cache with clear_cache option and my system was fixed. Since all this logs are written only onto screen, I made some photographs :), and I uploaded them. System parameters: - openSUSE 12.3, Kernel 3.8, x86_64 - Linux linux-suse 3.8.0-5-desktop #1 SMP PREEMPT Tue Feb 26 20:55:05 UTC 2013 (db497a6) x86_64 x86_64 x86_64 GNU/Linux - Photos are made on Kernel 3.7 - Default mount options: compress=lzo,space_cache - On 3.7, the problematic line: fs/btrfs/free-space-cache.c:1553 - Photos location: www.morrohun.hu/temp/btrfs/[123].jpeg If you need any other information, ask me :). Best regards, Ákos Szőts -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/7] Btrfs: add a necessary check before creating/destroying a qgroup
From: Wang Shilong wangsl-f...@cn.fujitsu.com
This patch try to add a check before creating/destroying a qgroup.
Signed-off-by: Wang Shilong wangsl-f...@cn.fujitsu.com
Reviewed-by: Miao Xie mi...@cn.fujitsu.com
---
fs/btrfs/ioctl.c | 41 -
1 files changed, 40 insertions(+), 1 deletions(-)
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 3c4fde0..8a01326 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -3784,6 +3784,42 @@ drop_write:
return ret;
}
+static int may_qgroup_create(struct btrfs_root *root,
+struct btrfs_ioctl_qgroup_create_args *sa)
+{
+ int ret = 0;
+ struct btrfs_key key;
+ struct btrfs_path *path;
+
+ if (!root-fs_info-quota_root)
+ return -EINVAL;
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ ret = -ENOMEM;
+ return ret;
+ }
+
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_INFO_KEY;
+ key.offset = sa-qgroupid;
+
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (sa-create) {
+ if (!ret)
+ ret = -EEXIST;
+ if (ret 0)
+ ret = 0;
+ goto out;
+ }
+ if (ret 0)
+ ret = -ENOENT;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
static long btrfs_ioctl_qgroup_create(struct file *file, void __user *arg)
{
struct btrfs_root *root = BTRFS_I(file_inode(file))-root;
@@ -3810,13 +3846,16 @@ static long btrfs_ioctl_qgroup_create(struct file
*file, void __user *arg)
goto out;
}
mutex_lock(root-fs_info-quota_lock);
+ ret = may_qgroup_create(root, sa);
+ if (ret)
+ goto out_unlock;
+
trans = btrfs_join_transaction(root);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
goto out_unlock;
}
- /* FIXME: check if the IDs really exist */
if (sa-create) {
ret = btrfs_create_qgroup(trans, root-fs_info, sa-qgroupid,
NULL);
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/7] Btrfs: add a necessary check before assigning/removing qgroup relations
From: Wang Shilong wangsl-f...@cn.fujitsu.com
This is one of most important reason why we introduce a mutex here,
the previous code dosen't check 'src' and 'dst' before assigning/removing
a qgroup relations. Without the mutex lock, and checks may leads
a race conditon.
Signed-off-by: Wang Shilong wangsl-f...@cn.fujitsu.com
Reviewed-by: Miao Xie mi...@cn.fujitsu.com
---
fs/btrfs/ioctl.c | 81 +-
1 files changed, 80 insertions(+), 1 deletions(-)
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 8a01326..1cd96f6 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -3735,6 +3735,82 @@ drop_write:
return ret;
}
+static int may_qgroup_assign(struct btrfs_root *root,
+struct btrfs_ioctl_qgroup_assign_args *sa)
+{
+ int ret = 0;
+ struct btrfs_key key;
+ struct btrfs_path *path;
+
+ if (!root-fs_info-quota_root)
+ return -EINVAL;
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ ret = -ENOMEM;
+ return ret;
+ }
+
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_INFO_KEY;
+ key.offset = sa-src;
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+
+ btrfs_release_path(path);
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_INFO_KEY;
+ key.offset = sa-dst;
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+
+ btrfs_release_path(path);
+ key.objectid = sa-src;
+ key.type = BTRFS_QGROUP_RELATION_KEY;
+ key.offset = sa-dst;
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (sa-assign) {
+ if (!ret)
+ ret = -EEXIST;
+ if (ret 0)
+ goto out;
+ } else {
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+ }
+
+ btrfs_release_path(path);
+ key.objectid = sa-dst;
+ key.type = BTRFS_QGROUP_RELATION_KEY;
+ key.offset = sa-src;
+ if (sa-assign) {
+ if (!ret)
+ ret = -EEXIST;
+ if (ret 0)
+ goto out;
+ } else {
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+ }
+ ret = 0;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
static long btrfs_ioctl_qgroup_assign(struct file *file, void __user *arg)
{
struct btrfs_root *root = BTRFS_I(file_inode(file))-root;
@@ -3757,13 +3833,16 @@ static long btrfs_ioctl_qgroup_assign(struct file
*file, void __user *arg)
}
mutex_lock(root-fs_info-quota_lock);
+ ret = may_qgroup_assign(root, sa);
+ if (ret)
+ goto out;
+
trans = btrfs_join_transaction(root);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
goto out;
}
- /* FIXME: check if the IDs really exist */
if (sa-assign) {
ret = btrfs_add_qgroup_relation(trans, root-fs_info,
sa-src, sa-dst);
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/7] Btrfs: fix missing check before btrfs_qgroup_inherit()
From: Wang Shilong wangsl-f...@cn.fujitsu.com
The original code dosen't have necessary checks before
doing qgroup_inherit. Fix it.
Signed-off-by: Wang Shilong wangsl-f...@cn.fujitsu.com
Reviewed-by: Miao Xie mi...@cn.fujitsu.com
---
fs/btrfs/ioctl.c | 50 ++
1 files changed, 50 insertions(+), 0 deletions(-)
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 1cd96f6..86e7fb3 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -708,6 +708,49 @@ static inline int btrfs_may_create(struct inode *dir,
struct dentry *child)
return inode_permission(dir, MAY_WRITE | MAY_EXEC);
}
+static int check_qgroup_inherit(struct btrfs_root *root,
+ struct btrfs_qgroup_inherit *inherit)
+{
+ int ret = 0;
+ u64 *i_qgroups;
+ u64 nums = 0;
+ u64 i = 0;
+ struct btrfs_path *path;
+ struct btrfs_key key;
+
+ if (!inherit)
+ return 0;
+ if (!root-fs_info-quota_root)
+ return -EINVAL;
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ ret = -ENOMEM;
+ return ret;
+ }
+
+ i_qgroups = (u64 *)(inherit + 1);
+ nums = inherit-num_qgroups + inherit-num_ref_copies +
+ inherit-num_excl_copies;
+ for (i = 0; i nums; ++i) {
+ btrfs_release_path(path);
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_INFO_KEY;
+ key.offset = *i_qgroups;
+
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (ret 0)
+ ret = -EINVAL;
+ if (ret)
+ goto out;
+ ++i_qgroups;
+ }
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
/*
* Create a new subvolume below @parent. This is largely modeled after
* sys_mkdirat and vfs_mkdir, but we only do a single component lookup
@@ -753,6 +796,11 @@ static noinline int btrfs_mksubvol(struct path *parent,
if (btrfs_root_refs(BTRFS_I(dir)-root-root_item) == 0)
goto out_up_read;
+ mutex_lock(BTRFS_I(dir)-root-fs_info-quota_lock);
+ error = check_qgroup_inherit(BTRFS_I(dir)-root, inherit);
+ if (error)
+ goto out_unlock_mutex;
+
if (snap_src) {
error = create_snapshot(snap_src, dir, dentry, name, namelen,
async_transid, readonly, inherit);
@@ -762,6 +810,8 @@ static noinline int btrfs_mksubvol(struct path *parent,
}
if (!error)
fsnotify_mkdir(dir, dentry);
+out_unlock_mutex:
+ mutex_unlock(BTRFS_I(dir)-root-fs_info-quota_lock);
out_up_read:
up_read(BTRFS_I(dir)-root-fs_info-subvol_sem);
out_dput:
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 6/7] Btrfs: remove unnecessary spin_lock holding for find_qgroup_rb()
From: Wang Shilong wangsl-f...@cn.fujitsu.com
Since mutex has been used for quota operations, we don't have to
hold spin_lock when calling find_qgroup_rb.
Signed-off-by: Wang Shilong wangsl-f...@cn.fujitsu.com
Reviewed-by: Miao Xie mi...@cn.fujitsu.com
---
fs/btrfs/qgroup.c | 15 ++-
1 files changed, 6 insertions(+), 9 deletions(-)
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index e3598fa..e123d5c 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -98,7 +98,10 @@ struct btrfs_qgroup_list {
struct btrfs_qgroup *member;
};
-/* must be called with qgroup_lock held */
+/*
+ * don't need to be held by spin_lock, because it has been protected by
+ * quota_lock
+ */
static struct btrfs_qgroup *find_qgroup_rb(struct btrfs_fs_info *fs_info,
u64 qgroupid)
{
@@ -1041,15 +1044,12 @@ int btrfs_remove_qgroup(struct btrfs_trans_handle
*trans,
return -EINVAL;
/* check if there are no relations to this qgroup */
- spin_lock(fs_info-qgroup_lock);
qgroup = find_qgroup_rb(fs_info, qgroupid);
if (qgroup) {
if (!list_empty(qgroup-groups) ||
!list_empty(qgroup-members)) {
- spin_unlock(fs_info-qgroup_lock);
return -EBUSY;
}
}
- spin_unlock(fs_info-qgroup_lock);
ret = del_qgroup_item(trans, quota_root, qgroupid);
@@ -1081,22 +1081,19 @@ int btrfs_limit_qgroup(struct btrfs_trans_handle *trans,
(unsigned long long)qgroupid);
}
- spin_lock(fs_info-qgroup_lock);
-
qgroup = find_qgroup_rb(fs_info, qgroupid);
if (!qgroup) {
ret = -ENOENT;
- goto unlock;
+ return ret;
}
+ spin_lock(fs_info-qgroup_lock);
qgroup-lim_flags = limit-flags;
qgroup-max_rfer = limit-max_rfer;
qgroup-max_excl = limit-max_excl;
qgroup-rsv_rfer = limit-rsv_rfer;
qgroup-rsv_excl = limit-rsv_excl;
-unlock:
spin_unlock(fs_info-qgroup_lock);
-
return ret;
}
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 7/7] Btrfs: remove reduplicate check about quota_root in qgroup.c
From: Wang Shilong wangsl-f...@cn.fujitsu.com The check has been done just before starting/joining a transaction, so we don't need to check it again. Signed-off-by: Wang Shilong wangsl-f...@cn.fujitsu.com Reviewed-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/qgroup.c | 20 1 files changed, 0 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index e123d5c..6fb67df 100644 --- a/fs/btrfs/qgroup.c +++ b/fs/btrfs/qgroup.c @@ -927,9 +927,6 @@ int btrfs_quota_disable(struct btrfs_trans_handle *trans, btrfs_free_qgroup_config(fs_info); spin_unlock(fs_info-qgroup_lock); - if (!quota_root) - return -EINVAL; - ret = btrfs_clean_quota_tree(trans, quota_root); if (ret) goto out; @@ -965,9 +962,6 @@ int btrfs_add_qgroup_relation(struct btrfs_trans_handle *trans, int ret = 0; quota_root = fs_info-quota_root; - if (!quota_root) - return -EINVAL; - ret = add_qgroup_relation_item(trans, quota_root, src, dst); if (ret) return ret; @@ -993,9 +987,6 @@ int btrfs_del_qgroup_relation(struct btrfs_trans_handle *trans, int err; quota_root = fs_info-quota_root; - if (!quota_root) - return -EINVAL; - ret = del_qgroup_relation_item(trans, quota_root, src, dst); err = del_qgroup_relation_item(trans, quota_root, dst, src); if (err !ret) @@ -1017,9 +1008,6 @@ int btrfs_create_qgroup(struct btrfs_trans_handle *trans, int ret = 0; quota_root = fs_info-quota_root; - if (!quota_root) - return -EINVAL; - ret = add_qgroup_item(trans, quota_root, qgroupid); spin_lock(fs_info-qgroup_lock); @@ -1040,8 +1028,6 @@ int btrfs_remove_qgroup(struct btrfs_trans_handle *trans, int ret = 0; quota_root = fs_info-quota_root; - if (!quota_root) - return -EINVAL; /* check if there are no relations to this qgroup */ qgroup = find_qgroup_rb(fs_info, qgroupid); @@ -1068,9 +1054,6 @@ int btrfs_limit_qgroup(struct btrfs_trans_handle *trans, struct btrfs_qgroup *qgroup; int ret = 0; - if (!quota_root) - return -EINVAL; - ret = update_qgroup_limit_item(trans, quota_root, qgroupid, limit-flags, limit-max_rfer, limit-max_excl, limit-rsv_rfer, @@ -1394,9 +1377,6 @@ int btrfs_qgroup_inherit(struct btrfs_trans_handle *trans, if (!fs_info-quota_enabled) return 0; - if (!quota_root) - return -EINVAL; - /* * create a tracking group for the subvol itself */ -- 1.7.7.6 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 4/7] Btrfs: add a necessary check before assigning/removing qgroup relations
Hello Arne,
I have sent a patch-set about btrfs quota,
would you please review them for me. Any comment is welcome.
Thanks,
Wang
From: Wang Shilong wangsl-f...@cn.fujitsu.com
This is one of most important reason why we introduce a mutex here,
the previous code dosen't check 'src' and 'dst' before assigning/removing
a qgroup relations. Without the mutex lock, and checks may leads
a race conditon.
Signed-off-by: Wang Shilong wangsl-f...@cn.fujitsu.com
Reviewed-by: Miao Xie mi...@cn.fujitsu.com
---
fs/btrfs/ioctl.c | 81 +-
1 files changed, 80 insertions(+), 1 deletions(-)
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 8a01326..1cd96f6 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -3735,6 +3735,82 @@ drop_write:
return ret;
}
+static int may_qgroup_assign(struct btrfs_root *root,
+ struct btrfs_ioctl_qgroup_assign_args *sa)
+{
+ int ret = 0;
+ struct btrfs_key key;
+ struct btrfs_path *path;
+
+ if (!root-fs_info-quota_root)
+ return -EINVAL;
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ ret = -ENOMEM;
+ return ret;
+ }
+
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_INFO_KEY;
+ key.offset = sa-src;
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+
+ btrfs_release_path(path);
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_INFO_KEY;
+ key.offset = sa-dst;
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+
+ btrfs_release_path(path);
+ key.objectid = sa-src;
+ key.type = BTRFS_QGROUP_RELATION_KEY;
+ key.offset = sa-dst;
+ ret = btrfs_search_slot(NULL, root-fs_info-quota_root,
+ key, path, 0, 0);
+ if (sa-assign) {
+ if (!ret)
+ ret = -EEXIST;
+ if (ret 0)
+ goto out;
+ } else {
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+ }
+
+ btrfs_release_path(path);
+ key.objectid = sa-dst;
+ key.type = BTRFS_QGROUP_RELATION_KEY;
+ key.offset = sa-src;
+ if (sa-assign) {
+ if (!ret)
+ ret = -EEXIST;
+ if (ret 0)
+ goto out;
+ } else {
+ if (ret 0)
+ ret = -ENOENT;
+ if (ret 0)
+ goto out;
+ }
+ ret = 0;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
static long btrfs_ioctl_qgroup_assign(struct file *file, void __user *arg)
{
struct btrfs_root *root = BTRFS_I(file_inode(file))-root;
@@ -3757,13 +3833,16 @@ static long btrfs_ioctl_qgroup_assign(struct file
*file, void __user *arg)
}
mutex_lock(root-fs_info-quota_lock);
+ ret = may_qgroup_assign(root, sa);
+ if (ret)
+ goto out;
+
trans = btrfs_join_transaction(root);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
goto out;
}
- /* FIXME: check if the IDs really exist */
if (sa-assign) {
ret = btrfs_add_qgroup_relation(trans, root-fs_info,
sa-src, sa-dst);
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 26.03.2013 13:53, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: Hi, output here: [ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.548280] space_info 4 has 6439292928 free, is full [ 590.548283] space_info total=25748307968, used=19308916736, pinned=0, reserved=32768, may_use=6438354944, readonly=65536 [ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.552264] space_info 4 has 6439284736 free, is full [ 590.552267] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.556258] space_info 4 has 6439284736 free, is full [ 590.556261] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 591.147373] space_info 4 has 6439235584 free, is full [ 591.147375] space_info total=25748307968, used=19308916736, pinned=0, reserved=90112, may_use=6438354944, readonly=65536 [ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 595.562390] space_info 4 has 6439120896 free, is full [ 595.562393] space_info total=25748307968, used=19309047808, pinned=0, reserved=73728, may_use=6438297600, readonly=65536 Weird, we have all the flushing stuff set and yet there is still a whole lot of outstanding reservations. Do you have compression enabled? Thanks, no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Greets, Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel BUG on corrupted space cache under 3.7 and 3.8
On Tue, Mar 26, 2013 at 05:37:35AM -0600, Szőts Ákos wrote: Dear list members, Yesterday I was about to restart my computer and that's why I closed every opened applications: three Eclipse instances, two other Java applications, 2 web browsers etc. It involved a lot of IO operations and before everything could settle down my system became unresponsive. I mean X co. worked fine, but anything related to IO (opening a new screen, a new application etc.) hung. My process explorer showed that nearly all of the applications which were shutting down have very high IO, but in fact there were no HDD head movement at all. I did a hard reset. Today morning when I restarted Linux I got a kernel BUG from btrfs. I successfully started a recovery Linux (with kernel 3.7) and when I tried to mount this partition, the bug/panic appeared again. After I could rebuild the space cache with clear_cache option and my system was fixed. Since all this logs are written only onto screen, I made some photographs :), and I uploaded them. System parameters: - openSUSE 12.3, Kernel 3.8, x86_64 - Linux linux-suse 3.8.0-5-desktop #1 SMP PREEMPT Tue Feb 26 20:55:05 UTC 2013 (db497a6) x86_64 x86_64 x86_64 GNU/Linux - Photos are made on Kernel 3.7 - Default mount options: compress=lzo,space_cache - On 3.7, the problematic line: fs/btrfs/free-space-cache.c:1553 - Photos location: www.morrohun.hu/temp/btrfs/[123].jpeg If you need any other information, ask me :). Ok so I've just made a bunch of changes to btrfs-image so I can capture your tree log and free space cache so I can reproduce exactly this issue :). Can you clone this tree git://github.com/josefbacik/btrfs-progs.git and then build and run btrfs-image -c9 /dev/whatever blah.img and upload blah.img somewhere so I can pull it down? If you've already fixed the file system so it mounts then I don't need the image, I need it only if it is still panicing. So if you have already fixed it then still please clone that tree and next time it happens capture the fs image if you could so I can reproduce and fix the problem. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: compression btrfs
On Mon, Mar 25, 2013 at 10:03:20PM -0600, lonat_fr...@163.com wrote: Hi everyone, I have used btrfs as a work partition with compression=zlib. The compression ratio is not satisfied to me. So you probably want compress-force=zlib. With just compress we will bail out of the compression if the compressed pages are larger than the original size, which means if you wrote a particular file and then copmressed it with gzip you'd possibly see different results, but if you do compress-force=zlib then you'll see behavior more like gzip. I tracked my workloads in btrfs. The zlib module (zlib.c) seems work well: write size of each write operation in writepage function can be compressed into about 20%. I suspent the workloads may impact the btrfs behavior. My workloads include really a large number of overwrite operations. I briefly reviewed the code about the space reclaim in btrfs, and found the btrfs kicks the defrag off when the overwritten range is smaller than 16KB, And this is the only method of reclaiming freed extents with compression. Am I right? It's 64k, and what do you mean reclaiming freed extents? The freed extents will be reclaimed once they are completely overwritten. So my question is if btrfs can successfully reclaim the overwritten space when the cleaner thread can not be started, such as in the case that each overwrite operation is larger than 16KB? Not sure what you mean by reclaim. They won't be defragged if the overwrite is above 64k, but if any write is less than 64k then it will defrag the whole file. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: document mount options in Documentation/fs/btrfs.txt
On Sat, Mar 23, 2013 at 01:45:36PM -0500, Eric Sandeen wrote: The mount manpage is maintained in the util-linux pkg, but it's not kernel-version specific, and the util-linux maintainer does not have specific knowledge of all filesytem options. Absolutely true. The util-linux maintainer is not happy that we don't have filesystem specific man pages for many filesystems. The goal is to have in mount.8 only links to another man pages, for example see smart NFS guys: Mount options for nfs and nfs4: See the options section of the nfs(5) man page (nfs-utils package must be installed). I think kernel docs are the right place for the developers to first document these things Definitely. Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG wrote:
Hi Josef,
Am 26.03.2013 13:53, schrieb Josef Bacik:
On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote:
Hi,
output here:
[ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.548280] space_info 4 has 6439292928 free, is full
[ 590.548283] space_info total=25748307968, used=19308916736, pinned=0,
reserved=32768, may_use=6438354944, readonly=65536
[ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.552264] space_info 4 has 6439284736 free, is full
[ 590.552267] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.556258] space_info 4 has 6439284736 free, is full
[ 590.556261] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 591.147373] space_info 4 has 6439235584 free, is full
[ 591.147375] space_info total=25748307968, used=19308916736, pinned=0,
reserved=90112, may_use=6438354944, readonly=65536
[ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 595.562390] space_info 4 has 6439120896 free, is full
[ 595.562393] space_info total=25748307968, used=19309047808, pinned=0,
reserved=73728, may_use=6438297600, readonly=65536
Weird, we have all the flushing stuff set and yet there is still a whole
lot of
outstanding reservations. Do you have compression enabled? Thanks,
no - it's just mounted with mount -o noatime
:~# cat /proc/mounts | grep btrfs
/dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
Ok I think I see what's going on. Can you try this patch and see if it fixes
it? Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index bf6433f..84e8909 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3803,6 +3803,19 @@ static int can_overcommit(struct btrfs_root *root,
return 0;
}
+static int btrfs_try_writeback(struct super_block *sb, unsigned long nr_pages,
+ enum wb_reason reason)
+{
+ if (!writeback_in_progress(sb-s_bdi)
+ down_read_trylock(sb-s_umount)) {
+ writeback_inodes_sb_nr(sb, nr_pages, reason);
+ up_read(sb-s_umount);
+ return 1;
+ }
+
+ return 0;
+}
+
void btrfs_writeback_inodes_sb_nr(struct btrfs_root *root,
unsigned long nr_pages)
{
@@ -3810,8 +3823,7 @@ void btrfs_writeback_inodes_sb_nr(struct btrfs_root *root,
int started;
/* If we can not start writeback, just sync all the delalloc file. */
- started = try_to_writeback_inodes_sb_nr(sb, nr_pages,
- WB_REASON_FS_FREE_SPACE);
+ started = btrfs_try_writeback(sb, nr_pages, WB_REASON_FS_FREE_SPACE);
if (!started) {
/*
* We needn't worry the filesystem going from r/w to r/o though
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 26.03.2013 14:30, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG wrote: Hi Josef, Am 26.03.2013 13:53, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: Hi, output here: [ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.548280] space_info 4 has 6439292928 free, is full [ 590.548283] space_info total=25748307968, used=19308916736, pinned=0, reserved=32768, may_use=6438354944, readonly=65536 [ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.552264] space_info 4 has 6439284736 free, is full [ 590.552267] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.556258] space_info 4 has 6439284736 free, is full [ 590.556261] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 591.147373] space_info 4 has 6439235584 free, is full [ 591.147375] space_info total=25748307968, used=19308916736, pinned=0, reserved=90112, may_use=6438354944, readonly=65536 [ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 595.562390] space_info 4 has 6439120896 free, is full [ 595.562393] space_info total=25748307968, used=19309047808, pinned=0, reserved=73728, may_use=6438297600, readonly=65536 Weird, we have all the flushing stuff set and yet there is still a whole lot of outstanding reservations. Do you have compression enabled? Thanks, no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel BUG on corrupted space cache under 3.7 and 3.8
I fixed my file system here, so I can't provide more details now. Anyway, I cloned your repository and successfully compiled the tree (except for the mkfs program). But when I try to issue the command ./btrfs-image -c9 /dev/sda7 /home/linux/image.img I get the following: Could not open /dev/sda7 Open ctree failed I tried when it was unmounted and also when was mounted. An other thing happened in the meantime, but I write it to an other thread, because that's an other bug :). Ákos -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Kernel bug on mismatching generation_v2 in inode.c:835
Dear list members, In my previous thread at http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg2.html there was a space_cache kernel bug/panic on kernel 3.8. I could successfully fix that with rebuilding the cache. But some files were missing/corrupted. So I booted a rescue CD with kernel 3.7 and ran btrfsck --repair, which repaired quite a few things. After a reboot I got the following message: [ 469.457386] btrfs: disk space caching is enabled [ 469.503612] btrfs: mismatching generation and generation_v2 found in root item. This root was probably mounted with an older kernel. Resetting all new fields. As soon as anything had wanted to read a bit from the file system, the hard drive went crazy and was working for 5-10 minutes. After I got a kernel panic which said there's an error in fs/btrfs/inode.c:835. In the moment I don't just mount, but want to read something from the mounted file system under the rescue system, the same procedure happens. I made some pictures of it (since I cannot read anything from the logs, if there are any). You can find them here: www.morrohun.hu/temp/btrfs/v2/[123].jpg I wanted to create an image with the aforementined btrfs-image tool, but yet to have any success . Could you please give me an advice what can I do now? Living on a live-CD is not a life insurance :) Best regards, Ákos Szőts -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 07:49:16AM -0600, Stefan Priebe - Profihost AG wrote:
Hi Josef,
Am 26.03.2013 14:30, schrieb Josef Bacik:
On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG
wrote:
Hi Josef,
Am 26.03.2013 13:53, schrieb Josef Bacik:
On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote:
Hi,
output here:
[ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.548280] space_info 4 has 6439292928 free, is full
[ 590.548283] space_info total=25748307968, used=19308916736, pinned=0,
reserved=32768, may_use=6438354944, readonly=65536
[ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.552264] space_info 4 has 6439284736 free, is full
[ 590.552267] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.556258] space_info 4 has 6439284736 free, is full
[ 590.556261] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 591.147373] space_info 4 has 6439235584 free, is full
[ 591.147375] space_info total=25748307968, used=19308916736, pinned=0,
reserved=90112, may_use=6438354944, readonly=65536
[ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 595.562390] space_info 4 has 6439120896 free, is full
[ 595.562393] space_info total=25748307968, used=19309047808, pinned=0,
reserved=73728, may_use=6438297600, readonly=65536
Weird, we have all the flushing stuff set and yet there is still a whole
lot of
outstanding reservations. Do you have compression enabled? Thanks,
no - it's just mounted with mount -o noatime
:~# cat /proc/mounts | grep btrfs
/dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
Ok I think I see what's going on. Can you try this patch and see if it
fixes
it? Thanks,
It still does not fix the problem.
The rsync output looks like this so it does not work for file a but then
continues on c d e, ...
sync -av --progress /backup/ /mnt/
sending incremental file list
.etc_openvpn/ipp.txt
229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196)
.etc_openvpn/openvpn-status.log
360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196)
rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX -
.etc_openvpn/ipp.txt: No space left on device (28)
.log/
.log/UcliEvt.log
104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700)
.log/auth.log
15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700)
.log/auth.log.1
19431424 61%7.35MB/s0:00:01
the dmesg output looks like this:
[ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 551.323694] space_info 4 has 6439526400 free, is full
[ 551.323696] space_info total=25748307968, used=19308666880, pinned=0,
reserved=49152, may_use=6438453248, readonly=65536
Ok so then this is probably it, let me know if it helps. Thanks,
Josef
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index dc08d77..005c45d 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -557,6 +557,7 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root,
int delay_iput)
INIT_LIST_HEAD(splice);
INIT_LIST_HEAD(works);
+ mutex_lock(root-fs_info-ordered_operations_mutex);
spin_lock(root-fs_info-ordered_extent_lock);
list_splice_init(root-fs_info-ordered_extents, splice);
while (!list_empty(splice)) {
@@ -600,6 +601,7 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root,
int delay_iput)
cond_resched();
}
+ mutex_unlock(root-fs_info-ordered_operations_mutex);
}
/*
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug on mismatching generation_v2 in inode.c:835
On Tue, Mar 26, 2013 at 08:33:27AM -0600, Szőts Ákos wrote:
Dear list members,
In my previous thread at
http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg2.html
there was a space_cache kernel bug/panic on kernel 3.8. I could
successfully fix that with rebuilding the cache. But some files were
missing/corrupted. So I booted a rescue CD with kernel 3.7 and ran
btrfsck --repair, which repaired quite a few things.
After a reboot I got the following message:
[ 469.457386] btrfs: disk space caching is enabled
[ 469.503612] btrfs: mismatching generation and generation_v2 found
in root item. This root was probably mounted with an older kernel.
Resetting all new fields.
As soon as anything had wanted to read a bit from the file system, the
hard drive went crazy and was working for 5-10 minutes. After I got a
kernel panic which said there's an error in fs/btrfs/inode.c:835.
In the moment I don't just mount, but want to read something from the
mounted file system under the rescue system, the same procedure
happens.
I made some pictures of it (since I cannot read anything from the
logs, if there are any).
You can find them here: www.morrohun.hu/temp/btrfs/v2/[123].jpg
I wanted to create an image with the aforementined btrfs-image tool,
but yet to have any success .
Could you please give me an advice what can I do now? Living on a
live-CD is not a life insurance :)
Best regards,
Ok I'd like to get btrfs-image working so I can run fsck on it here locally and
see what's wrong. Can you apply this patch to my tree and rebuild and run
btrfs-image again, it should tell us why it's having trouble opening the device.
It also fixes that slight mkfs not compiling problem ;). Thanks,
Josef
diff --git a/disk-io.c b/disk-io.c
index 72b33da..6d879c5 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -1047,7 +1047,8 @@ struct btrfs_fs_info *open_ctree_fs_info(const char
*filename,
fp = open(filename, flags, 0600);
if (fp 0) {
- fprintf (stderr, Could not open %s\n, filename);
+ fprintf (stderr, Could not open %s, %d (%s)\n, filename,
+errno, strerror(errno));
return NULL;
}
info = __open_ctree_fd(fp, filename, sb_bytenr, 0, writes, partial);
diff --git a/mkfs.c b/mkfs.c
index bc68350..003a8fa 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -296,7 +296,7 @@ static int create_raid_groups(struct btrfs_trans_handle
*trans,
if (!mixed) {
u64 total_bytes =
- btrfs_super_total_bytes(root-fs_info-super_copy);
+ btrfs_super_total_bytes(root-fs_info-super_copy);
u64 alloced_bytes = 0;
u64 alloc_flags = BTRFS_BLOCK_GROUP_ENOSPC |
(allowed metadata_profile);
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Then this again: rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.vhost_net.mod.GrhWet - .software/kernel/linux-3.9-rc3/.tmp_versions/vhost_net.mod: No space left on device (28) rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.via-rhine.mod.X2Ofhz - .software/kernel/linux-3.9-rc3/.tmp_versions/via-rhine.mod: No space left on device (28) rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.via-rng.mod.RAyxkF - .software/kernel/linux-3.9-rc3/.tmp_versions/via-rng.mod: No space left on device (28) rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.virtio.mod.w7INoL - .software/kernel/linux-3.9-rc3/.tmp_versions/virtio.mod: No space left on device (28)^C [ 5012.468988] space_info total=25748307968, used=19308773376, pinned=0, reserved=69632, may_use=6438354944, readonly=65536 [ 5012.472981] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 5012.472982] space_info 4 has 6439399424 free, is full [ 5012.472982] space_info total=25748307968, used=19308773376, pinned=0, reserved=69632, may_use=6438354944, readonly=65536 [ 5012.476974] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 5012.476975] space_info 4 has 6439399424 free, is full [ 5012.476976] space_info total=25748307968, used=19308773376, pinned=0, reserved=69632, may_use=6438354944, readonly=65536 [ 5012.480968] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 5012.480969] space_info 4 has 6439399424 free, is full Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[bug] mount and /proc/mounts disagrees
3.8.0+ #3 This happened after 'umount /btrfs' was interrupted by ctl-C # mount | egrep btrfs /dev/mapper/mpathe on /btrfs type btrfs (rw,degraded) # cat /etc/mtab | egrep btrfs /dev/mapper/mpathe /btrfs btrfs rw,degraded 0 0 # cat /proc/mounts | egrep btrfs # umount /btrfs umount: /btrfs: not mounted # -Anand -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote:
Hi,
Am 26.03.2013 15:44, schrieb Josef Bacik:
Am 26.03.2013 13:53, schrieb Josef Bacik:
no - it's just mounted with mount -o noatime
:~# cat /proc/mounts | grep btrfs
/dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
Ok I think I see what's going on. Can you try this patch and see if it
fixes
it? Thanks,
It still does not fix the problem.
The rsync output looks like this so it does not work for file a but then
continues on c d e, ...
sync -av --progress /backup/ /mnt/
sending incremental file list
.etc_openvpn/ipp.txt
229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196)
.etc_openvpn/openvpn-status.log
360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196)
rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX -
.etc_openvpn/ipp.txt: No space left on device (28)
.log/
.log/UcliEvt.log
104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700)
.log/auth.log
15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700)
.log/auth.log.1
19431424 61%7.35MB/s0:00:01
the dmesg output looks like this:
[ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 551.323694] space_info 4 has 6439526400 free, is full
[ 551.323696] space_info total=25748307968, used=19308666880, pinned=0,
reserved=49152, may_use=6438453248, readonly=65536
Ok so then this is probably it, let me know if it helps. Thanks,
OK it now has copied a lot of files (170) without an error all were very
small.
Welp progress is good. Throw this into the mix and go again, it's just adding
some more debugging so I can make sure I'm going down the right rabbit hole.
Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 84e8909..1cf810a 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4026,6 +4026,15 @@ static int flush_space(struct btrfs_root *root,
return ret;
}
+
+static void dump_block_rsv(struct btrfs_block_rsv *block_rsv)
+{
+ spin_lock(block_rsv-lock);
+ printk(KERN_ERR dumping block rsv %u, size %Lu reserved %Lu\n,
+ block_rsv-type, block_rsv-size, block_rsv-reserved);
+ spin_unlock(block_rsv-lock);
+}
+
/**
* reserve_metadata_bytes - try to reserve bytes from the block_rsv's space
* @root - the root we're allocating for
@@ -4179,6 +4188,9 @@ out:
flush %d, flush_state %d dumping space
info\n, block_rsv-type,
block_rsv-size, block_rsv-reserved, flush,
flush_state);
spin_unlock(block_rsv-lock);
+ dump_block_rsv(root-fs_info-delalloc_block_rsv);
+ dump_block_rsv(root-fs_info-delayed_block_rsv);
+ dump_block_rsv(root-fs_info-global_block_rsv);
dump_space_info(space_info, 0, 0);
}
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [bug] mount and /proc/mounts disagrees
On Tue, Mar 26, 2013 at 09:24:02AM -0600, anand jain wrote: 3.8.0+ #3 This happened after 'umount /btrfs' was interrupted by ctl-C # mount | egrep btrfs /dev/mapper/mpathe on /btrfs type btrfs (rw,degraded) # cat /etc/mtab | egrep btrfs /dev/mapper/mpathe /btrfs btrfs rw,degraded 0 0 # cat /proc/mounts | egrep btrfs # umount /btrfs umount: /btrfs: not mounted # That's because your mount still uses /etc/mtab for some reason, I thought it had stopped doing that and was using /proc/mounts? Anyway it's just because the entry made it into mtab, if umount and /proc/mounts says its not there then its not there. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug on mismatching generation_v2 in inode.c:835
The error was just a silly permission denied one. I uploaded the created image to www.morrohun.hu/temp/btrfs/btrfs.img SHA1: 90ee61e0724700495d26678d58dc229e04a04cc4 Please, write me when you downloaded and I'll delete it. I have the idea to run btrfsck from your tree on the file system again, but I won't touch the fs until you said so :). Ákos -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
HI, Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug on mismatching generation_v2 in inode.c:835
On Tue, Mar 26, 2013 at 09:59:57AM -0600, Szőts Ákos wrote: The error was just a silly permission denied one. I uploaded the created image to www.morrohun.hu/temp/btrfs/btrfs.img SHA1: 90ee61e0724700495d26678d58dc229e04a04cc4 Please, write me when you downloaded and I'll delete it. I have the idea to run btrfsck from your tree on the file system again, but I won't touch the fs until you said so :). Ok you can go ahead and delete it, I just pulled it down, I'm restoring it and then I'll see whats going on. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 10:19:19AM -0600, Stefan Priebe wrote: HI, Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Josef diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1cf810a..ac415cf7 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4471,7 +4471,12 @@ static void update_global_block_rsv(struct btrfs_fs_info *fs_info) spin_lock(sinfo-lock); spin_lock(block_rsv-lock); - block_rsv-size = num_bytes; + /* +* Limit the global block rsv to 512mb, we have infrastructure in place +* to throttle reservations if we start getting low on global block rsv +* space. +*/ + block_rsv-size = min_t(u64, num_bytes, 512 * 1024 * 1024); num_bytes = sinfo-bytes_used + sinfo-bytes_pinned + sinfo-bytes_reserved + sinfo-bytes_readonly + -- 1.7.7.6 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Re: compression btrfs
On Tue, Mar 26, 2013 at 10:27:34AM -0600, yiletian wrote: Yes, I use compress-force=zlib for my partition. Consider this scenario. We first write a file with size of 256KB. Assume all data is compressed to 128KB size, btrfs create a extent item in extent-tree to record the 128KB disk range (named E). and btrfs also creates a single file extent to records the disk range of E. Then we overwrite from 16KB to the end of file, with size of 240KB. Btrfs will create a new file extent for the overwritten range. That is, the file has two file extents: the first one is to record the first 16KB and the second one record the remaining 240KB. Then we are in a dilemma: 1. the first one only occupies a disk range of 16KB, but entire E is reserved for it. This is because the __btrfs_drop_exte nts function do not decrease the number of back refs of E. 2. because the overwritten range is large enough, the compress_file_range does not call btrfs_add_inode_defrag to kick off a defrag for the file automatically. With this dilemma, how can btrfs reclaim the 112KB disk range (at least) recorded in E. Oh yeah welcome to btrfs, you must be new here ;). So yeah this is the way it works, until we overwrite the entire extent we don't reclaim any of the space. This includes the prealloc an 8 gig vm image and then random write inside of it workload, you could end up using up to 16gb in the worst case scenario. The thing we could do to fix this would be to instead of splitting the file extents and then inc'ing the ref of the original extent we instead split the extent ref as well, so we can reclaim this space. It's on my list of things to do down the road, but it keeps getting supplanted by other priorities. THanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug on mismatching generation_v2 in inode.c:835
On Tue, Mar 26, 2013 at 09:59:57AM -0600, Szőts Ákos wrote: The error was just a silly permission denied one. I uploaded the created image to www.morrohun.hu/temp/btrfs/btrfs.img SHA1: 90ee61e0724700495d26678d58dc229e04a04cc4 Please, write me when you downloaded and I'll delete it. I have the idea to run btrfsck from your tree on the file system again, but I won't touch the fs until you said so :). You did take this image while the file system was unmounted right? I'm seeing different weirdness with the restored image, trying to figure out if it's a problem with the restore or with the dump. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re:Re: Re: compression btrfs
I think the biggest problem is how we can reclaim the space when the extent is a compressed one. In this case, we may need to read and decompress data in the extent, and then compress the valid range to generate a new extent. Is this process a performance killer? At 2013-03-27 02:03:57,Josef Bacik jba...@fusionio.com wrote: On Tue, Mar 26, 2013 at 10:27:34AM -0600, yiletian wrote: Yes, I use compress-force=zlib for my partition. Consider this scenario. We first write a file with size of 256KB. Assume all data is compressed to 128KB size, btrfs create a extent item in extent-tree to record the 128KB disk range (named E). and btrfs also creates a single file extent to records the disk range of E. Then we overwrite from 16KB to the end of file, with size of 240KB. Btrfs will create a new file extent for the overwritten range. That is, the file has two file extents: the first one is to record the first 16KB and the second one record the remaining 240KB. Then we are in a dilemma: 1. the first one only occupies a disk range of 16KB, but entire E is reserved for it. This is because the __btrfs_drop_exte nts function do not decrease the number of back refs of E. 2. because the overwritten range is large enough, the compress_file_range does not call btrfs_add_inode_defrag to kick off a defrag for the file automatically. With this dilemma, how can btrfs reclaim the 112KB disk range (at least) recorded in E. Oh yeah welcome to btrfs, you must be new here ;). So yeah this is the way it works, until we overwrite the entire extent we don't reclaim any of the space. This includes the prealloc an 8 gig vm image and then random write inside of it workload, you could end up using up to 16gb in the worst case scenario. The thing we could do to fix this would be to instead of splitting the file extents and then inc'ing the ref of the original extent we instead split the extent ref as well, so we can reclaim this space. It's on my list of things to do down the road, but it keeps getting supplanted by other priorities. THanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug on mismatching generation_v2 in inode.c:835
If my memory servers me well, it was taken in unmounted state. Do you want me to take an other one and compare the SHA1 values? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel bug on mismatching generation_v2 in inode.c:835
On Tue, Mar 26, 2013 at 12:25:44PM -0600, Szőts Ákos wrote: If my memory servers me well, it was taken in unmounted state. Do you want me to take an other one and compare the SHA1 values? Nope I found the bug, just a problem with the restore. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 26.03.2013 18:45, schrieb Josef Bacik: Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Yes - this works now. Which of the patches can i drop? Do i just need the last one? Is it safe to add another 18TB raid via converting it to btrfs raid0? Will the fix be part of 3.9-rc5? Thanks and greets, Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 01:05:36PM -0600, Stefan Priebe wrote: Hi Josef, Am 26.03.2013 18:45, schrieb Josef Bacik: Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Yes - this works now. Which of the patches can i drop? Do i just need the last one? Is it safe to add another 18TB raid via converting it to btrfs raid0? Will the fix be part of 3.9-rc5? So I'll put together all of the patches that actually need to go up for this and post them, but basically its the mutex patch, the last patch I sent you and the one that adjusts the reservations for rename and delete. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: fix space accounting for unlink and rename
We are way over-reserving for unlink and rename. Rename is just some random
huge number and unlink accounts for tree log operations that don't actually
happen during unlink, not to mention the tree log doesn't take from the trans
block rsv anyway so it's completely useless. Thanks,
Signed-off-by: Josef Bacik jba...@fusionio.com
---
fs/btrfs/inode.c |6 ++
1 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 106b261..d38c9b3 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -3683,11 +3683,9 @@ static struct btrfs_trans_handle
*__unlink_start_trans(struct inode *dir,
* 1 for the dir item
* 1 for the dir index
* 1 for the inode ref
-* 1 for the inode ref in the tree log
-* 2 for the dir entries in the log
* 1 for the inode
*/
- trans = btrfs_start_transaction(root, 8);
+ trans = btrfs_start_transaction(root, 5);
if (!IS_ERR(trans) || PTR_ERR(trans) != -ENOSPC)
return trans;
@@ -8130,7 +8128,7 @@ static int btrfs_rename(struct inode *old_dir, struct
dentry *old_dentry,
* inodes. So 5 * 2 is 10, plus 1 for the new link, so 11 total items
* should cover the worst case number of items we'll modify.
*/
- trans = btrfs_start_transaction(root, 20);
+ trans = btrfs_start_transaction(root, 11);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
goto out_notrans;
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi, but when i transfer big files i see now this one: [20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds. [20368.821978] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. [20368.895140] rsync D 8160f580 0 14911 1 0x [20368.895148] 8801ca63fc78 0086 8800c28f8198 88022394f800 [20368.895158] 8801ca63ffd8 8801ca63ffd8 8801ca63ffd8 00012c40 [20368.895163] 81a11440 8801c9d36340 8801ca63fc88 8801cefce130 [20368.895169] Call Trace: [20368.895180] [8151a774] schedule+0x24/0x70 [20368.895207] [a0158c75] wait_current_trans.isra.32+0x95/0x100 [btrfs] [20368.895214] [8106d4f0] ? add_wait_queue+0x60/0x60 [20368.895236] [a015a45d] start_transaction.part.33+0x13d/0x4d0 [btrfs] [20368.895252] [811420f3] ? inode_permission+0x13/0x50 [20368.895271] [a015a814] start_transaction+0x24/0x30 [btrfs] [20368.895287] [a015aae3] btrfs_start_transaction+0x13/0x20 [btrfs] [20368.895302] [a015b2f0] __unlink_start_trans+0x70/0x460 [btrfs] [20368.895307] [8150ee3e] ? check_acl+0x5a/0x122 [20368.895312] [81055ff0] ? ns_capable+0x30/0x60 [20368.895317] [811413bd] ? generic_permission+0xbd/0x110 [20368.895336] [a0163f92] btrfs_unlink+0x32/0xc0 [btrfs] [20368.895341] [8114186d] vfs_unlink.part.61+0x6d/0xd0 [20368.895345] [81143ad7] vfs_unlink+0x37/0x50 [20368.895349] [81143c8b] do_unlinkat+0x19b/0x240 [20368.895354] [81146171] sys_unlink+0x11/0x20 [20368.895359] [8151c2e9] system_call_fastpath+0x16/0x1b Speed is just 100kb/s instead of 100MB/s. Stefan Am 26.03.2013 20:16, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:05:36PM -0600, Stefan Priebe wrote: Hi Josef, Am 26.03.2013 18:45, schrieb Josef Bacik: Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Yes - this works now. Which of the patches can i drop? Do i just need the last one? Is it safe to add another 18TB raid via converting it to btrfs
[PATCH] Btrfs: hold the ordered operations mutex when waiting on ordered extents
We need to hold the ordered_operations mutex while waiting on ordered extents
since we splice and run the ordered extents list. We need to make sure anybody
else who wants to wait on ordered extents does actually wait for them to be
completed. This will keep us from bailing out of flushing in case somebody is
already waiting on ordered extents to complete. Thanks,
Signed-off-by: Josef Bacik jba...@fusionio.com
---
fs/btrfs/ordered-data.c |2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index dc08d77..005c45d 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -557,6 +557,7 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root,
int delay_iput)
INIT_LIST_HEAD(splice);
INIT_LIST_HEAD(works);
+ mutex_lock(root-fs_info-ordered_operations_mutex);
spin_lock(root-fs_info-ordered_extent_lock);
list_splice_init(root-fs_info-ordered_extents, splice);
while (!list_empty(splice)) {
@@ -600,6 +601,7 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root,
int delay_iput)
cond_resched();
}
+ mutex_unlock(root-fs_info-ordered_operations_mutex);
}
/*
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: limit the global reserve to 512mb
A user reported a problem where he was getting early ENOSPC with hundreds of gigs of free data space and 6 gigs of free metadata space. This is because the global block reserve was taking up the entire free metadata space. This is ridiculous, we have infrastructure in place to throttle if we start using too much of the global reserve, so instead of letting it get this huge just limit it to 512mb so that users can still get work done. This allowed the user to complete his rsync without issues. Thanks Cc: sta...@vger.kernel.org Reported-and-tested-by: Stefan Priebe s.pri...@profihost.ag Signed-off-by: Josef Bacik jba...@fusionio.com --- fs/btrfs/extent-tree.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c08c7c8..5791da2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4554,7 +4554,7 @@ static void update_global_block_rsv(struct btrfs_fs_info *fs_info) spin_lock(sinfo-lock); spin_lock(block_rsv-lock); - block_rsv-size = num_bytes; + block_rsv-size = min_t(u64, num_bytes, 512 * 1024 * 1024); num_bytes = sinfo-bytes_used + sinfo-bytes_pinned + sinfo-bytes_reserved + sinfo-bytes_readonly + -- 1.7.7.6 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V2] btrfs: document mount options in Documentation/fs/btrfs.txt
Document all current btrfs mount options. Signed-off-by: Eric Sandeen sand...@redhat.com --- V2: * reflect that btrfs is no longer new ;) * make it clear that alloc_start is for each device * highlight potential perf impacts of -o discard * reword skip_balance docs to refer to resume diff --git a/Documentation/filesystems/btrfs.txt b/Documentation/filesystems/btrfs.txt index 7671352..b349d57 100644 --- a/Documentation/filesystems/btrfs.txt +++ b/Documentation/filesystems/btrfs.txt @@ -1,8 +1,8 @@ - BTRFS - = +BTRFS += -Btrfs is a new copy on write filesystem for Linux aimed at +Btrfs is a copy on write filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Initially developed by Oracle, Btrfs is licensed under the GPL and open for contribution from anyone. @@ -34,9 +34,175 @@ The main Btrfs features include: * Online filesystem defragmentation +Mount Options += - MAILING LIST - +When mounting a btrfs filesystem, the following option are accepted. +Unless otherwise specified, all options default to off. + + alloc_start=bytes + Debugging option to force all block allocations above a certain + byte threshold on each block device. The value is specified in + bytes, optionally with a K, M, or G suffix, case insensitive. + Default is 1MB. + + autodefrag + Detect small random writes into files and queue them up for the + defrag process. Works best for small files; Not well suited for + large database workloads. + + check_int + check_int_data + check_int_print_mask=value + These debugging options control the behavior of the integrity checking + module (the BTRFS_FS_CHECK_INTEGRITY config option required). + + check_int enables the integrity checker module, which examines all + block write requests to ensure on-disk consistency, at a large + memory and CPU cost. + + check_int_data includes extent data in the integrity checks, and + implies the check_int option. + + check_int_print_mask takes a bitmask of BTRFSIC_PRINT_MASK_* values + as defined in fs/btrfs/check-integrity.c, to control the integrity + checker module behavior. + + See comments at the top of fs/btrfs/check-integrity.c for more info. + + compress + compress=type + compress-force + compress-force=type + Control BTRFS file data compression. Type may be specified as zlib + lzo or no (for no compression, used for remounting). If no type + is specified, zlib is used. If compress-force is specified, + all files will be compressed, whether or not they compress well. + If compression is enabled, nodatacow and nodatasum are disabled. + + degraded + Allow mounts to continue with missing devices. A read-write mount may + fail with too many devices missing, for example if a stripe member + is completely missing. + + device=devicepath + Specify a device during mount so that ioctls on the control device + can be avoided. Especialy useful when trying to mount a multi-device + setup as root. May be specified multiple times for multiple devices. + + discard + Issue frequent commands to let the block device reclaim space freed by + the filesystem. This is useful for SSD devices, thinly provisioned + LUNs and virtual machine images, but may have a significant + performance impact. (The fstrim command is also available to + initiate batch trims from userspace). + + enospc_debug + Debugging option to be more verbose in some ENOSPC conditions. + + fatal_errors=action + Action to take when encountering a fatal error: + bug - BUG() on a fatal error. This is the default. + panic - panic() on a fatal error. + + flushoncommit + The 'flushoncommit' mount option forces any data dirtied by a write in a + prior transaction to commit as part of the current commit. This makes + the committed state a fully consistent view of the file system from the + application's perspective (i.e., it includes all completed file system + operations). This was previously the behavior only when a snapshot is + created. + + inode_cache + Enable free inode number caching. Defaults to off due to an overflow + problem when the free space crcs don't fit inside a single page. + + max_inline=bytes + Specify the maximum amount of space, in bytes, that can be inlined in + a metadata B-tree leaf. The value is specified in bytes, optionally + with a K, M, or G suffix, case insensitive. In practice, this value + is limited by the root sector size, with some space unavailable due + to leaf headers. For a 4k sectorsize, max inline data is ~3900 bytes. + + metadata_ratio=value + Specify that 1 metadata chunk should
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 01:22:20PM -0600, Stefan Priebe wrote:
Hi,
but when i transfer big files i see now this one:
[20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds.
[20368.821978] echo 0 /proc/sys/kernel/hung_task_timeout_secs
disables this message.
[20368.895140] rsync D 8160f580 0 14911 1
0x
[20368.895148] 8801ca63fc78 0086 8800c28f8198
88022394f800
[20368.895158] 8801ca63ffd8 8801ca63ffd8 8801ca63ffd8
00012c40
[20368.895163] 81a11440 8801c9d36340 8801ca63fc88
8801cefce130
[20368.895169] Call Trace:
[20368.895180] [8151a774] schedule+0x24/0x70
[20368.895207] [a0158c75]
wait_current_trans.isra.32+0x95/0x100 [btrfs]
[20368.895214] [8106d4f0] ? add_wait_queue+0x60/0x60
[20368.895236] [a015a45d]
start_transaction.part.33+0x13d/0x4d0 [btrfs]
[20368.895252] [811420f3] ? inode_permission+0x13/0x50
[20368.895271] [a015a814] start_transaction+0x24/0x30 [btrfs]
[20368.895287] [a015aae3] btrfs_start_transaction+0x13/0x20
[btrfs]
[20368.895302] [a015b2f0] __unlink_start_trans+0x70/0x460 [btrfs]
[20368.895307] [8150ee3e] ? check_acl+0x5a/0x122
[20368.895312] [81055ff0] ? ns_capable+0x30/0x60
[20368.895317] [811413bd] ? generic_permission+0xbd/0x110
[20368.895336] [a0163f92] btrfs_unlink+0x32/0xc0 [btrfs]
[20368.895341] [8114186d] vfs_unlink.part.61+0x6d/0xd0
[20368.895345] [81143ad7] vfs_unlink+0x37/0x50
[20368.895349] [81143c8b] do_unlinkat+0x19b/0x240
[20368.895354] [81146171] sys_unlink+0x11/0x20
[20368.895359] [8151c2e9] system_call_fastpath+0x16/0x1b
Speed is just 100kb/s instead of 100MB/s.
Hrm I wonder if 512 is too small for your case, can you add this patch to the
pile and see what dmesg says when you are having these problems? Thanks,
Josef
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 50767bb..d19c9f6 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -31,6 +31,7 @@
#include inode-map.h
#include volumes.h
#include dev-replace.h
+#include math.h
#define BTRFS_ROOT_TRANS_TAG 0
@@ -576,10 +577,19 @@ void btrfs_throttle(struct btrfs_root *root)
static int should_end_transaction(struct btrfs_trans_handle *trans,
struct btrfs_root *root)
{
- int ret;
+ struct btrfs_block_rsv *block_rsv = root-fs_info-global_block_rsv;
+ u64 num_bytes = 0;
+ int ret = 1;
- ret = btrfs_block_rsv_check(root, root-fs_info-global_block_rsv, 5);
- return ret ? 1 : 0;
+ spin_lock(block_rsv-lock);
+ num_bytes = div_factor(block_rsv-size, 5);
+ if (block_rsv-reserved = num_bytes)
+ ret = 0;
+ else
+ printk(KERN_ERR we're pretty low, setting blocked, reserved
%Lu, size %Lu, num %Lu\n,
+ block_rsv-reserved, block_rsv-size, num_bytes);
+ spin_unlock(block_rsv-lock);
+ return ret;
}
int btrfs_should_end_transaction(struct btrfs_trans_handle *trans,
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
HI, Am 26.03.2013 20:38, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:22:20PM -0600, Stefan Priebe wrote: Hi, but when i transfer big files i see now this one: [20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds. [20368.821978] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. [20368.895140] rsync D 8160f580 0 14911 1 0x [20368.895148] 8801ca63fc78 0086 8800c28f8198 88022394f800 [20368.895158] 8801ca63ffd8 8801ca63ffd8 8801ca63ffd8 00012c40 [20368.895163] 81a11440 8801c9d36340 8801ca63fc88 8801cefce130 [20368.895169] Call Trace: [20368.895180] [8151a774] schedule+0x24/0x70 [20368.895207] [a0158c75] wait_current_trans.isra.32+0x95/0x100 [btrfs] [20368.895214] [8106d4f0] ? add_wait_queue+0x60/0x60 [20368.895236] [a015a45d] start_transaction.part.33+0x13d/0x4d0 [btrfs] [20368.895252] [811420f3] ? inode_permission+0x13/0x50 [20368.895271] [a015a814] start_transaction+0x24/0x30 [btrfs] [20368.895287] [a015aae3] btrfs_start_transaction+0x13/0x20 [btrfs] [20368.895302] [a015b2f0] __unlink_start_trans+0x70/0x460 [btrfs] [20368.895307] [8150ee3e] ? check_acl+0x5a/0x122 [20368.895312] [81055ff0] ? ns_capable+0x30/0x60 [20368.895317] [811413bd] ? generic_permission+0xbd/0x110 [20368.895336] [a0163f92] btrfs_unlink+0x32/0xc0 [btrfs] [20368.895341] [8114186d] vfs_unlink.part.61+0x6d/0xd0 [20368.895345] [81143ad7] vfs_unlink+0x37/0x50 [20368.895349] [81143c8b] do_unlinkat+0x19b/0x240 [20368.895354] [81146171] sys_unlink+0x11/0x20 [20368.895359] [8151c2e9] system_call_fastpath+0x16/0x1b Speed is just 100kb/s instead of 100MB/s. Hrm I wonder if 512 is too small for your case, can you add this patch to the pile and see what dmesg says when you are having these problems? Thanks, No idea why but i can't reproduce... ;-( Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
No space left on device although df reports only 55% in use
Hi, I am using a btrfs loopback mounted file with lzo-compression on Linux-3.7.9, and I ran into No space left on device messages, although df reports only 55% of space is used: # touch testfile touch: cannot touch `testfile': No space left on device # df Filesystem 1K-blocks Used Available Use% Mounted on /dev/loop0 32768000 17383924 14649172 55% /home/ce/anditest # btrfs filesystem df . Data: total=28.22GB, used=14.25GB System, DUP: total=8.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=1.50GB, used=1.16GB Metadata: total=8.00MB, used=0.00 Any ideas what is going wrong here? Thank you in advance, Clemens -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
The -c option of btrfs qgroup limit
All, The btrfs qgroup limit command has an option -c which means limit amount of data after compression. Whether this option is specified or not, I seem to be able to write more well-compressible data than the specified limit. I was expecting that omitting the -c option would never allow me to write more data than specified by the limit. Am I missing something? Which difference in behavior should I expect from the -c option? Koen. --- This is what I did: Mount a btrfs filesystem with the compress option and turn on quota: # mkfs.btrfs -f /dev/sdg2 # mount -o compress /dev/sdg2 /mnt # cd /mnt # btrfs quota enable ./ Create a subvol and limit the amount of data after compression. Write some well-compressible files: # btrfs subvol create subvol1 # btrfs qgroup limit -c 3m ./subvol1 # for i in `seq 1 5`; do dd if=/dev/zero of=subvol1/file$i bs=1024 count=1000; sync; done (no errors) # du -s subvol1 5000 subvol1 We can write 5 MB of data to a file that is limited to 3 MB, which is normal in this case because data from /dev/zero is very good compressible. Create a subvol and limit the amount of data after compression. Write some poorly compressible files: # btrfs subvol create subvol2 # btrfs qgroup limit -c 3m ./subvol2 # for i in `seq 1 5`; do dd if=/dev/urandom of=subvol2/file$i bs=1024 count=1000; sync; done dd: writing `subvol2/file4': Disk quota exceeded dd: opening `subvol2/file5': Disk quota exceeded # du -s subvol2 3056 subvol2 Here we get quota violations, because data from /dev/urandom is poorly compressible. Now write some well-compressible data to a subvol that is limited without the -c option: # btrfs subvol create subvol3 # btrfs qgroup limit 3m ./subvol3 # for i in `seq 1 5`; do dd if=/dev/zero of=subvol3/file$i bs=1024 count=1000; sync; done (no errors) # du -s subvol3 5000 subvol3 We're still able to write 5 MB of data to a subvol that is limited to 3 MB. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC] How to enable btrfs reserve block space from specific device?
HI, When i work on btrfs hot relocation feature, i hit one question about block reservation. btrfs hot relocation need to reserve block space from specific devices such as SSD, but current btrfs reserving code doesn't differentiate if block space is reserved from SSD or HDD. In order to make btrfs support this, I thought that we can introduce one new block group or flag, but this will maybe make large impact on current existing other functions. For this, does any guy have some better idea? thanks. -- Regards, Zhi Yong Wu -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
Bit late - but that could be explored in future. The main downside I see with automatic redundancy/optimisation is the complexity it introduces. Likely this would be best served with user-space tools. On 11/03/13 02:21, Roger Binns wrote: On 10/03/13 15:04, Hugo Mills wrote: Given that this is going to end up rewriting *all* of the data on the FS, Why does all data have to be rewritten? Why does every piece of data have to have exactly the same storage parameters in terms of non-redundancy/performance/striping options? This is a good point. You don't necessarily have to rewrite everything all at once so the performance penalty is not necessarily that bad. More importantly, some restripe operations actually don't need much change on-disk (in theory). Let's say we have disks with 3c chunk allocation and this needs to be reallocated into 2c chunks. In practice at present what would actually happen is that it would first create a *new* 2c chunk and migrate the data over from the 3c chunk. Once the data is moved across we finally mark the space taken up by the original 3c chunk as available for use. Rinse; Repeat. In theory we can skip this rebalance/migration step by thinning out the chunks in-place: relabel the chunk as 2c and mark the unneeded copies as available diskspace. A similar situation applies to other types of conversions in that they could be converted in-place with much less I/O or that the I/O could be optimised (for example sequential I/O between disks with minimal buffering needed vs moving data between two locations on the same disk). I'm sure there are other possibilities for in-place conversions too, such as moving from 4c to 2c2s or 2c to 2s. xC - (x-1)C xCmS - (x/2)C(m*2)S The complexity of the different types of conversions hasn't escaped me, and I do see another downside as well. With the 3C-2C conversion there is the inevitability of macro fragmentation. Again, there could be long-term performance implication or it might even be negligible. -- __ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/03/13 21:27, Brendan Hide wrote: On 11/03/13 02:21, Roger Binns wrote: Why does all data have to be rewritten? Why does every piece of data have to have exactly the same storage parameters in terms of non-redundancy/performance/striping options? This is a good point. You don't necessarily have to rewrite everything all at once so the performance penalty is not necessarily that bad. More importantly, some restripe operations actually don't need much change on-disk (in theory). Note that is not what I was describing. What I meant was that if I put 10GB of data onto 100GB of space that btrfs is free to go above and beyond the minimums, and to do so differently for different pieces of data. For example btrfs could make 6 copies of files beginning with 'a', 10 of files beginning with 'c' and 274 of all others. Obviously that is a bad heuristic, but anything it deems useful for all that unused space is fine by me, and there is absolutely no need for every block to have exactly the same parameters all others. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlFSgqcACgkQmOOfHg372QRIWwCgs/1ou96E5S0d93XEcAnIDvTd f08AoNn6F4zjfQSzXAnkZk4RS4KWZq0b =+kL4 -END PGP SIGNATURE- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
