[PATCH 0/2] send: Avoid sending disknr==0 and PREALLOC extents when possible V2
These two patches address the issue of sending unneeded zero data for disknr==0 and PREALLOC extents. There is room for additional improvement for PREALLOC extents, but it requires adding a new send command, so for now this is not addressed. Changes in V2: no functional change; previous patchset was malformed, so cleanup done with checkpatch.pl. Please review and comment. Alex Lyakas (2): Avoid sending disknr==0 extents when possible. On a diff-send, avoid sending PREALLOC extents, if the parent root has only PREALLOC extents on an appropriate file range. fs/btrfs/send.c | 184 +-- 1 file changed, 178 insertions(+), 6 deletions(-) -- 1.7.9.5 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] V2 Avoid sending disknr==0 extents when possible.
In the following cases we do not send disknr==0 extents:
1) full send 2) new inode in a diff-send 3) when disknr==0 extents are
added to the end of an inode
Original-version-by: Chen Yang chenyang.f...@cn.fujitsu.com
Signed-off-by: Alex Lyakas a...@zadarastorage.com
---
fs/btrfs/send.c | 30 --
1 file changed, 28 insertions(+), 2 deletions(-)
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 5445454..bdef966 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -3844,7 +3844,8 @@ static int is_extent_unchanged(struct send_ctx *sctx,
btrfs_item_key_to_cpu(eb, found_key, slot);
if (found_key.objectid != key.objectid ||
found_key.type != key.type) {
- ret = 0;
+ /* No need to send a no-data extent it in this case */
+ ret = (left_disknr == 0) ? 1 : 0;
goto out;
}
@@ -3870,7 +3871,8 @@ static int is_extent_unchanged(struct send_ctx *sctx,
* This may only happen on the first iteration.
*/
if (found_key.offset + right_len = ekey-offset) {
- ret = 0;
+ /* No need to send a no-data extent it in this case */
+ ret = (left_disknr == 0) ? 1 : 0;
goto out;
}
@@ -3951,6 +3953,30 @@ static int process_extent(struct send_ctx *sctx,
ret = 0;
goto out;
}
+ } else {
+ struct extent_buffer *eb;
+ struct btrfs_file_extent_item *ei;
+ u8 extent_type;
+ u64 extent_disknr;
+
+ eb = path-nodes[0];
+ ei = btrfs_item_ptr(eb, path-slots[0],
+ struct btrfs_file_extent_item);
+
+ extent_type = btrfs_file_extent_type(eb, ei);
+ extent_disknr = btrfs_file_extent_disk_bytenr(eb, ei);
+ if (extent_type == BTRFS_FILE_EXTENT_REG
+ extent_disknr == 0) {
+ /*
+* This is disknr=0 extent in a full-send or a
+* new inode in a diff-send. Since we will
+* send truncate command in finish_inode_if_needed
+* anyways, the inode size will be correct,
+* and we don't have to send all-zero data.
+*/
+ ret = 0;
+ goto out;
+ }
}
ret = find_extent_clone(sctx, path, key-objectid, key-offset,
--
1.7.9.5
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/11 V3] add show command to the subvol sub command
Gene, I should do that in couple of days. Thanks. Anand On 01/21/2013 08:19 AM, Gene Czarcinski wrote: On 01/10/2013 06:41 AM, Anand Jain wrote: This is an attempt to make btrfs cli more end user friendly. And adds show subcommand to display all known (as of now) information of the given subvol including its snapshot(s). The below patch: Btrfs-progs: accommodate different layout for printing subvol list obsoletes the previosuly submitted patch Btrfs-progs: make provision to print subvol list tree format Please provide your review comments. Thanks. An example output: First list out all the subvols/snapshots using the list subcommand. btrfs su list /btrfs ID 256 gen 10 top level 5 path sv1 ID 259 gen 6 top level 5 path sv1/snap ID 260 gen 7 top level 5 path snap ID 261 gen 8 top level 5 path sv2 ID 262 gen 9 top level 5 path sv1/snap2 ID 263 gen 10 top level 5 path sv1/snap3 Then use show subcommand to know all known information about the given subvol. btrfs su show /btrfs/sv1 /btrfs/sv1 uuid: 508b12a5-ed2a-3942-af63-830967201070 Parent uuid: - Creation time: 2013-01-07 13:53:30 Object ID: 256 Generation (Gen): 10 Gen at creation: 5 Parent: 5 Top Level: 5 Snapshot(s): sv1/snap sv1/snap2 sv1/snap3 v2-v3: List snapshots of the given subvol if any. Uses a more simple approach to obtain the snapshots. You have certainly been busy. However, your patches need to be rebased considering some of the patch-sets currently sent to the mailing list. If nothing else, I would appreciate it if you could rebase to David Sterba's set. And then there are the two patch-sets I submitted. BTW, I do not claim anything except for the effort to scrapes these off the list to attempt getting btrfs-progs a bit more current with respect to bugfixes. I was going to include su show but your latest submission is a bit too much for now. Gene -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] V2 On a diff-send, avoid sending PREALLOC extents
On a diff-send, avoid sending PREALLOC extents, if the parent root has
only PREALLOC extents on an appropriate file range.
This does not fully avoid sending PREALLOC extents, because on full-send or
on new inode we need a new send command to do that. But this patch improves
the situation by handling diff-sends.
Signed-off-by: Alex Lyakas a...@zadarastorage.com
---
fs/btrfs/send.c | 154 +--
1 file changed, 150 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index bdef966..a162fa6 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -3763,6 +3763,148 @@ out:
return ret;
}
+static int is_prealloc_extent_unchanged(struct send_ctx *sctx,
+ struct btrfs_path *left_path,
+ struct btrfs_key *ekey)
+{
+ int ret = 0;
+ struct btrfs_key key;
+ struct btrfs_path *path = NULL;
+ struct extent_buffer *eb;
+ int slot;
+ struct btrfs_key found_key;
+ struct btrfs_file_extent_item *ei;
+ u64 left_len;
+ u64 right_len;
+ u8 right_type;
+
+ eb = left_path-nodes[0];
+ slot = left_path-slots[0];
+ ei = btrfs_item_ptr(eb, slot, struct btrfs_file_extent_item);
+ left_len = btrfs_file_extent_num_bytes(eb, ei);
+
+ /*
+* The logic is similar, but much simpler than in is_extent_unchanged().
+* We need to check extents on the parent root, and make sure that only
+* PREALLOC extents are on the same file range as our current extent.
+*
+* Following comments will refer to these graphics. L is the left
+* extents which we are checking at the moment. 1-8 are the right
+* extents that we iterate.
+*
+*|-L-|
+* |-1-|-2a-|-3-|-4-|-5-|-6-|
+*
+*|-L-|
+* |--1--|-2b-|...(same as above)
+*
+* Alternative situation. Happens on files where extents got split.
+*|-L-|
+* |---7---|-6-|
+*
+* Alternative situation. Happens on files which got larger.
+*|-L-|
+* |-8-|
+* Nothing follows after 8.
+*/
+
+ path = alloc_path_for_send();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = ekey-objectid;
+ key.type = BTRFS_EXTENT_DATA_KEY;
+ key.offset = ekey-offset;
+ ret = btrfs_search_slot_for_read(sctx-parent_root, key, path, 0, 0);
+ if (ret 0)
+ goto out;
+ if (ret) {
+ ret = 0;
+ goto out;
+ }
+
+ /*
+* Handle special case where the right side has no extents at all.
+*/
+ eb = path-nodes[0];
+ slot = path-slots[0];
+ btrfs_item_key_to_cpu(eb, found_key, slot);
+ if (found_key.objectid != key.objectid ||
+ found_key.type != key.type) {
+ /*
+* We need to send a prealloc command,
+* which we don't have yet, just send
+* this extent fully.
+*/
+ ret = 0;
+ goto out;
+ }
+
+ key = found_key;
+ while (key.offset ekey-offset + left_len) {
+ ei = btrfs_item_ptr(eb, slot, struct btrfs_file_extent_item);
+ right_type = btrfs_file_extent_type(eb, ei);
+ right_len = btrfs_file_extent_num_bytes(eb, ei);
+
+ if (right_type != BTRFS_FILE_EXTENT_PREALLOC) {
+ ret = 0;
+ goto out;
+ }
+
+ /*
+* Are we at extent 8? If yes, we know the extent is changed.
+* This may only happen on the first iteration.
+*/
+ if (found_key.offset + right_len = ekey-offset) {
+ /*
+* We need to send a prealloc command,
+* which we don't have yet,
+* just send this extent fully.
+*/
+ ret = 0;
+ goto out;
+ }
+
+ /*
+* The right extent is also PREALLOC, so up to
+* here we are ok, continue checking
+*/
+ ret = btrfs_next_item(sctx-parent_root, path);
+ if (ret 0)
+ goto out;
+ if (!ret) {
+ eb = path-nodes[0];
+ slot = path-slots[0];
+ btrfs_item_key_to_cpu(eb, found_key, slot);
+ }
+ if (ret || found_key.objectid != key.objectid ||
+ found_key.type != key.type) {
+ key.offset += right_len;
+ break;
+ } else {
+ if (found_key.offset
Re: [PATCH 2/2] On a diff-send, avoid sending PREALLOC extent
Hi Josef, I have resent the patches. There are still some please, no space before tabs warnings, which I am not sure how to correct, because I don't see space before tab in that place. If there is still a problem, can you please advise what is wrong. Thanks, Alex. On Mon, Jan 14, 2013 at 8:45 PM, Josef Bacik jba...@fusionio.com wrote: On Wed, Jan 09, 2013 at 10:41:10AM -0700, Alex Lyakas wrote: Subject: [PATCH 2/2] On a diff-send, avoid sending PREALLOC extents, if the parent root has only PREALLOC extents on an appropriate file range. This does not fully avoids sending PREALLOC extents, because on full-send or new inode we need a new send command to do that. But this patch improves the situation by handling diff-sends. Signed-off-by: Alex Lyakas alex.bt...@zadarastorage.com --- Malformed patch and it confused patchwork, please resend. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] btrfs-progs: more bugfixes for 0.20-rc1
On Fri, Jan 18, 2013 at 04:02:09PM -0500, Gene Czarcinski wrote: Please send them to the list. Josef Bacik jo...@toxicpanda.com is the current Fedora maintainer as indicated in bugzilla. mailto:jo...@toxicpanda.com Currently in Fedora 18, btrfs-progs is the v0.20-rc1 with git commit id mailto:jo...@toxicpanda.com91d9eec plus a total of seven patches. Two of those patches from the list which I have already indicated. The rest of those patches have been around for a while (at least since mid 2011) and I cannot find any doc. While I could send them in (and will if needed), I feel a little uneasy since Josef is responsible in Fedora. While this may be a different individual since the email address is different, I suspect this Josef is the same Josef who wrote the other two patches and he may be able to speak to them better. Sorry, I somehow assumed you were speaking on behalf of Fedora. I have been scraping mailing list for all of the btrfs-progs patches and plan to gradually test them. Since it appears you will be doing something similar, if you want to point me to some for testing, I will give it a shot with some virtual and some real-iron testing. And sometimes it is also a matter of the time needed to refit/rebase patches. Great, thanks! Testing of the userspace utilities is relatively safe (compared to the kernel module) and I'm using using the tool compiled from the master+patches on my regular testboxes. So far no disasterous problems encountered and I can immediatelly see if something goes wrong during normal use of the tools. Hugo once started a regression testsuite that should eg. verify sanity of the option parsing, so extending these tests would be welcome. http://git.darksatanic.net/cgi/gitweb.cgi?p=btrfs-progs-unstable.git;a=commit;h=c18d54a5bab5f5e0c6dc3c8cf4a1cc073e82f876 david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS dmesg error
On Mon, Jan 21, 2013 at 05:01:36PM +1100, Skender wrote:
I noticed in the dmesg logs that there seems to be a bug/issue with btrfs.
Here is the trace/message from dmesg. Im going by the instructions from
the
https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#How_do_I_report_bugs_and_issues.3F
. Let me know if you need anything else.
Please paste a few lines around the messages, the error code is missing,
though I'm suspecting it to be ENOSPC:
submit_compressed_extents:
659 if (IS_ERR(trans)) {
660 ret = PTR_ERR(trans);
661 } else {
662 trans-block_rsv =
root-fs_info-delalloc_block_rsv;
663 ret = btrfs_reserve_extent(trans, root,
664async_extent-compressed_size,
665async_extent-compressed_size,
6660, alloc_hint, ins, 1);
667 if (ret)
668 btrfs_abort_transaction(trans, root, ret);
(here it aborts)
669 btrfs_end_transaction(trans, root);
670 }
I've never seen this error before.
david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: scrub fails, any way to recover?
On Sun, Jan 20, 2013 at 09:54:38AM -0600, Neil Schemenauer wrote: I have a ~350 GB Btrfs filesystem that is corrupted. I think the damage was caused by a bad SATA cable. I can mount the filesystem and read most of the data (I already have backups of most everything). The scrub is aborted after a few seconds with the following error in the kernel log: parent transid verify failed on 795639808 wanted 102145 found 101462 parent transid verify failed on 795639808 wanted 102145 found 101462 verify_parent_transid: 16273 callbacks suppressed the difference between 102145 and 101462 is small and looks like a bunch lost writes (ie. not a random corruption), this supports the 'bad cable' root cause. From '16273 callbacks suppressed', there is a large number of broken b-tree connections. So far the rescue operation is to run btrfs-restore and copy the data out. I've tried btrfsck but it fails as well. Is there some way I can remove the damaged data and save the good or is a re-format the only solution? IIRC removing the damaged data hasn't been proposed yet, there was a patch to ignore the failures in a read-only mount https://patchwork.kernel.org/patch/913642/ (probably does not apply today) I think that the -o recovery mode could be extended in a way that a read-only + recovery would ignore the failures. I see two ways how to fix the on-disk b-tree structure (via fsck): 1) wipe the broken blocks and unlink from b-tree -- but a broen node on high level would kill lots of data unpredictably 2) in some cases it would be possible to promote the old transids to the current ones (to satisfy the transid verify check), however there may be some blocks already overwritten so it only pushes the problem farther Level of success depends on amount of data lost during the cable unplug and whether data or metadata were affected. It's more likely to rescue the filesystem if less metadata were affected. david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] Mutually exclusive ops fixes
On Sun, Jan 20, 2013 at 05:22:27PM +0200, Ilya Dryomov wrote: This is a set of fixes for mutually exclusive ops stuff added by Stefan as part of the dev-replace patchset. The main one here is the first patch, it fixes a regression in balancing interface. The second patch brings EINVAL error code back. The rest are relatively minor ordering fixes. Ilya Dryomov (5): Btrfs: bring back balance pause/resume logic Btrfs: fix mutually exclusive op is running error code Btrfs: fix unlock order in btrfs_ioctl_resize Btrfs: fix unlock order in btrfs_ioctl_rm_dev Btrfs: reorder locks and sanity checks in btrfs_ioctl_defrag 1-5 Reviewed-by: David Sterba dste...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/6] Btrfs-progs: detect if the disk we are formatting is a ssd
On Sat, Jan 19, 2013 at 11:14:09PM +0200, Brendan Hide wrote: On 2013/01/19 08:06 PM, Gene Czarcinski wrote: Signed-off-by: Josef Bacik jba...@fusionio.com Signed-off-by: Gene Czarcinski g...@czarc.net --- -values are raid0, raid1, raid10 or single. +values are raid0, raid1, raid10, single or dup. Single device will have dup +set by default except in the case of SSDs which will default to single. This is +because SSDs can remap blocks internally so duplicate blocks could end up in the +same erase block which negates the benefits of doing metadata duplication. Can't help but suggest that a NO_DEDUP command could be added to the SATA Transport Protocol/SCSI Command set. Not sure where to submit that idea ... :-/ You might get your answer from lkml, fsdevel or linux-scsi. david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: fix a regression in balance usage filter
Commit 3fed40cc (Btrfs: cleanup duplicated division functions), which was merged into 3.8-rc1, has introduced a regression by removing logic that was guarding us against bad user input. Bring it back. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c |9 - 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 86279c3..b577aee 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -2614,7 +2614,14 @@ static int chunk_usage_filter(struct btrfs_fs_info *fs_info, u64 chunk_offset, cache = btrfs_lookup_block_group(fs_info, chunk_offset); chunk_used = btrfs_block_group_used(cache-item); - user_thresh = div_factor_fine(cache-key.offset, bargs-usage); + if (bargs-usage == 0) + user_thresh = 0; + else if (bargs-usage 100) + user_thresh = cache-key.offset; + else + user_thresh = div_factor_fine(cache-key.offset, + bargs-usage); + if (chunk_used user_thresh) ret = 0; -- 1.7.9.1 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 12/13] Btrfs-progs: disable qgroupid 0 for quota_tree
At the request of the patch's author, this should be dropped as it has
been addressed by a patch to the kernel.
On 01/20/2013 04:04 PM, Gene Czarcinski wrote:
From: Wang Shilong wangsl-f...@cn.fujitsu.com
In kernel, qgroupid 0 is a special number when we run the quota group limit
command.
So, we should not be able to create a quota group whose id is 0, otherwise the
kernel
can't deal with it. Fix it.
Signed-off-by: Wang Shilong wangsl-f...@cn.fujitsu.com
Signed-off-by: Miao Xie mi...@cn.fujitsu.com
Signed-off-by: Gene Czarcinski g...@czarc.net
---
cmds-qgroup.c | 4
1 file changed, 4 insertions(+)
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 70019d0..dfff1b9 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -86,6 +86,10 @@ static int qgroup_create(int create, int argc, char **argv)
args.create = create;
args.qgroupid = parse_qgroupid(argv[1]);
+ if (!args.qgroupid) {
+ fprintf(stderr, ERROR: qgroup 0 is not supported\n);
+ return 30;
+ }
fd = open_file_or_dir(path);
if (fd 0) {
fprintf(stderr, ERROR: can't access '%s'\n, path);
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 01/13] btrfs-progs: btrfs-image.c: Added NULL pointer check.
On Sun, Jan 20, 2013 at 04:04:06PM -0500, Gene Czarcinski wrote: From: Nageswara R Sastry nasas...@in.ibm.com Check for the return value of 'open_ctree()' before dereferencing it. --- a/btrfs-image.c +++ b/btrfs-image.c @@ -491,6 +491,7 @@ static int create_metadump(const char *input, FILE *out, int num_threads, int ret; root = open_ctree(input, 0, 0); + BUG_ON(!root); Bug_on is not the right fix here, I prefer a more extensive fix http://permalink.gmane.org/gmane.comp.file-systems.btrfs/15305 which prints a message and exits. BUG_ON(root-nodesize != root-leafsize); ret = metadump_init(metadump, root, out, num_threads, -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 02/13] Btrfs-progs: fix resolving of loop devices
On Sun, Jan 20, 2013 at 04:04:07PM -0500, Gene Czarcinski wrote: From: Nirbheek Chauhan nirbheek.chau...@collabora.co.uk The LOOP_GET_STATUS ioctl truncates filenames to 64 characters. We should get the backing file for a given loop device from /sys/. This is how losetup does it as well. Signed-off-by: Gene Czarcinski g...@czarc.net I'd expect Nirbheek's signed off here as well and I've noticed Tested-By: Hector Oron hector.o...@collabora.co.uk (https://patchwork.kernel.org/patch/1629581/) Are you guys ok with adding them? david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/13] btrfs-show-super.c
On Sun, Jan 20, 2013 at 04:04:11PM -0500, Gene Czarcinski wrote: remove extra blank line at EOF The patch contents and description do not match. --- a/man/Makefile +++ b/man/Makefile @@ -1,4 +1,4 @@ -GZIP=gzip +GZIPCMD=gzip I'm not sure if this change is needed, does it stick to some well-known or established style in makefiles? david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
kernel BUG at fs/btrfs/extent-tree.c:5151
Hi All, I have has a machine doesn't get on with btrfs at all. I've been using it as a testbed for several months and it never seems to last more than a couple of weeks before FS corruption. My versions are: - OS - CentOS 6.3 - Kernel - 3.7.1-2 from http://elrepo.org/tiki/kernel-ml - btrfs-progs - v0.20-rc1-37-g91d9eec. Built from git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git The /var/log/messages log for the latest failure is given below. Two things of note: 1. The btrfs device is sitting on top of a 16.3 TB, 7 partition software RAID 5 array. The bug seems to have occurred during a md data-check on that device. By default CentOS 6 does these checks every week. 2. After I rebooted the server the first thing I did was try to unmount the btrfs device. However the umount command hangs in an uninterruptible sleep in btrfs_error_commit_super. Thoughts? - Richard Jan 20 11:30:04 backup3 kernel: leaf 556658688 total ptrs 102 free space 851 Jan 20 11:30:04 backup3 kernel: item 0 key (5846384640 b8 282234880) itemoff 3991 itemsize 4 Jan 20 11:30:04 backup3 kernel: shared data backref count 1 Jan 20 11:30:04 backup3 kernel: item 1 key (5846384640 b8 369684480) itemoff 3987 itemsize 4 Jan 20 11:30:04 backup3 kernel: shared data backref count 1 [snip a bunch of similar lines] Jan 20 11:30:04 backup3 kernel: item 32 key (5846384640 b8 225396207616) itemoff 3863 itemsize 4 Jan 20 11:30:04 backup3 kernel: shared data backref count 1 Jan 20 11:30:04 backup3 kernel: item 33 key (5846429696 a8 65536) itemoff 3797 itemsize 66 Jan 20 11:30:04 backup3 kernel: extent refs 35 gen 40 flags 1 Jan 20 11:30:04 backup3 kernel: extent data backref root 257 objectid 172094 offset 0 count 1 Jan 20 11:30:04 backup3 kernel: shared data backref parent 73255260160 count 1 Jan 20 11:30:04 backup3 kernel: item 34 key (5846429696 b8 282234880) itemoff 3793 itemsize 4 Jan 20 11:30:04 backup3 kernel: shared data backref count 1 [snip a bunch of similar lines] Jan 20 11:30:04 backup3 kernel: item 67 key (5846495232 a8 77824) itemoff 3599 itemsize 66 Jan 20 11:30:04 backup3 kernel: extent refs 35 gen 40 flags 1 Jan 20 11:30:04 backup3 kernel: extent data backref root 257 objectid 172091 offset 0 count 1 Jan 20 11:30:04 backup3 kernel: shared data backref parent 73255256064 count 1 Jan 20 11:30:04 backup3 kernel: item 68 key (5846495232 b8 28592) itemoff 3595 itemsize 4 Jan 20 11:30:04 backup3 kernel: shared data backref count 1 [snip a bunch of similar lines] Jan 20 11:30:04 backup3 kernel: item 100 key (5846495232 b8 225396203520) itemoff 3467 itemsize 4 Jan 20 11:30:04 backup3 kernel: shared data backref count 1 Jan 20 11:30:04 backup3 kernel: item 101 key (5846573056 a8 90112) itemoff 3401 itemsize 66 Jan 20 11:30:04 backup3 kernel: extent refs 36 gen 40 flags 1 Jan 20 11:30:04 backup3 kernel: extent data backref root 257 objectid 172096 offset 0 count 1 Jan 20 11:30:04 backup3 kernel: shared data backref parent 73255260160 count 1 Jan 20 11:30:04 backup3 kernel: [ cut here ] Jan 20 11:30:04 backup3 kernel: WARNING: at fs/btrfs/extent-tree.c:5134 __btrfs_free_extent+0x714/0x860 [btrfs]() Jan 20 11:30:04 backup3 kernel: Hardware name: MS-7522 Jan 20 11:30:04 backup3 kernel: Modules linked in: btrfs libcrc32c ipv6 ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iptable_filter ip_tables gpio_ich iTCO_wdt iTCO_vendor_support coretemp kvm_intel kvm crc32c_intel microcode pcspkr i2c_i801 lpc_ich r8169 mii sg i7core_edac edac_core ext4 mbcache jbd2 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx raid1 sd_mod crc_t10dif pata_acpi ata_generic pata_jmicron ahci libahci nouveau ttm drm_kms_helper hwmon mxm_wmi video wmi dm_mirror dm_region_hash dm_log dm_mod Jan 20 11:30:04 backup3 kernel: Pid: 25973, comm: btrfs-cleaner Not tainted 3.7.1-2.el6.elrepo.x86_64 #1 Jan 20 11:30:04 backup3 kernel: Call Trace: Jan 20 11:30:04 backup3 kernel: [8105715f] warn_slowpath_common+0x7f/0xc0 Jan 20 11:30:04 backup3 kernel: [810571ba] warn_slowpath_null+0x1a/0x20 Jan 20 11:30:04 backup3 kernel: [a0477384] __btrfs_free_extent+0x714/0x860 [btrfs] Jan 20 11:30:04 backup3 kernel: [a0477799] run_delayed_data_ref+0x159/0x160 [btrfs] Jan 20 11:30:04 backup3 kernel: [a047785a] run_one_delayed_ref+0xba/0xc0 [btrfs] Jan 20 11:30:04 backup3 kernel: [a0477976] run_clustered_refs+0x116/0x370 [btrfs] Jan 20 11:30:04 backup3 kernel: [a0477ca0] btrfs_run_delayed_refs+0xd0/0x300 [btrfs] Jan 20 11:30:04 backup3 kernel: [a0488894] btrfs_should_end_transaction+0x44/0x90
Re: [PATCH 05/13] Add btrfs-show-super
On Sun, Jan 20, 2013 at 04:04:10PM -0500, Gene Czarcinski wrote: From: Goffredo Baroncelli kreij...@inwind.it What I'm missing here is Stefan's S-o-B line, as he's the original author of the tool, adding it, JFYI. Signed-off-by: Goffredo Baroncelli kreij...@inwind.it Signed-off-by: Gene Czarcinski g...@czarc.net david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/13] Btrfs-progs: more patches for integration (integration-20130121)
On Sun, Jan 20, 2013 at 04:04:05PM -0500, Gene Czarcinski wrote: I have done some additional scraping of the mailing list to identify some low hanging fruit which I consider should be merged into the btrfs-progs repository. Thanks, I went through them and put together into an integration branch git://repo.or.cz/btrfs-progs-unstable/devel.git integration-20130121 I've modified the signed-off-by lines when I knew who was the author or contributed to the patch. Please let me know if I've messed the credits and fame attribution. There are three more patches that I've picked from my distro patch queue (and were sent to the list long ago) they're placed at the top of the shortlog. Arne Jansen (1): Btrfs-progs: bugfix for scrubbing single devices Wang Sheng-Hui (1): btrfs-progs: add malloc check in transaction.h/btrfs_start_transaction Goffredo Baroncelli (1): Ignore the error ENXIO and ENOMEDIUM during a devs scan Arvin Schnell (1): btrfs-progs: initialize data before send ioctl Danny Kukawka (1): btrfs-progs: Handle errors returned from open_ctree Jan Schmidt (2): Btrfs-progs: correcting misnamed parameter options for btrfs send Btrfs-progs: bugfix for subvolume parent determination in btrfs send Josef Bacik (2): Btrfs-progs: detect if the disk we are formatting is a ssd Btrfs-progs: add btrfs device ready command Nirbheek Chauhan (1): Btrfs-progs: fix resolving of loop devices Stefan Behrens (1): btrfs-progs: Add btrfs-show-super Ulrik (1): Btrfs-progs: correct btrfs receive usage string Wang Shilong (3): Btrfs-progs: fix arg parsing for btrfs qgroup limit commands Btrfs-progs: clean up reduplicate parse_qgroupid() and replace atoi with strtoull Btrfs-progs: check the relation of two group by real level numbers david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs_start_delalloc_inodes livelocks when creating snapshot under IO
Greetings all, I see the following issue during snap creation under IO: Transaction commit calls btrfs_start_delalloc_inodes() that locks the delalloc_inodes list, fetches the first inode, unlocks the list, triggers btrfs_alloc_delalloc_work/btrfs_queue_worker for this inode and then locks the list again. Then it checks the head of the list again. In my case, this is always exactly the same inode. As a result, this function allocates a huge amount of btrfs_delalloc_work structures, and I start seeing OOM messages in the kernel log, killing processes etc. During that time this transaction commit is stuck, so, for example, other requests to create snapshot (that must wait for this transaction to commit first) get stuck. The transaction_kthread also gets stuck in attempt to commit the transaction. Is this an intended behavior? Shouldn't we ensure that every inode in the delalloc list gets handled at most once? If the delalloc work is processed asynchronously, maybe the delalloc list can be locked once and traversed once? Josef, I see in commit 996d282c7ff470f150a467eb4815b90159d04c47 that you mention that btrfs_start_delalloc_inodes will just walk the list of delalloc inodes and start writing them out, but it doesn't splice the list or anything so as long as somebody is doing work on the box you could end up in this section _forever_. I guess I am hitting this here also. Miao, I tested the behavior before your commit 8ccf6f19b67f7e0921063cc309f4672a6afcb528 Btrfs: make delalloc inodes be flushed by multi-task, on kernel 3.6. I see same issue there as well, but OOM doesn't happen, because before your change btrfs_start_delalloc_inodes() was calling filemap_flush() directly. But I see still that btrfs_start_delalloc_inodes() handles same inode more than once, and in some cases never returns in 15 minutes or more, delaying all other transactions. And snapshot creation gets stuck for all this time. (The stack I see on kernel 3.6 is like this: [812f26c6] get_request_wait+0xf6/0x1d0 [812f35df] blk_queue_bio+0x7f/0x380 [812f0374] generic_make_request.part.50+0x74/0xb0 [812f0788] generic_make_request+0x68/0x70 [812f0815] submit_bio+0x85/0x110 [a034ace5] btrfs_map_bio+0x165/0x2f0 [btrfs] [a032880f] btrfs_submit_bio_hook+0x7f/0x170 [btrfs] [a033b7da] submit_one_bio+0x6a/0xa0 [btrfs] [a033f8a4] submit_extent_page.isra.34+0xe4/0x230 [btrfs] [a034084c] __extent_writepage+0x5ec/0x810 [btrfs] [a0340d22] extent_write_cache_pages.isra.26.constprop.40+0x2b2/0x410 [btrfs] [a03410c5] extent_writepages+0x45/0x60 [btrfs] [a0327178] btrfs_writepages+0x28/0x30 [btrfs] [81122b21] do_writepages+0x21/0x40 [81118e5b] __filemap_fdatawrite_range+0x5b/0x60 [8111982c] filemap_flush+0x1c/0x20 [a0334289] btrfs_start_delalloc_inodes+0xc9/0x1f0 [btrfs] [a0324f5d] btrfs_commit_transaction+0x44d/0xaf0 [btrfs] [a035200d] btrfs_mksubvol.isra.53+0x37d/0x440 [btrfs] [a03521ca] btrfs_ioctl_snap_create_transid+0xfa/0x190 [btrfs] [a03523e3] btrfs_ioctl_snap_create_v2+0x103/0x140 [btrfs] [a03546cf] btrfs_ioctl+0x80f/0x1bf0 [btrfs] [8118a01a] do_vfs_ioctl+0x8a/0x340 [8118a361] sys_ioctl+0x91/0xa0 [81665c42] system_call_fastpath+0x16/0x1b [] 0x Somehow the request queue of the block device gets empty and the transaction waits for a long time to allocate a request.) Some details about my setup: I am testing for-linus Chris's branch I have one subvolume with 8 large files (10GB each). I am running two fio processes (one per file, so only 2 out of 8 files are involved) with 8 threads each like this: fio --thread --directory=/btrfs/subvol1 --rw=randwrite --randrepeat=1 --fadvise_hint=0 --fallocate=posix --size=1000m --filesize=10737418240 --bsrange=512b-64k --scramble_buffers=1 --nrfiles=1 --overwrite=1 --ioengine=sync --filename=file-1 --name=job0 --name=job1 --name=job2 --name=job3 --name=job4 --name=job5 --name=job6 --name=job7 The files are preallocated with fallocate before the fio run. Mount options: noatime,nodatasum,nodatacow,nospace_cache Can somebody please advise on how to address this issue, and, if possible, how to solve it on kernel 3.6. Thanks, Alex. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/1] Btrfs-progs: Fix typo in balance start usage; filetrs - filters.
From: Andreas Fischer i...@andreasfischer.net
Fix typo in btrfs balance start usage documentation: 'filetrs' should have
been 'filters'.
Signed-off-by: Andreas Fischer i...@andreasfischer.net
---
cmds-balance.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/cmds-balance.c b/cmds-balance.c
index 38a7426..dfa0dfe 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -349,7 +349,7 @@ static const char * const cmd_balance_start_usage[] = {
,
-d[filters]act on data chunks,
-m[filters]act on metadata chunks,
- -s[filetrs]act on system chunks (only under -f),
+ -s[filters]act on system chunks (only under -f),
-v be verbose,
-f force reducing of metadata integrity,
NULL
--
1.7.10.4
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/13] Add btrfs-show-super
On 01/21/2013 07:04 PM, David Sterba wrote: On Sun, Jan 20, 2013 at 04:04:10PM -0500, Gene Czarcinski wrote: From: Goffredo Baroncellikreij...@inwind.it What I'm missing here is Stefan's S-o-B line, as he's the original author of the tool, adding it, JFYI. I am not sure if he want to Sign it. During the review he blamed about some style issues. But for time constraints I was not able to update the patches. So before adding Stefen's S-o-B we need an ack from him. Signed-off-by: Goffredo Baroncellikreij...@inwind.it Signed-off-by: Gene Czarcinskig...@czarc.net david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS dmesg error
Here are a couple of lines before and after.
Cheers,
Skender Haxhimolla
[432770.724678] virtuoso-t[17472]: segfault at 8 ip 007f4147 sp
7fff69250848 error 4 in virtuoso-t[40+9ad000]
[615597.825359] mythtranscode[10886]: segfault at 4 ip 004243c9 sp
7fff27970368 error 4 in mythtranscode[40+68000]
[655330.131113] [ cut here ]
[655330.131123] WARNING: at fs/btrfs/super.c:246
__btrfs_abort_transaction+0x46/0xa0()
[655330.131125] Hardware name: Precision WorkStation T3400
[655330.131126] btrfs: Transaction aborted
[655330.131128] Modules linked in: xt_limit xt_tcpudp nf_conntrack_ipv4
nf_defrag_ipv4 xt_state nf_conntrack ip6table_filter ip6_tables iptable_filter
ip_tables x_tables rfcomm bnep bluetooth dm_crypt dvb_usb_cxusb
snd_hda_codec_analog dib7000p dibx000_common dvb_usb snd_hda_intel
snd_hda_codec snd_hwdep dvb_core snd_pcm snd_seq_midi snd_rawmidi
snd_seq_midi_event rc_core dib0070 snd_seq coretemp snd_timer snd_seq_device
snd kvm_intel soundcore kvm psmouse snd_page_alloc microcode x38_edac lpc_ich
dcdbas serio_raw edac_core ppdev parport_pc lp parport floppy firewire_ohci
firewire_core crc_itu_t nouveau tg3 ttm drm_kms_helper drm i2c_algo_bit mxm_wmi
video wmi
[655330.131182] Pid: 5838, comm: btrfs-delalloc- Not tainted 3.6.11-custom #1
[655330.131183] Call Trace:
[655330.131190] [810507dc] warn_slowpath_common+0x83/0x9c
[655330.131193] [81050898] warn_slowpath_fmt+0x46/0x48
[655330.131197] [81231e0b] ? btrfs_reserve_extent+0xbd/0x1c6
[655330.131200] [812208a7] __btrfs_abort_transaction+0x46/0xa0
[655330.131203] [812468f4] submit_compressed_extents+0x1bd/0x415
[655330.131207] [81246bd2] async_cow_submit+0x86/0x8b
[655330.131210] [8125ec27] run_ordered_completions+0x76/0xd5
[655330.131213] [8125f49e] worker_loop+0x183/0x4b9
[655330.131216] [8125f31b] ? btrfs_queue_worker+0x288/0x288
[655330.131220] [8106d9d0] kthread+0x6f/0x77
[655330.131225] [815b59c4] kernel_thread_helper+0x4/0x10
[655330.131228] [8106d961] ? kthread_freezable_should_stop+0x48/0x48
[655330.131231] [815b59c0] ? gs_change+0x13/0x13
[655330.131233] ---[ end trace bc2b68accbd79b9c ]---
[655330.131235] BTRFS warning (device sda2): Aborting unused transaction.
[655729.210531] BTRFS warning (device sda2): Aborting unused transaction.
On 21/01/13 12:00 +0100, David Sterba wrote:
On Mon, Jan 21, 2013 at 05:01:36PM +1100, Skender wrote:
I noticed in the dmesg logs that there seems to be a bug/issue with btrfs.
Here is the trace/message from dmesg. Im going by the instructions from
the
https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#How_do_I_report_bugs_and_issues.3F
. Let me know if you need anything else.
Please paste a few lines around the messages, the error code is missing,
though I'm suspecting it to be ENOSPC:
submit_compressed_extents:
659 if (IS_ERR(trans)) {
660 ret = PTR_ERR(trans);
661 } else {
662 trans-block_rsv =
root-fs_info-delalloc_block_rsv;
663 ret = btrfs_reserve_extent(trans, root,
664async_extent-compressed_size,
665async_extent-compressed_size,
6660, alloc_hint, ins, 1);
667 if (ret)
668 btrfs_abort_transaction(trans, root,
ret);
(here it aborts)
669 btrfs_end_transaction(trans, root);
670 }
I've never seen this error before.
david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/13] Add btrfs-show-super
On 01/21/2013 20:54, Goffredo Baroncelli wrote: On 01/21/2013 07:04 PM, David Sterba wrote: On Sun, Jan 20, 2013 at 04:04:10PM -0500, Gene Czarcinski wrote: From: Goffredo Baroncellikreij...@inwind.it What I'm missing here is Stefan's S-o-B line, as he's the original author of the tool, adding it, JFYI. I am not sure if he want to Sign it. During the review he blamed about some style issues. But for time constraints I was not able to update the patches. So before adding Stefen's S-o-B we need an ack from him. For the record, all the extra spaces are not from me. Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de Signed-off-by: Goffredo Baroncellikreij...@inwind.it Signed-off-by: Gene Czarcinskig...@czarc.net -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs: failed to read chunk tree on vde1 (f18 anaconda crash)
Thanks for asking, but in the end it was not btrfs related at all. Virt-Manager failed to issue a warning (for yet unknown reasons) and a single hard disk image was being accessed as two different devices... I spent quite a bit of time until i found it. (normally, virt-manager prevents this) Cheers. On Sun, Jan 20, 2013 at 10:22 AM, Hugo Mills h...@carfax.org.uk wrote: On Sat, Jan 19, 2013 at 07:53:08PM -0300, Reartes Guillermo wrote: Hi, I reported a bug to anaconda (rhbz 901905), but it might be btfs related. Anaconda does this: 18:49:10,317 INFO program: Running... mkfs.btrfs --data=single --label=fedora /dev/vdb1 /dev/vdc1 /dev/vdd1 /dev/vde1 18:49:10,742 INFO program: 18:49:10,746 INFO program: WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL 18:49:10,746 INFO program: WARNING! - see http://btrfs.wiki.kernel.org before using 18:49:10,748 INFO program: 18:49:10,749 INFO program: SMALL VOLUME: forcing mixed metadata/data groups 18:49:10,750 INFO program: Created a data/metadata chunk of size 8388608 18:49:10,750 INFO program: adding device /dev/vdc1 id 2 18:49:10,751 INFO program: adding device /dev/vdd1 id 3 18:49:10,752 INFO program: adding device /dev/vde1 id 4 18:49:10,752 INFO program: fs created label fedora on /dev/vdb1 Out of interest, how big are these devices? Hugo. [snip] I booted again and did both the mk.btrfs and the mount manually and i got the same error. Is it a known issue? -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- We believe in free will because we have no choice. --- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] xfstests: fix 284 ENOENT error
Hey Liu, On Sun, Jan 06, 2013 at 10:30:38PM +0800, Liu Bo wrote: 284 misses a 'mkdir' operation. Signed-off-by: Liu Bo liub.li...@gmail.com Reviewed-by: Ben Myers b...@sgi.com Applied to git://oss.sgi.com/xfs/cmds/xfstests.git Regards, Ben -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH V2] btrfs-progs: remove device show from btrfs man page
Add SOB line: Signed-off-by: Simon Xu xu.si...@oracle.com Thanks David for pointing this out. Simon On 2013/1/9 15:31, Simon Xu wrote: Remove device show from btrfs man page as it's not supported by the btrfs utility. --- man/btrfs.8.in | 2 -- 1 file changed, 2 deletions(-) diff --git a/man/btrfs.8.in b/man/btrfs.8.in index 9222580..e663da2 100644 --- a/man/btrfs.8.in +++ b/man/btrfs.8.in @@ -33,8 +33,6 @@ btrfs \- control a btrfs filesystem .PP \fBbtrfs\fP \fBdevice scan\fP\fI [--all-devices|device [device...]]\fP .PP -\fBbtrfs\fP \fBdevice show\fP\fI [--all-devices|uuid|label]\fP -.PP \fBbtrfs\fP \fBdevice add\fP\fI device [device...] path \fP .PP \fBbtrfs\fP \fBdevice delete\fP\fI device [device...] path \fP -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
