Re: btrfs double send
Filipe, Its still not perfect. Here I can do sequential sends a few times then I get something like this: [root@grover snap]# sh -x brh + base=/snap/shot ++ date +%Y-%V-%u_%m-%d_%H:%M + stamp=2015-43-6_10-24_10:24 + btrfs subv snapshot -r /snap/homevol /snap/shot.2015-43-6_10-24_10:24 Create a readonly snapshot of '/snap/homevol' in '/snap/shot.2015-43-6_10-24_10:24' + sync + /usr/bin/time -f 'send %E' nice -19 ionice -c3 btrfs send -v -p /snap/shot /snap/shot.2015-43-6_10-24_10:24 + btrfs receive -v /backup/snap At subvol /snap/shot.2015-43-6_10-24_10:24 At snapshot shot.2015-43-6_10-24_10:24 receiving snapshot shot.2015-43-6_10-24_10:24 uuid=cb3ad856-ca0f-f744-b7c3-b33f2d5bc8d3, ctransid=625267 parent_uuid=cb3ad856-ca0f-f744-b7c3-b33f2d5bc8d3, parent_ctransid=619893 ERROR: unlink home/ed/Maildir/.spam/dovecot.index.log.2 failed. No such file or directory Command terminated by signal 13 send 1:49.64 + sync + btrfs subv delete /snap/shot Delete subvolume (no-commit): '/snap/shot' + sync + mv /snap/shot.2015-43-6_10-24_10:24 /snap/shot so there is another bug hiding in the code, its usually something in my Maildir that shows in the log. Please let me know if there is anything you would like me to try. I am running 4.2 with the 4.3 for-linus tree applied and the 4.2.x patches with btrfs fixes removed. On top of this are a few patches from this list. TIA Ed Tomlinson On Saturday, October 24, 2015 1:52:21 PM EDT, Filipe Manana wrote: On Sat, Oct 24, 2015 at 6:36 PM, <kol...@kolcon.net> wrote: Hello, I would like to do backups based on btrfs send/receive. So I though I would do a transfer over portable HDD and then incremental sends (using -p) over network. Initial : ... It's a bug, we got it recently reported and fixed. The fix is in Chris' integration branch for the upcoming 4.4 merge window: http://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/commit/?h=integration-4.4=b96b1db039ebc584d03a9933b279e0d3e704c528 cheers Best regards, Lubos -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs double send
Filipe, I realize its another bug. Two additional pieces of info that might help. One, btrfs-progs was at 4.1.2 (I missed a tag= in my git pull). Second is that I have been able to recreate this issue three times over a period of two to three days (tring again with 4.2.3). My fs is probably a good testcase for any patches that appear. Meanwhile I've been falling back to rsync which always works but is so much slower. TIA Ed On Sunday, October 25, 2015 9:42:54 AM EDT, Filipe Manana wrote: On Sun, Oct 25, 2015 at 1:38 PM, Ed Tomlinson <e...@aei.ca> wrote: Filipe, Its still not perfect. Here I can do sequential sends a few times then I get something like this: [root@grover snap]# sh -x brh + base=/snap/shot ... Different and unrelated problem. Yes, we know there are still some problems regarding send issuing invalid/outdated paths to the send stream. Nothing to do with incorrect uuids in the send stream. Please let me know if there is anything you would like me to try. I am running 4.2 with the 4.3 for-linus tree applied and the 4.2.x patches with btrfs fixes removed. On top of this are a few patches from this list. TIA Ed Tomlinson ... -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: fix resending received snapshot with parent
On Friday, October 9, 2015 4:24:10 PM EDT, Filipe Manana wrote: On Wed, Sep 30, 2015 at 8:23 PM, Robin Ruede <r.ru...@gmail.com> wrote: This fixes a regression introduced by 37b8d27d between v4.1 and v4.2. When a snapshot is received, its received_uuid is set to the original uuid of the subvolume. When that snapshot is then resent to a third filesystem, it's received_uuid is set to the second uuid instead of the original one. The same was true for the parent_uuid. ... Reviewed-by: Filipe Manana <fdman...@suse.com> Thanks for fixing this. I've added this to my integration branch [1] and will send soon a pull request to Chris for 4.4, including this fix plus a few others for send/receive, after some more testing. I've also made an xfstest for it [1, 2] Another thanks for this fix. It fixes things here. I am runing 4.2.3 with the 4.3 btrfs tree pulled on top of it along with this fix. Incremental sends are now working again. Tested-by: Ed Tomlinson <e...@aei.ca> This fixes a regression, can we please get into 4.3? Thanks Ed Tomlinson -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs send -c fails saying "parent determination failed"
Hi, I have the same problem here. I'd love to stop using rsync in place of send recieve. As with Paride my _working_ scripts just stop functioning... Thanks Ed on Tuesday, September 29, 2015 6:20:11 PM EDT, Paride Legovini wrote: Hi, btrfs send -c stopped working for me several months ago. My setup is actually very simple. On the "send" side I have: # btrfs sub list -u / | grep rootfs-snapshot- ID 2221 gen 93340 top level 5 uuid adf43113-0442-9847-9ce4-7d9c06e77602 path rootfs-snapshot-20150923 ID 2262 gen 96026 top level 5 uuid 49f34ca5-42da-7d4b-a159-c3027650d8e6 path rootfs-snapshot-20150929 Several other subvolumes also exist there, just in case it matters. On the receive side: # btrfs sub list -R /mnt/cryptobackup/ | grep rootfs-snapshot-20150923 ID 1532 gen 3066 top level 5 received_uuid adf43113-0442-9847-9ce4-7d9c06e77602 path snapshots/rootfs-snapshot-20150923 As you can see the uuids for rootfs-snapshot-20150923 are the same on both sides. If I try to send|receive /rootfs-snapshot-20150929 using rootfs-snapshot-20150923 as clone source it fails: # btrfs send -c /rootfs-snapshot-20150923 /rootfs-snapshot-20150929 | btrfs receive /mnt/cryptobackup/snapshots/ At subvol /rootfs-snapshot-20150929 ERROR: parent determination failed for 2221 while it works fine if I use btrfs send -p. This is always reproducible, it fails every time across reboots and kernel upgrades; it works every time with -p. In both / and /mnt/cryptobackup I didn't mount any special subvolid: # btrfs sub get-default / ID 5 (FS_TREE) # btrfs inspect-internal rootid / 5 # btrfs sub get-default /mnt/cryptobackup/ ID 5 (FS_TREE) # btrfs inspect-internal rootid /mnt/cryptobackup/ 5 so nothing strange here. Some other possibly useful information: $ uname -a Linux mandragola 4.2.0-1-amd64 #1 SMP Debian 4.2.1-1 (2015-09-25) x86_64 GNU/Linux so you have guessed that I'm running an up to date Debian system. Keep in mind that it is several kernel versions that I'm getting this problem, it's nothing specific to this one. $ btrfs --version btrfs-progs v4.1.2 # btrfs fi show Label: none uuid: 5651d651-5c48-425f-9fc9-56f2a9ad004f Total devices 1 FS bytes used 118.24GiB devid1 size 237.97GiB used 218.02GiB path /dev/sda2 Label: none uuid: c4db7f4f-b976-4d36-bd76-76e818341cef Total devices 1 FS bytes used 934.71GiB devid1 size 1.82TiB used 941.03GiB path /dev/mapper/cryptobackup $ btrfs fi df / Data, single: total=212.01GiB, used=116.98GiB System, single: total=4.00MiB, used=48.00KiB Metadata, single: total=6.01GiB, used=1.27GiB GlobalReserve, single: total=448.00MiB, used=0.00B $ btrfs fi df /mnt/cryptobackup/ Data, single: total=932.00GiB, used=931.70GiB System, DUP: total=8.00MiB, used=128.00KiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=4.50GiB, used=3.01GiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=512.00MiB, used=0.00B $ dmesg | grep -i btrfs | curl -F "sprunge=<-" http://sprunge.us http://sprunge.us/EiHE If I hit a bug I'll be happy to help in finding it, just tell me if you need further information. Otherwise, if I'm just missing something, I'll appreciate any hint. Thank you, Paride -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
What am I doing wrong?
Hi This process used to work. I am not sure what is going wrong now and/or what requirements have changed. I'd really appriecate another set of eyes looking at this. The sending volume has: # btrfs subvol list -pqu /snap ID 258 gen 416417 parent 5 top level 5 parent_uuid - uuid 6447cef2-9b1a-c146-8dc3-b759ed2dd694 path homevol ID 749 gen 399865 parent 258 top level 258 parent_uuid 6447cef2-9b1a-c146-8dc3-b759ed2dd694 uuid c309c50a-8558-a242-86bd-d64b7f6ca0cd path homevol/backup ID 750 gen 401012 parent 258 top level 258 parent_uuid 6447cef2-9b1a-c146-8dc3-b759ed2dd694 uuid d29ffd19-8ca1-e248-ace0-fc174902f1f3 path homevol/backup.2015-27-6_07-04_22:00 The recieving volume has: # btrfs subvol list -pquR /backup | grep c309c50a-8558-a242-86bd-d64b7f6ca0cd ID 3698 gen 3116 parent 5 top level 5 parent_uuid - received_uuid c309c50a-8558-a242-86bd-d64b7f6ca0cd uuid 18a46efa-3cc3-4742-925e-ea0681709592 path home/backup But send/recieve is not working: # btrfs send -ve -p homevol/backup homevol/backup.2015-27-6_07-04_22:00 | btrfs receive -v /backup/home At subvol homevol/backup.2015-27-6_07-04_22:00 At snapshot backup.2015-27-6_07-04_22:00 receiving snapshot backup.2015-27-6_07-04_22:00 uuid=d29ffd19-8ca1-e248-ace0-fc174902f1f3, ctransid=401012 parent_uuid=cb3ad856-ca0f-f744-b7c3-b33f2d5bc8d3, parent_ctransid=399865 ERROR: could not find parent subvolume Yet the subvol homevol/backup has been sent (full) to and exists on /backup filesystem as can be seen in the two subvol lists, yet send/recieve cannot seem to find it for an incremental backup. Why? TIA Ed Tomlinson -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs bug fixes
Chris, Have you looked at Omar Sandoval's [PATCH v2 0/5] Btrfs: RAID 5/6 missing device scrub+replace It would be really nice to have these when/if a disk dies... I been running with them since v1 without issue. Thanks Ed Tomlinson On Wednesday, July 1, 2015 1:24:41 PM EDT, Chris Mason wrote: On Tue, Jun 30, 2015 at 11:20:31PM +0100, fdman...@kernel.org wrote: From: Filipe Manana fdman...@suse.com Hi Chris, Please consider the following changes (or a subset at your will in case they are too many or too large) for the kernel 4.2 release. All these patches have been available in the mailing list for at least 2 weeks. ... Thanks! These are queued along with a few others I pulled from the list, Mark's dedup fixes, and the ones Qu mentioned. Once it passes tests here I'll push out. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 0/5] Btrfs: RAID 5/6 missing device scrub+replace
On Wednesday, June 24, 2015 12:15:29 AM EDT, Omar Sandoval wrote: On Tue, Jun 23, 2015 at 11:07:00AM +0800, wangyf wrote: Hi, I have tested your PATCH v2 , but something wrong happened. kernel: 4.1.0-rc7+ with your five patches vitrualBox ubuntu14.10-server + LVM I make a new btrfs.ko with your patches, rmmod original module and insmod the new. When I use the profile RAID1/10, mkfs successfully But when mount the fs, dmesg dumped: trans: 18446612133975020584 running 5 btrfs transid mismatch buffer 29507584, found 18446612133975020584 running 5 btrfs transid mismatch buffer 29507584, found 18446612133975020584 running 5 btrfs transid mismatch buffer 29507584, found 18446612133975020584 running 5 ... ... When use the RAID5/6, mkfs and mount system stoped at the 'mount -t btrfs /dev/mapper/server-dev1 /mnt' cmd. That's all. Hm, that's really weird, I can't reproduce that here at all. I don't see what would cause that in this series, and the changes from v1 are minimal. For my sake, could you make sure that there's nothing else going on? Omar, I've been running v1 of this patch and now with v2 and have done numberious reboots without issues... Just another data point. Ed -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID1, SSD+non-SSD (RAID 5/6 question)
On Saturday, February 7, 2015 1:39:07 AM EST, Duncan wrote: The btrfs raid1 read-mode device choice algorithm Duncan, Very interesting suff on the raid1 read select alg. What changes with raid5/6? Is that alg 'smarter'? TIA Ed Tomlinson -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole
Hi, Its a great idea to test the patches before submitting. However I think its even more importent to tell us the state of testing. eg. this is an RFC or in production in abc's kernel, and this version is untested or has been compile tested, boot tested, improves xfstests (before x failures out of z, after the patch the number of failures was y, where yx). A little bit of info like this will help those of us using the info in the lists and should aid getting your patches into the kernel faster. Thanks Ed On Tuesday, February 3, 2015 6:36:40 AM EST, Forrest Liu wrote: 2015-02-03 2:40 GMT+08:00 Ed Tomlinson e...@aei.ca: On Monday, February 2, 2015 9:39:06 AM EST, Ed Tomlinson wrote: Hi Booting a kernel with the three patches: [PATCH] Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole ... My fault, i should test these patches before i submit these patches. The oops was caused by patch Btrfs: btrfs_release_extent_buffer_page() didn't free pages of dummy extent I will resend these patches after test on linux-3.19-rc7. Thanks Forrest Anyone else? Thanks Ed Tomlinson ... -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole
Hi,
Found a problem compile testing this.
hole_size = key_offset - search_start;
Should not that be key.offset ?
TIA
Ed Tomlinson
On Monday, February 2, 2015 2:31:39 AM EST, Forrest Liu wrote:
If device tree has hole, find_free_dev_extent() cannot find available
address properly.
The example below, has one BIG hole in device tree, and can only
allocate just one chunk in a transaction.
item 9 key (1 DEV_EXTENT 273841913856) itemoff 15811 itemsize 48
dev extent chunk_tree 3
chunk objectid 256 chunk offset 272759783424 length 1073741824
item 10 key (1 DEV_EXTENT 1071632089088) itemoff 15763 itemsize 48
dev extent chunk_tree 3
chunk objectid 256 chunk offset 1070549958656 length 1073741824
item 11 key (1 DEV_EXTENT 1072705830912) itemoff 15715 itemsize 48
dev extent chunk_tree 3
chunk objectid 256 chunk offset 1071623700480 length
Signed-off-by: Forrest Liu forre...@synology.com
---
fs/btrfs/volumes.c | 15 +++
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index da7e0e1..61be789 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1060,6 +1060,7 @@ static int contains_pending_extent(struct
btrfs_trans_handle *trans,
struct extent_map *em;
struct list_head *search_list = trans-transaction-pending_chunks;
int ret = 0;
+ u64 physical_start = *start;
again:
list_for_each_entry(em, search_list, list) {
@@ -1070,9 +1071,9 @@ again:
for (i = 0; i map-num_stripes; i++) {
if (map-stripes[i].dev != device)
continue;
- if (map-stripes[i].physical = *start + len ||
+ if (map-stripes[i].physical = physical_start + len ||
map-stripes[i].physical + em-orig_block_len =
- *start)
+ physical_start)
continue;
*start = map-stripes[i].physical +
em-orig_block_len;
@@ -1195,8 +1196,14 @@ again:
*/
if (contains_pending_extent(trans, device,
search_start,
- hole_size))
- hole_size = 0;
+ hole_size)) {
+ if (key.offset = search_start)
+ hole_size = key_offset - search_start;
+ else {
+ WARN_ON(1);
+ hole_size = 0;
+ }
+ }
if (hole_size max_hole_size) {
max_hole_start = search_start;
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/6] btrfs-progs: Add fixing function for inodes whose nlink dismatch
Hi,
Looks like
Reported-by: Daniel Miranda danielk...@gmail.com
also needs to be added see: Re: Apparent metadata corruption (file that
simultaneously does/does not exist) on kernel 3.17.3
Where Daniel reports these patches fixed his fs too.
I expect an fsck with --repair specified to change my fs and creating and
populating an lost+found directory is not an unexpected result.
While btrfs is not supposed to need this - its obvious that it does and that
this corruption is not that uncommon.
Three victims repaired in less than two weeks - how many more are out there?
Thanks
Ed
PS. The other important question is how do we find the bug causing this and fix
it?
On Wednesday 03 December 2014 12:18:32 Qu Wenruo wrote:
[BUG]
At least two users have already hit a bug in btrfs causing file
missing(Chromium config file).
The missing file's nlink is still 1 but its backref points to non-exist
parent inode.
This should be a kernel bug, but btrfsck fix is needed anyway.
[FIX]
For such nlink mismatch inode, we will delete all the invalid backref,
if there is no valid backref for it, create 'lost+found' under the root
of the subvolume and add link to the directory.
Reported-by: Mike Gavrilov mikhail.v.gavri...@gmail.com
Reported-by: Ed Tomlinson e...@aei.ca
Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
changelog:
v2:
Use the a more generic fucntion, reset_nlink(), to repair the inode
nlink.
It will remove all, including valid, backref(along with
dir_item/index) and set nlink to 0, and add valid ones back.
This reset_nlink() method can handle more nlink error, not only
invalid inode_ref but also pure nlinks mismatch(2 valid inode_ref but
nlink is 1)
---
cmds-check.c | 236
++-
1 file changed, 232 insertions(+), 4 deletions(-)
diff --git a/cmds-check.c b/cmds-check.c
index 6419caf..627b794 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -27,6 +27,7 @@
#include unistd.h
#include getopt.h
#include uuid/uuid.h
+#include math.h
#include ctree.h
#include volumes.h
#include repair.h
@@ -1837,6 +1838,18 @@ static int repair_inode_backrefs(struct btrfs_root
*root,
struct btrfs_trans_handle *trans;
struct btrfs_key location;
+ ret = check_dir_conflict(root, backref-name,
+ backref-namelen,
+ backref-dir,
+ backref-index);
+ if (ret) {
+ /*
+ * let nlink fixing routine to handle it,
+ * which can do it better.
+ */
+ ret = 0;
+ break;
+ }
location.objectid = rec-ino;
location.type = BTRFS_INODE_ITEM_KEY;
location.offset = 0;
@@ -1874,20 +1887,233 @@ static int repair_inode_backrefs(struct btrfs_root
*root,
return ret ? ret : repaired;
}
+/*
+ * Search for the inode_ref of given 'ino' to get the filename and
+ * btrfs type.
+ * This will only use the first inode_ref.
+ */
+static int find_file_name_type(struct btrfs_root *root,
+struct btrfs_path *path,
+u64 ino, char *buf,
+int *namelen, u8 *type)
+{
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct btrfs_inode_ref *inode_ref;
+ struct btrfs_inode_item *inode_item;
+ struct extent_buffer *node;
+ u32 ret_namelen;
+ int slot;
+ int ret = 0;
+
+ /* Search for name in backref(Use the first one) */
+ key.objectid = ino;
+ key.type = BTRFS_INODE_REF_KEY;
+ key.offset = 0;
+
+ ret = btrfs_search_slot(NULL, root, key, path, 0, 0);
+ if (ret 0)
+ goto out;
+ node = path-nodes[0];
+ slot = path-slots[0];
+ btrfs_item_key_to_cpu(node, found_key, slot);
+ if (found_key.objectid != ino ||
+ found_key.type != BTRFS_INODE_REF_KEY) {
+ ret = -ENOENT;
+ goto out;
+ }
+ inode_ref = btrfs_item_ptr(node, slot, struct btrfs_inode_ref);
+ ret_namelen = btrfs_inode_ref_name_len(node, inode_ref);
+ read_extent_buffer(node, buf, (unsigned long)(inode_ref + 1),
+ret_namelen);
+ /* Search for inode type */
+ ret = btrfs_previous_item(root, path, ino, BTRFS_INODE_ITEM_KEY);
+ if (ret) {
+ if (ret 0)
+ ret = -ENOENT;
+ goto out;
+ }
+ node = path-nodes[0];
+ slot = path-slots[0];
+ inode_item = btrfs_item_ptr(node, slot, struct btrfs_inode_item);
+
+ if (namelen
Re: [PATCH 0/6] More generic inode nlink repair function
Hi, I'd really like to see these patches included in btrfsck - they repaired my fs. Once Qu got them working they found additional corruptions. This time there was no crash or stall just an umount that left (chromium) files unlinked... The bug with these files has been hitting me for a while - just did not recognize what was causing it or notice the corruption. The only objection I have seen to these patches is that they may create a lost+found directory. I submit this is an expected behavior for a fsck utility. When --repair is specified I expect a fsck to make changes to my fs one of which may be adding and populating a lost+found directory. Thanks Ed Tomlinson PS. It would be very interesting to find out WHY these files are ending up unlinked. Ideas? On Wednesday 03 December 2014 12:18:26 you wrote: Update on patch 4 and 6, other is not changed. This nlink repair function is more generic than the original one. The old one can only handle a specific case that the inode_ref is invalid, either point to a non-exist parent inode or point to a invalid inode(not dir or conflicting index/name). The new one will reset all the backref, no matter it is valid or not, and re-add all the valid backref, this make the nlink handles more corrupt cases. Qu Wenruo (6): btrfs-progs: print root dir verbose error in fsck btrfs-progs: Import btrfs_insert/del/lookup_extref() functions. btrfs-progs: Import lookup/del_inode_ref() function. btrfs-progs: Add btrfs_unlink() and btrfs_add_link() functions. btrfs-progs: Add btrfs_mkdir() function for the incoming 'lost+found' fsck mechanism. btrfs-progs: Add fixing function for inodes whose nlink dismatch Makefile | 2 +- cmds-check.c | 311 -- ctree.c | 6 + ctree.h | 38 + inode-item.c | 318 +++ inode.c | 484 +++ 6 files changed, 1148 insertions(+), 11 deletions(-) create mode 100644 inode.c -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 3.0 release - btrfs possible locking deadlock
On Monday 25 July 2011 15:49:37 Chris Mason wrote:
Excerpts from Ed Tomlinson's message of 2011-07-22 19:21:00 -0400:
On Thursday 21 July 2011 22:59:53 Linus Torvalds wrote:
So there it is. Gone are the 2.6.bignum days, and 3.0 is out.
Hi,
Managed to get this with btrfs rsync(ing) from ext4 to a btrfs fs with
three partitions using raid1.
[16018.211493] device fsid f7186eeb-60df-4b1a-890a-4a1eb42f81fe devid 1
transid 10 /dev/sdd4
[16018.230643] btrfs: use lzo compression
[16018.234619] btrfs: enabling disk space caching
[25949.414011]
[25949.414011] ===
[25949.416549] [ INFO: possible circular locking dependency detected ]
[25949.423187] 3.0.0-crc+ #348
[25949.423187] ---
[25949.423187] rsync/20237 is trying to acquire lock:
[25949.423187] (btrfs-extent-01){+.+...}, at: [a047ce88]
btrfs_try_spin_lock+0x78/0xb0 [btrfs]
[25949.423187]
[25949.423187] but task is already holding lock:
[25949.423187] ((eb-lock)-rlock){+.+...}, at: [a047cee2]
btrfs_clear_lock_blocking+0x22/0x30 [btrfs]
[25949.423187]
[25949.423187] which lock already depends on the new lock.
Kernel is 3.0.0 without any extras.
Ideas?
Did this actually deadlock? lockdep has issues with the btrfs
clear_lock_blocking code, and I need to redo the annotations a bit. The
problem is that we have the same lock class representing unrelated locks from
different trees.
It did not stop any processes that I could see and the rsync did complete ok.
Thats why I said possible.
Figured it might be something you needed to see and/or fix though.
Thanks
Ed
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 3.0 release - btrfs possible locking deadlock
On Thursday 21 July 2011 22:59:53 Linus Torvalds wrote:
So there it is. Gone are the 2.6.bignum days, and 3.0 is out.
Hi,
Managed to get this with btrfs rsync(ing) from ext4 to a btrfs fs with three
partitions using raid1.
[16018.211493] device fsid f7186eeb-60df-4b1a-890a-4a1eb42f81fe devid 1 transid
10 /dev/sdd4
[16018.230643] btrfs: use lzo compression
[16018.234619] btrfs: enabling disk space caching
[25949.414011]
[25949.414011] ===
[25949.416549] [ INFO: possible circular locking dependency detected ]
[25949.423187] 3.0.0-crc+ #348
[25949.423187] ---
[25949.423187] rsync/20237 is trying to acquire lock:
[25949.423187] (btrfs-extent-01){+.+...}, at: [a047ce88]
btrfs_try_spin_lock+0x78/0xb0 [btrfs]
[25949.423187]
[25949.423187] but task is already holding lock:
[25949.423187] ((eb-lock)-rlock){+.+...}, at: [a047cee2]
btrfs_clear_lock_blocking+0x22/0x30 [btrfs]
[25949.423187]
[25949.423187] which lock already depends on the new lock.
[25949.423187]
[25949.423187]
[25949.423187] the existing dependency chain (in reverse order) is:
[25949.423187]
[25949.423187] - #1 ((eb-lock)-rlock){+.+...}:
[25949.423187][8108bb75] lock_acquire+0x95/0x140
[25949.423187][815792eb] _raw_spin_lock+0x3b/0x50
[25949.423187][a047ce88] btrfs_try_spin_lock+0x78/0xb0 [btrfs]
[25949.423187][a0427959] btrfs_search_slot+0x2e9/0x800 [btrfs]
[25949.423187][a0433bee]
lookup_inline_extent_backref+0xbe/0x490 [btrfs]
[25949.423187][a0434cbb] __btrfs_free_extent+0x13b/0x900
[btrfs]
[25949.423187][a0435ca3] run_clustered_refs+0x823/0xaf0
[btrfs]
[25949.423187][a043603d] btrfs_run_delayed_refs+0xcd/0x290
[btrfs]
[25949.423187][a0445ecb] btrfs_commit_transaction+0x8b/0x9d0
[btrfs]
[25949.423187][a0440c06] transaction_kthread+0x2b6/0x2e0
[btrfs]
[25949.423187][81071536] kthread+0xb6/0xc0
[25949.423187][81582314] kernel_thread_helper+0x4/0x10
[25949.423187]
[25949.423187] - #0 (btrfs-extent-01){+.+...}:
[25949.423187][8108b468] __lock_acquire+0x1588/0x16a0
[25949.423187][8108bb75] lock_acquire+0x95/0x140
[25949.423187][815792eb] _raw_spin_lock+0x3b/0x50
[25949.423187][a047ce88] btrfs_try_spin_lock+0x78/0xb0 [btrfs]
[25949.423187][a0427959] btrfs_search_slot+0x2e9/0x800 [btrfs]
[25949.423187][a0439dd2] btrfs_lookup_dir_item+0x82/0x120
[btrfs]
[25949.423187][a04532a5] btrfs_lookup_dentry+0xc5/0x4c0
[btrfs]
[25949.423187][a04536c4] btrfs_lookup+0x24/0x70 [btrfs]
[25949.423187][8115a863] d_alloc_and_lookup+0xc3/0x100
[25949.423187][8115cfa0] do_lookup+0x260/0x480
[25949.423187][8115d540] walk_component+0x60/0x1f0
[25949.423187][8115e7aa] path_lookupat+0xea/0x620
[25949.423187][8115ed15] do_path_lookup+0x35/0x1c0
[25949.423187][8115fc38] user_path_at+0x98/0xe0
[25949.423187][81153fac] vfs_fstatat+0x4c/0x90
[25949.423187][8115405e] vfs_lstat+0x1e/0x20
[25949.423187][81154084] sys_newlstat+0x24/0x50
[25949.423187][815814eb] system_call_fastpath+0x16/0x1b
[25949.423187]
[25949.423187] other info that might help us debug this:
[25949.423187]
[25949.423187] Possible unsafe locking scenario:
[25949.423187]
[25949.423187]CPU0CPU1
[25949.423187]
[25949.423187] lock((eb-lock)-rlock);
[25949.423187]lock(btrfs-extent-01);
[25949.423187]lock((eb-lock)-rlock);
[25949.423187] lock(btrfs-extent-01);
[25949.423187]
[25949.423187] *** DEADLOCK ***
[25949.423187]
[25949.423187] 2 locks held by rsync/20237:
[25949.423187] #0: (sb-s_type-i_mutex_key#14){+.+.+.}, at:
[8115cf5a] do_lookup+0x21a/0x480
[25949.423187] #1: ((eb-lock)-rlock){+.+...}, at: [a047cee2]
btrfs_clear_lock_blocking+0x22/0x30 [btrfs]
[25949.423187]
[25949.423187] stack backtrace:
[25949.423187] Pid: 20237, comm: rsync Not tainted 3.0.0-crc+ #348
[25949.423187] Call Trace:
[25949.423187] [810887de] print_circular_bug+0x20e/0x2f0
[25949.423187] [8108b468] __lock_acquire+0x1588/0x16a0
[25949.423187] [a0441ebb] ? verify_parent_transid+0xcb/0x290 [btrfs]
[25949.423187] [a047ce88] ? btrfs_try_spin_lock+0x78/0xb0 [btrfs]
[25949.423187] [8108bb75] lock_acquire+0x95/0x140
[25949.423187] [a047ce88] ? btrfs_try_spin_lock+0x78/0xb0 [btrfs]
[25949.423187] [815792eb] _raw_spin_lock+0x3b/0x50
[25949.423187] [a047ce88] ? btrfs_try_spin_lock+0x78/0xb0 [btrfs]
Re: [GIT PULL] Btrfs updates for 2.6.35
On Monday 14 June 2010 20:47:35 Chris Mason wrote: On Mon, Jun 14, 2010 at 03:24:19PM -0400, Ed Tomlinson wrote: On Friday 11 June 2010 15:37:31 Chris Mason wrote: Hello everyone, The master branch of the btrfs-unstable tree is a collection of fixes and cleanups, including two btrfs regressions from rc1: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master One is an freeing blocks on an FS converted from ext34 to btrfs, and the other is a fallocate fix. The rest are the usual small bug fixes. Looks like this still misses any fix for the oops I reported May 8th in thread: [OPPS] btrfs on 33-3 with latest from btrfs-unstable.git master Any chance this could be looked into? I've kept the fs just in case. The oops shows a crc failure and then we were not able to read the tree block. Yan Zheng is working on an fsck that can repair things now. Until then the best I can do is help copy things off. Would you rather save the FS to test fsck? I can hang on to the FS for a while longer. One interesting point. The FS has both both meta-data and data mirroring enabled. I gather the oops is implying that both versions are corrupt. Btw the fs problem occured after heavy vm activity caused by running too many kvm(s). Thanks Ed -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 2.6.35
On Friday 11 June 2010 15:37:31 Chris Mason wrote: Hello everyone, The master branch of the btrfs-unstable tree is a collection of fixes and cleanups, including two btrfs regressions from rc1: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master One is an freeing blocks on an FS converted from ext34 to btrfs, and the other is a fallocate fix. The rest are the usual small bug fixes. Looks like this still misses any fix for the oops I reported May 8th in thread: [OPPS] btrfs on 33-3 with latest from btrfs-unstable.git master Any chance this could be looked into? I've kept the fs just in case. Thanks Ed Tomlinson Dan Carpenter (11) commits (+24/-17): Btrfs: handle error returns from btrfs_lookup_dir_item() (+2/-0) Btrfs: btrfs_read_fs_root_no_name() returns ERR_PTRs (+4/-0) Btrfs: unwind after btrfs_start_transaction() errors (+1/-1) Btrfs: remove unneeded null check in btrfs_rename() (+1/-3) Btrfs: The file argument for fsync() is never null (+1/-1) Btrfs: handle ERR_PTR from posix_acl_from_xattr() (+2/-0) Btrfs: btrfs_lookup_dir_item() can return ERR_PTR (+1/-1) Btrfs: uninitialized data is check_path_shared() (+1/-1) Btrfs: handle kzalloc() failure in open_ctree() (+5/-2) Btrfs: silence sparse warnings in ioctl.c (+4/-6) Btrfs: btrfs_iget() returns ERR_PTR (+2/-2) Zheng Yan (2) commits (+6/-4): Btrfs: Fix BUG_ON for fs converted from extN (+2/-1) Btrfs: Fix null dereference in relocation.c (+4/-3) Liu Bo (2) commits (+14/-4): Btrfs: Add error check for add_to_page_cache_lru (+13/-3) Btrfs: fix break in btrfs_insert_some_items() (+1/-1) Julia Lawall (2) commits (+9/-17): Btrfs: Use memdup_user (+6/-14) Btrfs: Use ERR_CAST (+3/-3) Shi Weihua (2) commits (+6/-0): Btrfs: prohibit a operation of changing acl's mask when noacl mount option used (+3/-0) Btrfs: should add a permission check for setfacl (+3/-0) Miao Xie (2) commits (+9/-1): Btrfs: fix loop device on top of btrfs (+1/-0) Btrfs: fix remap_file_pages error (+8/-1) Sage Weil (1) commits (+0/-3): Btrfs: avoid BUG when dropping root and reference in same transaction Andi Kleen (1) commits (+2/-94): BTRFS: Clean up unused variables -- nonbugs Josef Bacik (1) commits (+1/-1): Btrfs: fix fallocate regression Prarit Bhargava (1) commits (+1/-1): Btrfs: Fix warning in tree_search() Total: (25) commits (+72/-142) fs/btrfs/acl.c |8 fs/btrfs/compression.c | 18 +- fs/btrfs/ctree.c| 20 +--- fs/btrfs/disk-io.c | 22 +- fs/btrfs/extent-tree.c |5 ++--- fs/btrfs/extent_io.c|9 - fs/btrfs/extent_map.c |4 ++-- fs/btrfs/file.c | 12 ++-- fs/btrfs/inode.c| 22 +++--- fs/btrfs/ioctl.c| 36 fs/btrfs/ordered-data.c |4 +--- fs/btrfs/relocation.c |7 --- fs/btrfs/root-tree.c|5 - fs/btrfs/super.c| 14 +++--- fs/btrfs/tree-defrag.c |2 -- fs/btrfs/tree-log.c | 15 --- fs/btrfs/volumes.c |4 fs/btrfs/xattr.c|2 -- fs/btrfs/zlib.c |5 - 19 files changed, 72 insertions(+), 142 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[OPPS] btrfs on 33-3 with latest from btrfs-unstable.git master
] RIP [a04e42c1] btrfs_print_leaf+0x21/0x920 [btrfs] [ 2302.339860] RSP 88014b0cf698 [ 2302.339860] CR2: 0030 [ 2302.343138] ---[ end trace e68668d9a065c83e ]--- idea? TIA Ed Tomlinson -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [LOCKING] 2.6.33-rc8 btrfs vs java
On Monday 15 February 2010 10:56:53 Chris Mason wrote: On Sun, Feb 14, 2010 at 07:14:50PM -0500, Ed Tomlinson wrote: Hi, Found this in my log for 2.6.33-rc8. Figgured it might be interesting since .33 is close. The btrfs tree locking is very tricky for lockdep. I don't quite see how you could hit this one without also deadlocking the machine, unless it is a false positive. So, did you deadlock the machine? ;) No. So maybe its a false positive? Ed -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[LOCKING] 2.6.33-rc8 btrfs vs java
Hi,
Found this in my log for 2.6.33-rc8. Figgured it might be interesting since
.33 is close.
Thanks,
Ed
[102331.564869]
[102331.564871] ===
[102331.565770] [ INFO: possible circular locking dependency detected ]
[102331.565770] 2.6.32.8-crc #104
[102331.580954] ---
[102331.580954] java/8004 is trying to acquire lock:
[102331.580954] (btrfs-extent-01){+.+...}, at: [a04e9110]
btrfs_try_spin_lock+0x70/0xa0 [btrfs]
[102331.580954]
[102331.580954] but task is already holding lock:
[102331.580954] (eb-lock){+.+...}, at: [a04e9160]
btrfs_clear_lock_blocking+0x20/0x30 [btrfs]
[102331.580954]
[102331.580954] which lock already depends on the new lock.
[102331.580954]
[102331.580954]
[102331.580954] the existing dependency chain (in reverse order) is:
[102331.580954]
[102331.580954] - #1 (eb-lock){+.+...}:
[102331.580954][81091b95] __lock_acquire+0xfc5/0x1550
[102331.580954][810921bc] lock_acquire+0x9c/0x140
[102331.580954][814ac11b] _spin_lock+0x3b/0x50
[102331.580954][a04e9110] btrfs_try_spin_lock+0x70/0xa0
[btrfs]
[102331.580954][a04a016d] btrfs_search_slot+0x81d/0x860
[btrfs]
[102331.670534][a04b267f] btrfs_lookup_inode+0x2f/0xb0
[btrfs]
[102331.670534][a04bcd3e] btrfs_update_inode+0x6e/0x100
[btrfs]
[102331.670534][a04bd469] btrfs_dirty_inode+0x49/0x70 [btrfs]
[102331.670534][8115beeb] __mark_inode_dirty+0x3b/0x1a0
[102331.670534][8114e1fb] file_update_time+0xfb/0x180
[102331.706273][a04c7114] btrfs_file_write+0x384/0x920
[btrfs]
[102331.706273][811c] vfs_write+0x11c/0x1e0
[102331.706273][8113464a] sys_pwrite64+0xaa/0xb0
[102331.706273][8100ba1b] system_call_fastpath+0x16/0x1b
[102331.706273]
[102331.706273] - #0 (btrfs-extent-01){+.+...}:
[102331.737064][81091fe8] __lock_acquire+0x1418/0x1550
[102331.737064][810921bc] lock_acquire+0x9c/0x140
[102331.753738][814ac11b] _spin_lock+0x3b/0x50
[102331.753738][a04e9110] btrfs_try_spin_lock+0x70/0xa0
[btrfs]
[102331.753738][a04a016d] btrfs_search_slot+0x81d/0x860
[btrfs]
[102331.753738][a04b0fd3] btrfs_lookup_csum+0x93/0x160 [btrfs]
[102331.753738][a04b1a7d]
Re: Btrfs for mainline
On January 4, 2009, KOSAKI Motohiro wrote: Hi One possibility would be to mimic ext4 and register the fs as btrfsdev until it's considered stable enough for production. I agree with the consensus that we want to use the upstream kernel as a nexus for coordinating btrfs development, so I don't think it's worth waiting a release or two to merge something. I like this idea. I also want to test btrfs. but I'm not interested out of tree code. I'll second this. Please get btrfsdev into mainline asap. TIA Ed Tomlinson -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
