[PATCH 1/1] Btrfs: Explicitly include vmalloc.h in send.c
Certain architectures or platforms or combinations of CONFIG options require an explicit #include . Signed-off-by: Mitch Harder --- fs/btrfs/send.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index bf232c8..118e76d 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -25,6 +25,7 @@ #include #include #include +#include #include "send.h" #include "backref.h" -- 1.7.8.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/1] Btrfs: Explicitly include vmalloc.h in send.c
When compiling without SMP and generic x86_64, I encountered the following errors due to vmalloc.h not being implicitly included: CC fs/btrfs/send.o fs/btrfs/send.c: In function ‘fs_path_free’: fs/btrfs/send.c:185:4: error: implicit declaration of function ‘vfree’ fs/btrfs/send.c: In function ‘fs_path_ensure_buf’: fs/btrfs/send.c:215:4: error: implicit declaration of function ‘vmalloc’ fs/btrfs/send.c:215:12: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c:225:12: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c:233:13: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c: In function ‘iterate_dir_item’: fs/btrfs/send.c:900:10: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c:909:11: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c: In function ‘btrfs_ioctl_send’: fs/btrfs/send.c:4462:17: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c:4468:17: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c:4474:2: error: implicit declaration of function ‘vzalloc’ fs/btrfs/send.c:4474:20: warning: assignment makes pointer from integer without a cast fs/btrfs/send.c:4482:21: warning: assignment makes pointer from integer without a cast make[2]: *** [fs/btrfs/send.o] Error 1 make[1]: *** [fs/btrfs] Error 2 If it makes sense, please feel free to include this minor change in with other send/receive fixes. Mitch Harder (1): Btrfs: Explicitly include vmalloc.h in send.c fs/btrfs/send.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) -- 1.7.8.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs GPF in read_extent_buffer() while scrubbing with kernel 3.4.2
On Sat, Jul 28, 2012 at 03:08:47PM +0300, Sami Liedes wrote: > I have not yet figured out if this can be reproduced using a pristine, > smaller btrfs filesystem in raid-1 configuration inside KVM or if > there's something about my specific filesystem that causes this. I can > investigate that too; it's easier to do for me than the above testing, > as I don't need to have continuous physical access to the computer to > do that. It seems the bug doesn't happen with a new filesystem made with mkfs.btrfs -mraid1 -draid1 /dev/mapper/btrfs_crypt /dev/mapper/btrfs2_crypt and subsequently filled with data... Unfortunate. I started to wonder what's so special about the second device of my filesystem. All the errors always seem to come from that device. The only thing that comes to mind is that that device was not originally part of the filesystem; it started as a single-device filesystem formatted with mkfs.btrfs with default options and was subsequently rebalanced under a 3.4.2 kernel. So I started to play with btrfs fi balance in my KVM instance with my two-device filesystem, and hit this oops, which may or may not be related to my previous bug and/or dm-crypt... [ 15.195342] device fsid a844eb60-eb9c-4e24-ae91-c1627bf2d439 devid 1 transid 176 /dev/mapper/btrfs_crypt [ 15.196606] device fsid a844eb60-eb9c-4e24-ae91-c1627bf2d439 devid 2 transid 176 /dev/mapper/btrfs2_crypt [ 15.197895] device fsid a844eb60-eb9c-4e24-ae91-c1627bf2d439 devid 1 transid 176 /dev/mapper/btrfs_crypt [ 15.200202] btrfs: disk space caching is enabled # btrfs device delete [something...] [ 1462.242456] btrfs: unable to go below two devices on raid1 # btrfs fi balance start -mconvert=dup -dconvert=raid0 /media [ 1895.048075] btrfs: unable to start balance with target metadata profile 32 # btrfs fi balance start -dconvert=raid0 /media [ 1917.106536] btrfs: relocating block group 10229907456 flags 17 [ 1929.188609] btrfs: relocating block group 8887730176 flags 17 [ 1944.690916] btrfs: found 2152 extents [ 1947.016210] btrfs: found 2152 extents [ 1947.421397] btrfs: relocating block group 7545552896 flags 17 [ 2024.225203] btrfs: found 36762 extents [ 2094.983055] btrfs: corrupt node block=8830455808,root=1: generation (197) too new in slot 0 (maximum expected 196) [ 2094.984858] [ cut here ] [ 2094.986076] WARNING: at fs/btrfs/super.c:219 __btrfs_abort_transaction+0xa5/0xc0() [ 2094.987912] Hardware name: Bochs [ 2094.988735] btrfs: Transaction abortedPid: 1623, comm: btrfs-transacti Not tainted 3.4.4 #9 [ 2094.988741] Call Trace: [ 2094.989361] [] warn_slowpath_common+0x75/0xb0 [ 2094.990829] [] warn_slowpath_fmt+0x41/0x50 [ 2094.992243] [] __btrfs_abort_transaction+0xa5/0xc0 [ 2094.993521] [] __btrfs_free_extent+0x213/0x7b0 [ 2094.994176] [] ? run_clustered_refs+0xd7/0xa10 [ 2094.994835] [] ? do_raw_spin_unlock+0x5d/0xb0 [ 2094.995481] [] run_clustered_refs+0x415/0xa10 [ 2094.996143] [] ? find_ref_head+0xb8/0xe0 [ 2094.996806] [] ? btrfs_run_delayed_refs+0x99/0x430 [ 2094.997505] [] btrfs_run_delayed_refs+0x16d/0x430 [ 2094.998188] [] ? next_block_group.isra.65+0x29/0x80 [ 2094.998890] [] ? _raw_spin_unlock+0x26/0x30 [ 2094.999520] [] btrfs_write_dirty_block_groups+0xb0/0x630 [ 2095.000281] [] ? free_extent_buffer+0x1a/0x70 [ 2095.000931] [] commit_cowonly_roots+0xe7/0x1c1 [ 2095.001610] [] btrfs_commit_transaction+0x519/0xa40 [ 2095.002317] [] ? abort_exclusive_wait+0xb0/0xb0 [ 2095.002997] [] transaction_kthread+0x245/0x2c0 [ 2095.003673] [] ? check_leaf.isra.105+0x300/0x300 [ 2095.004372] [] kthread+0x8e/0xa0 [ 2095.004902] [] kernel_thread_helper+0x4/0x10 [ 2095.005549] [] ? kthread_flush_work_fn+0x10/0x10 [ 2095.006222] [] ? gs_change+0x13/0x13 [ 2095.006787] ---[ end trace 8341f112debcf176 ]--- [ 2095.007287] BTRFS error (device dm-1) in __btrfs_free_extent:5134: IO failure [ 2095.008059] btrfs is forced readonly [ 2095.008454] btrfs: run_one_delayed_ref returned -5 [ 2095.008455] BTRFS error (device dm-1) in btrfs_run_delayed_refs:2454: IO failure [ 2176.876382] [ cut here ] [ 2176.877217] kernel BUG at fs/btrfs/relocation.c:3733! [ 2176.878139] invalid opcode: [#1] SMP [ 2176.879049] CPU 5 [ 2176.879435] Pid: 1676, comm: btrfs Tainted: GW3.4.4 #9 Bochs Bochs [ 2176.880383] RIP: 0010:[] [] relocate_block_group+0x643/0x690 [ 2176.880383] RSP: :880002bfdb08 EFLAGS: 00010206 [ 2176.880383] RAX: ffe2 RBX: ffe2 RCX: 880007aade10 [ 2176.880383] RDX: RSI: RDI: 880006fbe800 [ 2176.880383] RBP: 880002bfdb88 R08: 0003 R09: [ 2176.880383] R10: 000100072905 R11: 0001 R12: 8800037a8020 [ 2176.880383] R13: 5ee7 R14: 880006bbae10 R15: 8800037a8000 [ 2176.880383] FS: () GS:880007d4(0063) knlGS:0
Re: No SysRq remount because sb->s_bdev is NULL
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28.07.2012 16:50, Florian Albrechtskirchinger wrote: > On Saturday, July 28, 2012 15:46:50 Andreas Philipp wrote: >> On 28.07.2012 15:41, Florian Albrechtskirchinger wrote: >>> During a SysRq emergency remount Btrfs mounts are not remounted. I tracked >>> the issue down to this line in do_emergency_remount() in fs/super.c: >>> if (sb->s_root && sb->s_bdev && (sb->s_flags & MS_BORN) && >>> !(sb->s_flags & MS_RDONLY)) >>> s_bdev is NULL for Btrfs super blocks and subsequently do_remount_sb() is >>> never called. I couldn't think of a solution, besides resorting to an ugly >>> strcmp(sb->s_type->name, "btrfs"), without adding a field to struct >>> super_block or similar changes. >>> Thoughts? >> >> Just a first thought. Is there a possibility to write a dummy value into >> sb->s_bdev for btrfs super blocks. Thus it will not be NULL and >> everything in do_emergency_remount() in fs/super.c will work as wanted. > > Then other code paths testing sb->s_bdev for NULL and assuming a non-NULL sb- >> s_bdev is a valid block device would fail. Sorry, I thought that there are separate checks whether the value of sb->s_bdev is a valid block device. Thanks, Andreas >> > > Flo -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJQFBOrAAoJEIW3W1BiBxU7xCsH/RzDyJR7MneXLtDBQIQC43XS +jdaMrUIjEjBhs390iJJWebg5H6690XEO/wSM36ZMkHi0fSeJ7dC4WKjTtLyTkAB ok2+1Qix2LcqWxGSr9TC5OAjli3A8bl2Iy+P3bObfVOYnaTS781+ALl2PopiyZva ineLrXzGiUQoT059BK456ckceNkQ31YTbuwAEVP85ZdC4Bk8UviUYdoVtsFBb2aH 4UnHgnGdrPhz6BqIbxhrfTLNsNbnGskQ8MrxogRChv+P1SRKHslBKTLCvXugH5t4 iwrXtm1gtpFDWkmiMfYN6HpV/vGIrqstUns7zj/kYVbC2c0u4QGBSZLhy9S7UF4= =1I+D -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No SysRq remount because sb->s_bdev is NULL
On Saturday, July 28, 2012 15:46:50 Andreas Philipp wrote: > On 28.07.2012 15:41, Florian Albrechtskirchinger wrote: > > During a SysRq emergency remount Btrfs mounts are not remounted. I tracked > > the issue down to this line in do_emergency_remount() in fs/super.c: > > if (sb->s_root && sb->s_bdev && (sb->s_flags & MS_BORN) && > > !(sb->s_flags & MS_RDONLY)) > > s_bdev is NULL for Btrfs super blocks and subsequently do_remount_sb() is > > never called. I couldn't think of a solution, besides resorting to an ugly > > strcmp(sb->s_type->name, "btrfs"), without adding a field to struct > > super_block or similar changes. > > Thoughts? > > Just a first thought. Is there a possibility to write a dummy value into > sb->s_bdev for btrfs super blocks. Thus it will not be NULL and > everything in do_emergency_remount() in fs/super.c will work as wanted. Then other code paths testing sb->s_bdev for NULL and assuming a non-NULL sb- >s_bdev is a valid block device would fail. Flo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No SysRq remount because sb->s_bdev is NULL
Hi, Just a first thought. Is there a possibility to write a dummy value into sb->s_bdev for btrfs super blocks. Thus it will not be NULL and everything in do_emergency_remount() in fs/super.c will work as wanted. Thanks, Andreas On 28.07.2012 15:41, Florian Albrechtskirchinger wrote: > Hi, > > During a SysRq emergency remount Btrfs mounts are not remounted. I tracked > the > issue down to this line in do_emergency_remount() in fs/super.c: > if (sb->s_root && sb->s_bdev && (sb->s_flags & MS_BORN) && > !(sb->s_flags & MS_RDONLY)) > s_bdev is NULL for Btrfs super blocks and subsequently do_remount_sb() is > never called. I couldn't think of a solution, besides resorting to an ugly > strcmp(sb->s_type->name, "btrfs"), without adding a field to struct > super_block > or similar changes. > Thoughts? > > Flo > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
No SysRq remount because sb->s_bdev is NULL
Hi, During a SysRq emergency remount Btrfs mounts are not remounted. I tracked the issue down to this line in do_emergency_remount() in fs/super.c: if (sb->s_root && sb->s_bdev && (sb->s_flags & MS_BORN) && !(sb->s_flags & MS_RDONLY)) s_bdev is NULL for Btrfs super blocks and subsequently do_remount_sb() is never called. I couldn't think of a solution, besides resorting to an ugly strcmp(sb->s_type->name, "btrfs"), without adding a field to struct super_block or similar changes. Thoughts? Flo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs GPF in read_extent_buffer() while scrubbing with kernel 3.4.2
On Tue, Jul 17, 2012 at 12:29:33AM +0300, Sami Liedes wrote: > So, currently my idea is to boot the machine with a live USB stick, > install kvm and make qemu qcow images backed by the real (2*1.1T) > devices, but writing changes to the qcow images (I dare not mess with > the actual devices, and don't happen to have quite 2.2T extra space > outside of them...), and try to run scrub there. If that succeeds and > the bug happens there too, debugging *should* be easier, and it > *should* be possible to run it under KMEMCHECK too. If the bug doesn't > happen inside a virtual machine, that would be interesting information > too. I have now been able to reproduce the bug in KVM with the setup described above. I think it's safe to say now that the bug depends on some kind of interaction between btrfs and dm-crypt. With the following setup, the bug does NOT happen: * kvm, single cpu * sees 3 disks, /dev/vda=root, /dev/vdb=btrfs-dev1, /dev/vdc=btrfs-dev2 * The btrfs devices are essentially snapshots of the real btrfs devices in raid-1 configuration (2*1.1T). As the real devices are encrypted, the decryption is done outside the KVM, i.e. the KVM snapshots are backed by the decrypted devices. With the following setup, the bug DOES happen: * kvm, single cpu * sees 3 disks, /dev/vda=root, /dev/vdb=part1, /dev/vdc=part2, where part[12] is are LUKS containers containing the individual btrfs devices * inside kvm, they are opened using cryptsetup luksOpen /dev/vdb root1 cryptsetup luksOpen /dev/vdc root2 * after this, the filesystem is mounted with mount /dev/mapper/root1 /media -o device=/dev/mapper/root1,device=/dev/mapper/root2 * The devices are snapshots of the actual physical encrypted partitions containing the btrfs devices. I have not yet figured out if this can be reproduced using a pristine, smaller btrfs filesystem in raid-1 configuration inside KVM or if there's something about my specific filesystem that causes this. I can investigate that too; it's easier to do for me than the above testing, as I don't need to have continuous physical access to the computer to do that. Here's the .config of the kernel I used inside KVM to reproduce this: http://www.niksula.hut.fi/~sliedes/btrfs/config.3.4.4 I also ran the same tests with KMEMCHECK. Both with and without crypto, there were quite a number of (of course possibly false) warnings from btrfs code. I doubt any of them are related to this bug - there were no KMEMCHECK warnings during the scrub operation. Here are the logs, anyway: http://www.niksula.hut.fi/~sliedes/btrfs/screenlog.nocrypto.gz http://www.niksula.hut.fi/~sliedes/btrfs/screenlog.crypto.gz Sami signature.asc Description: Digital signature
Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent
On Fri, Jul 27, 2012 at 4:37 PM, Alex Lyakas
wrote:
> Hi Alexander,
> your solution is simple and elegant. I this this issue is solved now. Thanks!
> Two minor issues:
> 1)
> /*
> * We need some special handling for inodes that get processed before the
> parent
> * directory got created. See process_all_refs for details.
> * This function does the check if we already created the dir out of order.
> */
> /*
> * Only creates the inode if it is:
> * 1. Not a directory
> * 2. Or a directory which was not created already due to out of order
> *directories. See did_create_dir and process_all_refs for details.
> */
>
> These comments tell to look at process_all_refs(), while we should
> look at process_recorded_refs().
>
> 2)
> * We can however not delete the orphan in case the inode
> relies
> * in a directory that was not created yet (see
> * __record_new_ref)
> */
> This part should be removed, because your new solution does not work this way.
Whoops...corrected all comments.
>
>
> If you find, time, pls look at the two attached scripts.
>
> btrfs_test_1.sh:
> it tries to explore the is_first_ref() issue and founds a problem.
> Proposed patch - compare also the (dir,gen) tuple and only the name:
>
> diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
> index 68e504c..b83ec5f 100644
> --- a/fs/btrfs/send.c
> +++ b/fs/btrfs/send.c
> @@ -1597,7 +1597,7 @@ out:
>
> static int is_first_ref(struct send_ctx *sctx,
> struct btrfs_root *root,
> - u64 ino, u64 dir,
> + u64 ino, u64 dir, u64 dir_gen,
> const char *name, int name_len)
> {
> int ret;
> @@ -1613,6 +1613,11 @@ static int is_first_ref(struct send_ctx *sctx,
> if (ret < 0)
> goto out;
>
> + if (dir != tmp_dir || dir_gen != tmp_dir_gen) {
> + ret = 0;
> + goto out;
> + }
> +
> if (name_len != fs_path_len(tmp_name)) {
> ret = 0;
> goto out;
> @@ -2834,7 +2839,7 @@ verbose_printk("btrfs: process_recorded_refs
> %llu\n", sctx->cur_ino);
> goto out;
> if (ret) {
> ret = is_first_ref(sctx, sctx->parent_root,
> - ow_inode, cur->dir, cur->name,
> + ow_inode, cur->dir,
> cur->dir_gen, cur->name,
> cur->name_len);
> if (ret < 0)
> goto out;
>
I did not apply the patch but instead added a check for dir != tmp_dir
only. The reason to not check for gen is that I have a rule in my
mind: I only pass the generation number to functions where I want to
know the *current* state. is_first_ref is for permanent state, the
return value never changes while sending. I could however not
reproduce the problem with test_1.sh, but it should fix it.
>
> btrfs_test_2.sh
> The last test exposes an interesting issue: when a directory has a
> deleted reference recorded, this deleted reference is not added to the
> 'check_dirs' list. As a result, the upper directory (already
> orphanized) is not rmdir'd.
You'll find a commit in my repo to fix this. The problem was, that
moved directories do not get into the delete_refs loop and thus the
parent of the old location is never added to the check_dirs list.
I have force pushed to for-alex, if you have time I'd be happy if you
test again :)
>
> Thanks,
> Alex.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
