[PATCH v2] Btrfs: fix a spinlock warning when cleaning up aborted transaction

2014-02-07 Thread Liu Bo
Given now we have 2 spinlock for management of delayed refs, CONFIG_DEBUG_SPINLOCK=y helped me find this, [ 4723.413809] BUG: spinlock wrong CPU on CPU#1, btrfs-transacti/2258 [ 4723.414882] lock: 0x880048377670, .magic: dead4ead, .owner: btrfs-transacti/2258, .owner_cpu: 2 [ 4723.417146] CP

[PATCH] Btrfs: fix a lockdep warning when cleaning up aborted transaction

2014-02-07 Thread Liu Bo
Given now we have 2 spinlock for management of delayed refs, CONFIG_DEBUG_SPINLOCK=y helped me find this, [ 4723.413809] BUG: spinlock wrong CPU on CPU#1, btrfs-transacti/2258 [ 4723.414882] lock: 0x880048377670, .magic: dead4ead, .owner: btrfs-transacti/2258, .owner_cpu: 2 [ 4723.417146] CP

Re: Are nocow files snapshot-aware

2014-02-07 Thread Duncan
Kai Krakow posted on Fri, 07 Feb 2014 23:26:34 +0100 as excerpted: > So the question is: Do btrfs snapshots give the same guarantees on the > filesystem level that write-barriers give on the storage level which > exactly those processes rely upon? The cleanest solution would be if > processes coul

Re: [PATCH] Btrfs: convert to add transaction protection for btrfs send

2014-02-07 Thread Wang Shilong
Hi Josef, [..SNIP..] >> Yeah, very reasonable, if you don't mind, i would give a patch for this >> issue. > Go for it, you'll be faster than I will be, all I do is run xfstests and > try to reproduce things that will never reproduce for me. So Finally, i've figured everything out. Previous sen

Re: Can you keep reflink relationship during a copy/backup to another filesystem?

2014-02-07 Thread Marc MERLIN
On Wed, Jan 29, 2014 at 08:05:14AM +, Hugo Mills wrote: > On Tue, Jan 28, 2014 at 11:50:25PM -0800, Marc MERLIN wrote: > > So I used to use hardlinks to do historical backups of the same filesystem > > but I know it's preferable to use refllink with btrfs to avoid having too > > many hardlinks.

Re: [PATCH] xfstests: Btrfs: add test for large metadata blocks

2014-02-07 Thread Dave Chinner
On Fri, Feb 07, 2014 at 06:14:45PM +0100, Koen De Wit wrote: > Tests Btrfs filesystems with all possible metadata block sizes, by > setting large extended attributes on files. > > Signed-off-by: Koen De Wit There's a few things here that need fixing. > +pagesize=`$here/src/feature -s` > +pagesi

Re: Are nocow files snapshot-aware

2014-02-07 Thread Kai Krakow
Chris Murphy schrieb: > > On Feb 7, 2014, at 2:07 PM, Kai Krakow wrote: > >> Chris Murphy schrieb: >> If the database/virtual machine/whatever is crash safe, then the atomic state that a snapshot grabs will be useful. >>> >>> How fast is this state fixed on disk from the time of t

btrfs snapshot is killing IO and hanging my device with delayed writes for 10mn+

2014-02-07 Thread Marc MERLIN
On my workstation, which unfortunately I can't easily upgrade the kernel on, so it's running 3.8.0 for now, I've had pretty perpelexing hangs from btrfs snapshot when I make a new snapshot every hour. I see the btrfs snapshot command in iotop for over a minute, and it completes eventually. Other

Re: Are nocow files snapshot-aware

2014-02-07 Thread Kai Krakow
Duncan <1i5t5.dun...@cox.net> schrieb: >> The question here is: Does it really make sense to create such snapshots >> of disk images currently online and running a system. They will probably >> be broken anyway after rollback - or at least I'd not fully trust the >> contents. >> >> VM images shou

Re: Are nocow files snapshot-aware

2014-02-07 Thread Chris Murphy
On Feb 7, 2014, at 2:07 PM, Kai Krakow wrote: > Chris Murphy schrieb: > >>> If the database/virtual machine/whatever is crash safe, then the >>> atomic state that a snapshot grabs will be useful. >> >> How fast is this state fixed on disk from the time of the snapshot >> command? Loosely spea

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread David Rientjes
On Fri, 7 Feb 2014, Fengguang Wu wrote: > On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: > > On Fri, 7 Feb 2014, Fengguang Wu wrote: > > > > > [1.625020] BTRFS: selftest: Running btrfs_split_item tests > > > [1.627004] BTRFS: selftest: Running find delalloc tests > > > [

Re: Are nocow files snapshot-aware

2014-02-07 Thread Kai Krakow
Chris Murphy schrieb: >> If the database/virtual machine/whatever is crash safe, then the >> atomic state that a snapshot grabs will be useful. > > How fast is this state fixed on disk from the time of the snapshot > command? Loosely speaking. I'm curious if this is < 1 second; a few > seconds;

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Kai Krakow
Josef Bacik schrieb: > > On 02/05/2014 03:15 PM, Roman Mamedov wrote: >> Hello, >> >> On a freshly-created RAID1 filesystem of two 1TB disks: >> >> # df -h /mnt/p2/ >> Filesystem Size Used Avail Use% Mounted on >> /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 >> >> I cannot write 2TB of us

[PATCH] Btrfs: don't loop forever if we can't run because of the tree mod log

2014-02-07 Thread Josef Bacik
A user reported a 100% cpu hang with my new delayed ref code. Turns out I forgot to increase the count check when we can't run a delayed ref because of the tree mod log. If we can't run any delayed refs during this there is no point in continuing to look, and we need to break out. Thanks, Signe

[PATCH] btrfs-progs: Preserve process_one_leaf return value.

2014-02-07 Thread Mitch Harder
The return value in process_one_leaf could be over-written while looping over the items in the leaf. This patch will preserve a non-zero return value to the calling function if a non-zero return value is encountered in the loop. The return value of one (1) is consistent with non-zero values that

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Chris Murphy
On Feb 6, 2014, at 11:08 PM, Roman Mamedov wrote: > And what > if I am accessing that partition on a server via a network CIFS/NFS share and > don't even *have a way to find out* any of that. That's the strongest argument. And if the user is using Explorer/Finder/Nautilus to copy files to the

Re: integration-20140206 build failure

2014-02-07 Thread WorMzy Tykashi
> > Thanks. A mismerged patch, fixed in next integration. Cheers. I did briefly get the following failure from integration-20140207: [CC] cmds-filesystem.o cmds-filesystem.c: In function 'handle_print': cmds-filesystem.c:434:5: error: too few arguments to function 'pr

Re: integration-20140206 build failure

2014-02-07 Thread David Sterba
On Fri, Feb 07, 2014 at 12:38:22AM +, WorMzy Tykashi wrote: > Building the latest integration snapshot, I get the following failure: > > > ... > [CC] cmds-filesystem.o > cmds-filesystem.c: In function 'cmd_show': > cmds-filesystem.c:740:12: error: invalid storage class for functio

Re: lost with degraded RAID1

2014-02-07 Thread Chris Murphy
On Feb 7, 2014, at 4:34 AM, Johan Kröckel wrote: > Is there anything else I should do with this setup or may I nuke the > two partitions and reuse them? Well I'm pretty sure once you run 'btrfs check --repair' that you've hit the end of the road. Possibly btrfs restore can still extract some f

[PATCH] Btrfs: unlock extent and pages on error in cow_file_range

2014-02-07 Thread Josef Bacik
When I converted the BUG_ON() for the free_space_cache_inode in cow_file_range I made it so we just return an error instead of unlocking all of our various stuff. This is a mistake and causes us to hang when we run into this. This patch fixes this problem. Thanks, Signed-off-by: Josef Bacik --

[PATCH] xfstests: Btrfs: add test for large metadata blocks

2014-02-07 Thread Koen De Wit
Tests Btrfs filesystems with all possible metadata block sizes, by setting large extended attributes on files. Signed-off-by: Koen De Wit --- tests/btrfs/036 | 128 +++ tests/btrfs/036.out |7 +++ tests/btrfs/group |1 + 3 files chan

[PATCH] btrfs-progs: add dry-run option to restore command

2014-02-07 Thread Justin Maggard
Sometimes it is useful to see what btrfs restore is going to do before provisioning enough external storage to restore onto. Add a dry-run option so we can see what files and paths are found by restore, without actually restoring any data. --- cmds-restore.c | 14 -- 1 file changed,

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread Filipe David Manana
On Fri, Feb 7, 2014 at 3:10 PM, Chris Mason wrote: > On Fri 07 Feb 2014 07:10:38 AM EST, Fengguang Wu wrote: >> >> On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: >>> >>> On Fri, 7 Feb 2014, Fengguang Wu wrote: >>> [1.625020] BTRFS: selftest: Running btrfs_split_item tests

[PATCH] btrfs-progs: Change BUG() to use assert.

2014-02-07 Thread Mitch Harder
Change the definition of BUG() to use assert instead of abort to provide information about the location of the issue. Signed-off-by: Mitch Harder --- kerncompat.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kerncompat.h b/kerncompat.h index 1fc2b34..f370cd8 100644 --- a/k

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread Chris Mason
On Fri 07 Feb 2014 07:10:38 AM EST, Fengguang Wu wrote: On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: On Fri, 7 Feb 2014, Fengguang Wu wrote: [1.625020] BTRFS: selftest: Running btrfs_split_item tests [1.627004] BTRFS: selftest: Running find delalloc tests [2.2891

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Frank Kingswood
On 06/02/14 19:54, Goffredo Baroncelli wrote: Hi Roman On 02/06/2014 01:45 PM, Roman Mamedov wrote: There's not a lot of code to include (as my 3-line patch demonstrates), it could just as easily be removed when it's obsolete. But I did not have any high hopes of defeating the "broken by design

[PATCH v4] xfstests/btrfs: add a regression test for running snapshot and send concurrently

2014-02-07 Thread Wang Shilong
From: Wang Shilong Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug. Signed-off-by: Wang Shilong --- v3->v4: avoid to use silent send command(thanks to Dave Chinner) rename patch's subject from Btrfs to xfstest/btrfs(thanks to Anand) m

[PATCH 2/4] btrfs: reserve no transaction units in btrfs_ioctl_set_features

2014-02-07 Thread David Sterba
Added in patch "btrfs: add ioctls to query/change feature bits online" modifications to superblock don't need to reserve metadata blocks when starting a transaction. Signed-off-by: David Sterba --- fs/btrfs/ioctl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/ioct

[PATCH 3/4][RFC] btrfs: export global block reserve size as space_info

2014-02-07 Thread David Sterba
Introduce a block group type bit for a global reserve and fill the space info for SPACE_INFO ioctl. This should replace the newly added ioctl (01e219e8069516cdb98594d417b8bb8d906ed30d) to get just the 'size' part of the global reserve, while the actual usage can be now visible in the 'btrfs fi df'

[PATCH 1/4] btrfs: commit transaction after setting label and features

2014-02-07 Thread David Sterba
From: Jeff Mahoney The set_fslabel ioctl uses btrfs_end_transaction, which means it's possible that the change will be lost if the system crashes, same for the newly set features. Let's use btrfs_commit_transaction instead. Signed-off-by: Jeff Mahoney Signed-off-by: David Sterba --- fs/btrfs/

[PATCH 4/4] Revert "btrfs: add ioctl to export size of global metadata reservation"

2014-02-07 Thread David Sterba
This reverts commit 01e219e8069516cdb98594d417b8bb8d906ed30d. The information from the removed ioctl is newly provided by the existing SPACE_INFO ioctl where it logically belongs and needs only minor changes to the userspace code. The GLOBAL_RSV ioctl hasn't been used anywhere yet, introduced onl

[PATCH 0/4] Btrfs updates for 3.14-rc - sysfs, ioctl

2014-02-07 Thread David Sterba
Two minor fixes to the new sysfs code and a RFC change to replace the GLOBAL_RSV ioctl with the same information exported through SPACE_INFO ioctl. This requires to reserve a blockgroup type that will become part of ABI. David Sterba (3): btrfs: reserve no transaction units in btrfs_ioctl_set_f

Can i trust you?

2014-02-07 Thread ivan . zoric
I am Mrs. Aisha Al Mutasim from England, I have a business proposal worth of $10.6M US Dollars get back for more info. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/ma

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread Fengguang Wu
On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: > On Fri, 7 Feb 2014, Fengguang Wu wrote: > > > [1.625020] BTRFS: selftest: Running btrfs_split_item tests > > [1.627004] BTRFS: selftest: Running find delalloc tests > > [2.289182] tsc: Refined TSC clocksource calibration

Re: lost with degraded RAID1

2014-02-07 Thread Johan Kröckel
Is there anything else I should do with this setup or may I nuke the two partitions and reuse them? 2014-02-03 Johan Kröckel : > State is: I wont use this filesystem again. I have a backup. So I am > interested to give the necessary information for debuging it and > afterwards format it and create

Re: [PATCH v3] Btrfs: add regression test for running snapshot and send concurrently

2014-02-07 Thread Wang Shilong
Hi Dave, > On Fri, Feb 07, 2014 at 12:18:31PM +0800, Wang Shilong wrote: >> >> Hi Dave, >> >>> On Fri, Feb 07, 2014 at 12:10:08AM +0800, Wang Shilong wrote: +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \ + $SCRATCH_MNT/snap_1 >> $seqres.full 2>&1 + +do_snapshots & >

[PATCH v3 3/3] btrfs/035: add new clone overwrite regression test

2014-02-07 Thread David Disseldorp
This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests. Signed-off-by: David Disseldorp --- tests/btrfs/035 | 77 + tests/btrfs/035.out | 3 +++ tests/btrfs/group | 1 + 3 files ch

[PATCH v3 0/3] __btrfs_drop_extents() BUG_ON reproducer

2014-02-07 Thread David Disseldorp
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON() at: 692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans, ... 839 /* 840 * | range to drop - | 841 * | extent | 842

[PATCH v3 1/3] btrfs: add small program for clone testing

2014-02-07 Thread David Disseldorp
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls. Signed-off-by: David Disseldorp --- .gitignore | 1 + src/Makefile | 2 +- src/cloner.c | 188 +++ 3 files changed, 190 inserti

[PATCH v3 2/3] src/cloner: use btrfs/ioctl.h header if present

2014-02-07 Thread David Disseldorp
Check for the btrfsprogs-devel ioctl.h header at configure time. Use it in src/cloner if present, otherwise fall back to using the copied clone ioctl definitions. Signed-off-by: David Disseldorp --- configure.ac | 1 + src/cloner.c | 4 2 files changed, 5 insertions(+) diff --git a/configu

Re: [PATCH v2 3/3] btrfs/035: add new clone overwrite regression test

2014-02-07 Thread David Disseldorp
On Fri, 7 Feb 2014 09:45:09 +1100, Dave Chinner wrote: > Not exactly what I intended. > > If echo fails, it will output some kind of error message, and that > will cause the golden image mismatch. Fair enough. I'll resend with the two ' || echo "failed to create src"' checks removed. Thanks aga

Re: [PATCH] btrfs: introduce BTRFS_IOC_GET_DEVS

2014-02-07 Thread Hugo Mills
On Fri, Feb 07, 2014 at 06:08:56PM +0800, Anand Jain wrote: > > > Thanks for the comments. > > mainly here sysfs way defeats the purpose - debug as > mentioned. Sysfs would/should show only mounted disks, So let's find a way of showing the "known-about" data structure separately from the

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread David Rientjes
On Fri, 7 Feb 2014, Fengguang Wu wrote: > [1.625020] BTRFS: selftest: Running btrfs_split_item tests > [1.627004] BTRFS: selftest: Running find delalloc tests > [2.289182] tsc: Refined TSC clocksource calibration: 2299.967 MHz > [ 292.084537] kthreadd invoked oom-killer: gfp_mask=0x30

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Martin Steigerwald
Am Donnerstag, 6. Februar 2014, 22:30:46 schrieb Chris Murphy: > On Feb 6, 2014, at 9:40 PM, Roman Mamedov wrote: > > On Thu, 06 Feb 2014 20:54:19 +0100 > > > > Goffredo Baroncelli wrote: > > > > > >> I agree with you about the needing of a solution. However your patch to > >> me seems even wors

Re: [PATCH] btrfs: introduce BTRFS_IOC_GET_DEVS

2014-02-07 Thread Anand Jain
Thanks for the comments. mainly here sysfs way defeats the purpose - debug as mentioned. Sysfs would/should show only mounted disks, the ioctl way doesn't have such a limitation (of course as I commented memory dump way would have been best choice but I got stuck with that approach if any

Re: [PATCH 2/2] Btrfs: fix use-after-free in the finishing procedure of the device replace

2014-02-07 Thread Stefan Behrens
On Fri, 07 Feb 2014 17:17:45 +0800, Miao Xie wrote: > On Fri, 07 Feb 2014 09:14:42 +0100, Stefan Behrens wrote: >> On Thu, 30 Jan 2014 16:46:55 +0800, Miao Xie wrote: >>> During device replace test, we hit a null pointer deference (It was very >>> easy >>> to reproduce it by running xfstests' btrf

Re: [PATCH 2/2] Btrfs: fix use-after-free in the finishing procedure of the device replace

2014-02-07 Thread Miao Xie
On Fri, 07 Feb 2014 09:14:42 +0100, Stefan Behrens wrote: > On Thu, 30 Jan 2014 16:46:55 +0800, Miao Xie wrote: >> During device replace test, we hit a null pointer deference (It was very easy >> to reproduce it by running xfstests' btrfs/011 on the devices with the virtio >> scsi driver). There we

Re: [PATCH 1/2] btrfs-progs: Add missing devices check for mounted btrfs.

2014-02-07 Thread Anand Jain
IMO btrfs-progs shouldn't add its intelligence to know if disk is missing. If btrfs-kernel doesn't know when disk is missing that's a bug to fix in btrfs-kernel. yes that indeed true as of now in btrfs-kernel. btrfs kernel has no idea when disk goes missing, just -EIO doesn't tell btrfs tha

Re: [PATCH 2/2] btrfs-progs: Add -p/--print-missing options for btrfs fi show

2014-02-07 Thread Anand Jain
Whats needed is more comprehensive btrfs fi show which shows the flags (including missing) per disk. And also show the FS/Raid status. Which I am working on. sorry -p feature would be covered by default in the coming revamp of btrfs fi show. Thanks, Anand On 02/07/2014 02:46 PM, Qu Wenru

Re: [PATCH 2/2] Btrfs: fix use-after-free in the finishing procedure of the device replace

2014-02-07 Thread Stefan Behrens
On Thu, 30 Jan 2014 16:46:55 +0800, Miao Xie wrote: > During device replace test, we hit a null pointer deference (It was very easy > to reproduce it by running xfstests' btrfs/011 on the devices with the virtio > scsi driver). There were two bugs that caused this problem: > - We might allocate new