Re: [review] Btrfs: Allow shrinking close to used space
2009/2/26 Josef Bacik : > On Wed, Feb 25, 2009 at 02:13:10PM -0500, Chris Ball wrote: >> Hi, >> >> This patch (against experimental HEAD) attempts to make shrinking more >> robust, by only updating device size if we've succeeded in creating >> enough free space without any failures in btrfs_relocate_chunk(). >> >> Here's a log with my patch applied. The two things to note are that a >> near-limit shrink now works, and that a failed shrink (in this case, >> trying to shrink to less than the used space) no longer updates the >> device size erroneously: >> >> http://dev.laptop.org/~cjb/btrfs/shrink-log >> >> Please review carefully -- I'm still new to btrfs. The short version of >> the patch is: >> >> * create a success path, as a break out of the while(1) relocating >> (rather than going to the "done" label). >> * move the device size updating code into that path >> * leave "path->reada = 2;" behind in the entry path, since path is >> used by the searching operation rather than the later resize. >> >> Thanks! >> >> Signed-off-by: Chris Ball >> >> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c >> index 1316139..e2fa072 100644 >> --- a/fs/btrfs/volumes.c >> +++ b/fs/btrfs/volumes.c >> @@ -1815,30 +1815,8 @@ int btrfs_shrink_device(struct btrfs_device *device, >> u64 new_size) >> if (!path) >> return -ENOMEM; >> >> - trans = btrfs_start_transaction(root, 1); >> - if (!trans) { >> - ret = -ENOMEM; >> - goto done; >> - } >> - >> path->reada = 2; >> >> - lock_chunks(root); >> - >> - device->total_bytes = new_size; >> - if (device->writeable) >> - device->fs_devices->total_rw_bytes -= diff; > > So I think you still want to do this part, to keep the allocator from actually > allocating new space in the area we are trying to cull with the shrink, we > just > don't want to update the ondisk stuff just yet, so everything else can be > moved > to below the loop. > > So this >> - ret = btrfs_update_device(trans, device); >> - if (ret) { >> - unlock_chunks(root); >> - btrfs_end_transaction(trans, root); >> - goto done; >> - } >> - WARN_ON(diff > old_total); >> - btrfs_set_super_total_bytes(super_copy, old_total - diff); > > to here should all be moved below like you have it. Other than that it looks > good. Thanks, > This isn't working. we don't call btrfs_update_device here, but it can be called in other places. I think we should add a new field in btrfs_device to reflect the on disk device size, and update it when shrinking succeeds. Regards Yan Zheng -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Performance results on Experimental branch
On Miércoles 25 Febrero 2009 23:55:08 Steven Pratt escribió: > Unless I am missing something, what you are referring to is a simple > wraping/alignment issue in the key on the long name on the experimental > btrfs. It is the Brown bar. Let me know if this is not the issue. doh, you're right. Sorry, it's time to sleep here ;( -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Backport for 2.6.27 and 2.6.26 on the experimental branch
A second update to this. It seems the problem is that btrfs gets stuck in an infinite loop while in tree_insert. I'm not sure why yet. Below is my back trace incase anyone wants to have a look. [ 9824.393942] SysRq : Show Blocked State [ 9824.393942] taskPC stack pid father [ 9824.393942] lsD 0 3456 3334 [ 9824.393942]f1ada8c0 00200082 01c19000 01c19000 f1adaa4c c27f7fa0 [ 9824.393942]f518830c 001f711e c27e68a4 c013604c 00ff [ 9824.393942]c27f7fa0 0243d000 c27e68a4 c015688a c02b8498 f1b6fc64 f1b6fc64 c01568bd [ 9824.393942] Call Trace: [ 9824.393942] [] getnstimeofday+0x37/0xbc [ 9824.393942] [] sync_page+0x0/0x36 [ 9824.393942] [] io_schedule+0x49/0x80 [ 9824.393942] [] sync_page+0x33/0x36 [ 9824.393942] [] __wait_on_bit_lock+0x2a/0x52 [ 9824.393942] [] __lock_page+0x4e/0x54 [ 9824.393942] [] wake_bit_function+0x0/0x3c [ 9824.393942] [] read_extent_buffer_pages+0x133/0x2f1 [btrfs] [ 9824.393942] [] btree_read_extent_buffer_pages+0x39/0x8c [btrfs] [ 9824.393942] [] btree_get_extent+0x0/0x173 [btrfs] [ 9824.393942] [] read_tree_block+0x29/0x4c [btrfs] [ 9824.393942] [] btrfs_search_slot+0x4fe/0x638 [btrfs] [ 9824.393942] [] btrfs_lookup_inode+0x27/0x88 [btrfs] [ 9824.393942] [] btrfs_read_locked_inode+0x53/0x2fc [btrfs] [ 9824.393942] [] iget5_locked+0x7a/0x12b [ 9824.393942] [] btrfs_find_actor+0x0/0x27 [btrfs] [ 9824.393942] [] btrfs_iget+0x46/0x6c [btrfs] [ 9824.393942] [] btrfs_lookup_dentry+0x173/0x184 [btrfs] [ 9824.393942] [] d_alloc+0x138/0x17a [ 9824.393942] [] btrfs_lookup+0x18/0x2d [btrfs] [ 9824.393942] [] do_lookup+0xb6/0x153 [ 9824.393942] [] __link_path_walk+0x724/0xb0b [ 9824.393942] [] tree_insert+0x54/0x5b [btrfs] [ 9824.393942] [] path_walk+0x37/0x70 [ 9824.393942] [] do_path_lookup+0x122/0x184 [ 9824.393942] [] __user_walk_fd+0x29/0x3a [ 9824.393942] [] vfs_lstat_fd+0x12/0x39 [ 9824.393942] [] sys_lstat64+0xf/0x23 [ 9824.393942] [] sysenter_past_esp+0x78/0xb1 [ 9824.393942] [] acpi_pci_root_add+0x125/0x296 [ 9824.393942] === [ 9824.393942] Sched Debug Version: v0.07, 2.6.26-1-686 #1 [ 9824.393942] now at 8542803.745635 msecs [ 9824.393942] .sysctl_sched_latency: 40.00 [ 9824.393942] .sysctl_sched_min_granularity: 8.00 [ 9824.393942] .sysctl_sched_wakeup_granularity : 20.00 [ 9824.393942] .sysctl_sched_child_runs_first : 0.01 [ 9824.393942] .sysctl_sched_features : 895 [ 9824.393942] [ 9824.393942] cpu#0, 1862.040 MHz [ 9824.393942] .nr_running: 2 [ 9824.393942] .load : 2048 [ 9824.393942] .nr_switches : 475330 [ 9824.393942] .nr_load_updates : 61723 [ 9824.393942] .nr_uninterruptible: 4294966907 [ 9824.393942] .jiffies : 2060701 [ 9824.393942] .next_balance : 2.060723 [ 9824.393942] .curr->pid : 2199 [ 9824.393942] .clock : 8542803.113648 [ 9824.393942] .cpu_load[0] : 0 [ 9824.393942] .cpu_load[1] : 175 [ 9824.393942] .cpu_load[2] : 469 [ 9824.393942] .cpu_load[3] : 621 [ 9824.393942] .cpu_load[4] : 542 [ 9824.393942] [ 9824.393942] cfs_rq[0]: [ 9824.393942] .exec_clock: 0.00 [ 9824.393942] .MIN_vruntime : 179538.548824 [ 9824.393942] .min_vruntime : 179578.548817 [ 9824.393942] .max_vruntime : 179538.548824 [ 9824.393942] .spread: 0.00 [ 9824.393942] .spread0 : 0.00 [ 9824.393942] .nr_running: 2 [ 9824.393942] .load : 2048 [ 9824.393942] .nr_spread_over: 0 [ 9824.393942] [ 9824.393942] runnable tasks: [ 9824.393942] task PID tree-key switches prio exec-runtime sum-execsum-sleep [ 9824.393942] -- [ 9824.393942] R rsyslogd 2199179538.54882028 120 0 0 0.00 0.00 0.00 / [ 9824.393942] rsyslogd 179538.548824 9 120 0 0 0.00 0.00 0.00 / [ 9824.393942] [ 9824.393942] cpu#1, 1862.040 MHz [ 9824.393942] .nr_running: 2 [ 9824.393942] .load : 2048 [ 9824.393942] .nr_switches : 90067 [ 9824.393942] .nr_load_updates : 31570 [ 9824.393942] .nr_uninterruptible: 390 [ 9824.393942] .jiff
Re: Performance results on Experimental branch
Diego Calleja wrote: On Miércoles 25 Febrero 2009 22:07:33 Steven Pratt escribió: All in all good progress. Results and graphs can be found here: http://btrfs.boxacle.net/repository/raid/history/History.html Some graphs seem to be broken...btrfs gets a "transparent" color. Unless I am missing something, what you are referring to is a simple wraping/alignment issue in the key on the long name on the experimental btrfs. It is the Brown bar. Let me know if this is not the issue. Steve For example here: http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=16.FFSB_Ops_per_Sec.png -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs: warn_slowpath in clean_tree_block and others
On Wed, Feb 25, 2009 at 02:26:43PM -0500, Chris Mason wrote: > [ resend with the list cc'd ] > > On Wed, 2009-02-25 at 12:50 -0600, Mitch Harder (aka DontPanic) wrote: > > I'll try to test that out. > > > > I had just noticed that some of my kernel configuration settings (not > > sure which ones) seem to affect the clean_tree_block warnings I've > > been getting, and one of my customizations is usually to configure the > > kernel for a single processor. > > > > I'll push out a patch tonight that fixes things, the code to test for a > locked buffer is just broken on UP. For now, the patch below will do: Yup, that shuts it up quite effectively. :) Thanks, Hugo. -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- If it's December 1941 in Casablanca, what time is it --- in New York? signature.asc Description: Digital signature
Re: Performance results on Experimental branch
On Miércoles 25 Febrero 2009 22:07:33 Steven Pratt escribió: > All in all good progress. Results and graphs can be found here: > > http://btrfs.boxacle.net/repository/raid/history/History.html Some graphs seem to be broken...btrfs gets a "transparent" color. For example here: http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=16.FFSB_Ops_per_Sec.png -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Performance results on Experimental branch
Have completed the first run of the experimental branch on my RAID system and the results are encouraging. Compared to 2.6.29-rc2: Large file creates increase 50% plus across all thread counts. Mail Server is up about 20% Random reads are pretty much flat Random Writes are up 50% or more Sequential reads took a 10% hit on single thread, but 20% gain on 128 threads. All in all good progress. Results and graphs can be found here: http://btrfs.boxacle.net/repository/raid/history/History.html Steve -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: questions about GRUB and BTRFS
On Wed, 2009-02-25 at 13:04 -0700, Anthony Roberts wrote: > Hi Chris, > > Cheers for the informative response. :) > > > In the ideal implementation, the grub.conf has a list of devices it is > > allowed to scan, and we put the FS uuid directly in there, let grub scan > > them and we'll be able to boot off multiple volumes in that way. > > Hm... perhaps it doesn't even need that much. It we're specifying devices, > might it simply pull the UUID out of the initial device it's given, kinda > like how the current mounting process works? Enumeration of the drives > changes as they come and go, but it doesn't really matter what "hd0" is > today as long as it's a member device. The problem is that grub would need some way of knowing which devices to scan. I'd hate to see it off running through everything in a san. > > This suggests another question for me... right now you can specify a root= > command line to the kernel, though there's other stuff like nfsroot= where > more parameters are needed. Is it possible to add a btrfsroot= option with > a UUID+subvolume? I'm not a huge fan of initrds, but the distro initrd scripts already have this kind of goodness hooked in. I think that's the best place for it. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs: introduce btrfs_show_options
On Tue, 2009-02-24 at 17:48 -0500, Eric Paris wrote: > btrfs options can change at times other than mount, yet /proc/mounts shows the > options string used when the fs was mounted (an example would be when btrfs > determines that barriers aren't useful and turns them off.) This patch > instead outputs the actual options in use by btrfs. > Nice, thanks -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: questions about GRUB and BTRFS
Hi Chris, Cheers for the informative response. :) > In the ideal implementation, the grub.conf has a list of devices it is > allowed to scan, and we put the FS uuid directly in there, let grub scan > them and we'll be able to boot off multiple volumes in that way. Hm... perhaps it doesn't even need that much. It we're specifying devices, might it simply pull the UUID out of the initial device it's given, kinda like how the current mounting process works? Enumeration of the drives changes as they come and go, but it doesn't really matter what "hd0" is today as long as it's a member device. This suggests another question for me... right now you can specify a root= command line to the kernel, though there's other stuff like nfsroot= where more parameters are needed. Is it possible to add a btrfsroot= option with a UUID+subvolume? With these features exposed to the booting process, it seems like there are a *lot* of very powerful use cases that become possible with management of the OS itself. > But, that is a significant project, so we'll have to do it in stages. Indeed. It's not my expectation it be done initially or even soon, just my hope that it "eventually" become as flexible in booting as it is once the system goes multi-user. :) -Anthony -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [review] Btrfs: Allow shrinking close to used space
On Wed, Feb 25, 2009 at 02:13:10PM -0500, Chris Ball wrote: > Hi, > > This patch (against experimental HEAD) attempts to make shrinking more > robust, by only updating device size if we've succeeded in creating > enough free space without any failures in btrfs_relocate_chunk(). > > Here's a log with my patch applied. The two things to note are that a > near-limit shrink now works, and that a failed shrink (in this case, > trying to shrink to less than the used space) no longer updates the > device size erroneously: > > http://dev.laptop.org/~cjb/btrfs/shrink-log > > Please review carefully -- I'm still new to btrfs. The short version of > the patch is: > > * create a success path, as a break out of the while(1) relocating > (rather than going to the "done" label). > * move the device size updating code into that path > * leave "path->reada = 2;" behind in the entry path, since path is > used by the searching operation rather than the later resize. > > Thanks! > > Signed-off-by: Chris Ball > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index 1316139..e2fa072 100644 > --- a/fs/btrfs/volumes.c > +++ b/fs/btrfs/volumes.c > @@ -1815,30 +1815,8 @@ int btrfs_shrink_device(struct btrfs_device *device, > u64 new_size) > if (!path) > return -ENOMEM; > > - trans = btrfs_start_transaction(root, 1); > - if (!trans) { > - ret = -ENOMEM; > - goto done; > - } > - > path->reada = 2; > > - lock_chunks(root); > - > - device->total_bytes = new_size; > - if (device->writeable) > - device->fs_devices->total_rw_bytes -= diff; So I think you still want to do this part, to keep the allocator from actually allocating new space in the area we are trying to cull with the shrink, we just don't want to update the ondisk stuff just yet, so everything else can be moved to below the loop. So this > - ret = btrfs_update_device(trans, device); > - if (ret) { > - unlock_chunks(root); > - btrfs_end_transaction(trans, root); > - goto done; > - } > - WARN_ON(diff > old_total); > - btrfs_set_super_total_bytes(super_copy, old_total - diff); to here should all be moved below like you have it. Other than that it looks good. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Warning and BUG with btrfs and corrupted image
On Thu 2009-02-05 09:19:28, jim owens wrote: > Pavel Machek wrote: >>> If you don't want it, don't compile it in. The Kconfig text is very >>> clear. >> >> No, I'd not expect that option to panic systems. That's why I >> suggested: >> >> diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig >> index 29228f5..b7ac847 100644 >> --- a/fs/xfs/Kconfig >> +++ b/fs/xfs/Kconfig >> @@ -77,4 +77,7 @@ config XFS_DEBUG >>Note that the resulting code will be HUGE and SLOW, and probably >>not useful unless you are debugging a particular problem. >> + Turning this option on will result in kernel panicking any time >> + it detects on-disk corruption. >> + >>Say N unless you are an XFS developer, or you play one on TV. > > If you really want a better warning it should simply be: > > Choosing Y will make XFS panic on survivable events. Can we get that line there? > I understand you may have a concern that "normal users" will select > the debug option by mistake, but I don't think that is realistic. > My experience is they will not build custom debug kernels even if > you beg them to. They will only use the distro build and a > distro should never turn this option on outside their own labs. > > Any non-xfs kernel developer who turns this on and gets > snake bit will only do it once. One bite per developer is one bite too many.. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs: warn_slowpath in clean_tree_block and others
[ resend with the list cc'd ] On Wed, 2009-02-25 at 12:50 -0600, Mitch Harder (aka DontPanic) wrote: > I'll try to test that out. > > I had just noticed that some of my kernel configuration settings (not > sure which ones) seem to affect the clean_tree_block warnings I've > been getting, and one of my customizations is usually to configure the > kernel for a single processor. > I'll push out a patch tonight that fixes things, the code to test for a locked buffer is just broken on UP. For now, the patch below will do: diff --git a/fs/btrfs/locking.c b/fs/btrfs/locking.c index 85506c4..4513ecf 100644 --- a/fs/btrfs/locking.c +++ b/fs/btrfs/locking.c @@ -222,6 +222,5 @@ int btrfs_tree_unlock(struct extent_buffer *eb) int btrfs_tree_locked(struct extent_buffer *eb) { - return test_bit(EXTENT_BUFFER_BLOCKING, &eb->bflags) || - spin_is_locked(&eb->lock); + return 1; } -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs: warn_slowpath in clean_tree_block and others
On Wed, Feb 25, 2009 at 01:36:26PM -0500, Chris Mason wrote: > On Tue, 2009-02-24 at 23:02 +, Hugo Mills wrote: > > This is essentially a repost of a mail I made last week, to which I > > didn't get a reply. > > Sorry I missed replying to this one last week, thanks for resending. Not a problem. I know things go astray sometimes. > >I'm getting huge numbers of kernel warnings whilst using > > btrfs. They're all "warn_slowpath", and all seem to be in > > fs/btrfs/disk-io.c. I've included one typical example at the end of > > this mail. > > > >Kernel versions are 2.6.29-rc2, -rc4 and -rc6. > > > > The warnings look like i386, exactly what hardware is this? Is your > kernel compiled for SMP or UP? amd64 and a UP kernel: h...@vlad:linux-2.6 $ uname -a Linux vlad 2.6.29-rc6 #1 Mon Feb 23 19:53:22 GMT 2009 x86_64 GNU/Linux Hardware is an original-series Turion 64 (i.e. one core) in a Socket 745 desktop motherboard. The btrfs filesystem is in LVM-on-RAID1 in an eSATA port-multiplier storage rack. > The warning you're getting is that clean_tree_block expects this block > to be locked, and giving out a warning because it is showing up as > unlocked. > > So, hopefully you're on a UP kernel and my test for a locked spinlock is > broken in that config. OK. I can try patches if necessary. Hugo. -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- UNIX: British manufacturer of modular shelving units. --- signature.asc Description: Digital signature
Re: lots of kernel threads
On Wed, Feb 25, 2009 at 9:51 AM, Chris Mason wrote: > On Tue, 2009-02-24 at 20:09 -0800, Wil Reichert wrote: >> I'm a Gentoo user and figured mounting /usr/portage & /var/portage >> (distfiles, packages, persistent stuff normally in /usr/portage) and >> /tmp moiunted as btrfs. I figured it would be a decent test & its >> nothing I cann't replace readily. After about 6 hours of uptime and a >> couple package merges, I checked ps aux & am getting the following: >> > > Btrfs has a number of thread pools per mounted filesystem. It makes > sense to switch the thread pools to be more global, but for right now > these threads are not unusual. > > Kernel threads are lightweight enough that the only major downside to > these is making ps output ugly ;) > > -chris ps noise is fine, my only concern was that something was getting created & not cleaned up. Thanks, Wil -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[review] Btrfs: Allow shrinking close to used space
Hi, This patch (against experimental HEAD) attempts to make shrinking more robust, by only updating device size if we've succeeded in creating enough free space without any failures in btrfs_relocate_chunk(). Here's a log with my patch applied. The two things to note are that a near-limit shrink now works, and that a failed shrink (in this case, trying to shrink to less than the used space) no longer updates the device size erroneously: http://dev.laptop.org/~cjb/btrfs/shrink-log Please review carefully -- I'm still new to btrfs. The short version of the patch is: * create a success path, as a break out of the while(1) relocating (rather than going to the "done" label). * move the device size updating code into that path * leave "path->reada = 2;" behind in the entry path, since path is used by the searching operation rather than the later resize. Thanks! Signed-off-by: Chris Ball diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 1316139..e2fa072 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -1815,30 +1815,8 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size) if (!path) return -ENOMEM; - trans = btrfs_start_transaction(root, 1); - if (!trans) { - ret = -ENOMEM; - goto done; - } - path->reada = 2; - lock_chunks(root); - - device->total_bytes = new_size; - if (device->writeable) - device->fs_devices->total_rw_bytes -= diff; - ret = btrfs_update_device(trans, device); - if (ret) { - unlock_chunks(root); - btrfs_end_transaction(trans, root); - goto done; - } - WARN_ON(diff > old_total); - btrfs_set_super_total_bytes(super_copy, old_total - diff); - unlock_chunks(root); - btrfs_end_transaction(trans, root); - key.objectid = device->devid; key.offset = (u64)-1; key.type = BTRFS_DEV_EXTENT_KEY; @@ -1867,7 +1845,7 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size) length = btrfs_dev_extent_length(l, dev_extent); if (key.offset + length <= new_size) - goto done; + break; chunk_tree = btrfs_dev_extent_chunk_tree(l, dev_extent); chunk_objectid = btrfs_dev_extent_chunk_objectid(l, dev_extent); @@ -1880,6 +1858,31 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size) goto done; } + /* +* We've succeeded in freeing up enough space and can now update +* the device's size. +*/ + trans = btrfs_start_transaction(root, 1); + if (!trans) { + ret = -ENOMEM; + goto done; + } + + lock_chunks(root); + device->total_bytes = new_size; + if (device->writeable) + device->fs_devices->total_rw_bytes -= diff; + ret = btrfs_update_device(trans, device); + if (ret) { + unlock_chunks(root); + btrfs_end_transaction(trans, root); + goto done; + } + WARN_ON(diff > old_total); + btrfs_set_super_total_bytes(super_copy, old_total - diff); + unlock_chunks(root); + btrfs_end_transaction(trans, root); + done: btrfs_free_path(path); return ret; -- Chris Ball -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs: warn_slowpath in clean_tree_block and others
On Tue, 2009-02-24 at 23:02 +, Hugo Mills wrote: > This is essentially a repost of a mail I made last week, to which I > didn't get a reply. > Sorry I missed replying to this one last week, thanks for resending. >I'm getting huge numbers of kernel warnings whilst using > btrfs. They're all "warn_slowpath", and all seem to be in > fs/btrfs/disk-io.c. I've included one typical example at the end of > this mail. > >Kernel versions are 2.6.29-rc2, -rc4 and -rc6. > The warnings look like i386, exactly what hardware is this? Is your kernel compiled for SMP or UP? The warning you're getting is that clean_tree_block expects this block to be locked, and giving out a warning because it is showing up as unlocked. So, hopefully you're on a UP kernel and my test for a locked spinlock is broken in that config. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: questions about GRUB and BTRFS
On Tue, 2009-02-24 at 18:32 -0700, Anthony Roberts wrote: > Hi, > > A quick googling turns up posts that GRUB support for BTRFS is planned. My > curiosity is more towards how this will be managed, because the way this is > currently implemented with software RAID/LVM is quite haphazard. I > therefore have some questions about GRUB + BTRFS: > So, I haven't looked very hard at the grub code, and expected to start the btrfs grub support with something very minimal (single device only configs). > -With GRUB booting, it's easy to think of awkward use cases and limitations > unless it's capable of discovering BTRFS instances, and can boot by > specifying BTRFS UUID + subvolume. That seems quite ambitious, but is this > planned "eventually"? Grub today works by giving it a device and a path. Since all the btrfs subvolumes can be found from a path, device + path will actually work with the subvolume support. > > -Might it be possible to tweak the userspace component of GRUB to install > the bootloader to every member device? This seems necessary for reliable > booting and rebuilding after a dead disk. I need to look into the grub code in detail to answer this. > > -64 kb at the beginning of the device is plenty for MBR + GRUB stage 1 + > 1.5. Might this allow bootable BTRFS without paritions being used at all? > The space used for partitioning is negligible, however we're on the cusp of > disks that are too big to partition with MBR, and GPT booting doesn't seem > well supported yet. > Part of the reason we're 64kb in was to better support bootloaders. > There's obviously no point in getting worked up about this before > production ready support is available in the first place. :) However, I am > curious about what sort of implementation is planned. In the ideal implementation, the grub.conf has a list of devices it is allowed to scan, and we put the FS uuid directly in there, let grub scan them and we'll be able to boot off multiple volumes in that way. But, that is a significant project, so we'll have to do it in stages. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: questions about GRUB and BTRFS
On Mi, 25.02.09 01:03 Lee Trager wrote: > I'm not sure when we should start developing BTRFS support for GRUB > but I do agree that it will be very difficult to support all the > features of BTRFS. As far as I know GRUB does not support LVM and > only supports RAID1. Doing this shouldn't be that hard to do, in fact GRUB2 maybe? Using trunk I see things like /lib64/grub/i386-pc/lvm.mod /lib64/grub/i386-pc/ext2.mod /lib64/grub/i386-pc/reiserfs.mod and /lib64/grub/i386-pc/raid.mod ...which a grub-install will copy to /boot/grub I haven't tested those though. signature.asc Description: PGP signature
Re: lots of kernel threads
On Tue, 2009-02-24 at 20:09 -0800, Wil Reichert wrote: > I'm a Gentoo user and figured mounting /usr/portage & /var/portage > (distfiles, packages, persistent stuff normally in /usr/portage) and > /tmp moiunted as btrfs. I figured it would be a decent test & its > nothing I cann't replace readily. After about 6 hours of uptime and a > couple package merges, I checked ps aux & am getting the following: > Btrfs has a number of thread pools per mounted filesystem. It makes sense to switch the thread pools to be more global, but for right now these threads are not unusual. Kernel threads are lightweight enough that the only major downside to these is making ps output ugly ;) -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs: warn_slowpath in clean_tree_block and others
The messages attached are only "WARNING" messages. I have not been encountering a crash, nor does the data seem to get corrupted in my case (as far as I can tell). Btrfs seems to actually work fine, except for the large amount of log messages. On Wed, Feb 25, 2009 at 10:13 AM, Hugo Mills wrote: > On Wed, Feb 25, 2009 at 11:05:58AM -0500, Lee Trager wrote: >> But what are you doing to the filesystem when it crashes? How did you >> mount it? > > In my case, it's mounted with this fstab entry: > > /dev/media/scratch /media/vlad/video/video btrfs noatime,nosuid,nodev > 0 0 > > and I can trigger hundreds (literally) of these backtraces with a > single "touch /media/vlad/video/video/foo". If I encode a video to the > FS, the backtraces come in bursts at intervals of, say, 20 seconds > (it's not perfectly regular). > > Hugo. > >> On Wed, Feb 25, 2009 at 08:03:01AM -0600, Mitch Harder (aka DontPanic) wrote: >> > I've been creating a local git repository of full btrfs-unstable sources. >> > >> > I'll create a new branch off the master branch, and apply the patch >> > supplied in the Feb. 11 message to the M/L. >> > >> > I then create a kernel module based on the results in /fs/btrfs/ >> > >> > I have also tried replicating the experimental branch, and merging the >> > patch into that branch, but I get the same results. >> > >> > On Wed, Feb 25, 2009 at 12:26 AM, Lee Trager wrote: >> > > Mitch, I haven't seen any problems using BTRFS and my patch on 2.6.28 or >> > > 2.6.27, what are you doing to cause this error? Are you using the latest >> > > sources from btrfs-unstable? >> > > >> > > Lee >> > > >> > > Mitch Harder (aka DontPanic) wrote: >> > >> I have also been getting similar warnings filling up my logs. >> > >> >> > >> However, in my case, I have been experimenting with back-porting btrfs >> > >> to a 2.6.28 kernel. ?So I've been waiting for the back-porting efforts >> > >> to get a little further along. >> > >> >> > >> But I thought I'd respond in case this information helps. >> > >> >> > >> Here's an example of the warnings I've been seeing: >> > >> >> > >> [80577.151167] [ cut here ] >> > >> [80577.151169] WARNING: at >> > >> /var/tmp/portage/sys-fs/btrfs-9998/work/btrfs-9998/disk-io.c:860 >> > >> clean_tree_block+0xa4/0xb0 [btrfs]() >> > >> [80577.151172] Modules linked in: btrfs snd_pcm_oss snd_mixer_oss >> > >> snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device >> > >> ipv6 ppdev snd_intel8x0 snd_ac97_codec parport_pc nvidia(P) ac97_bus >> > >> snd_pcm snd_timer ohci_hcd ssb shpchp pci_hotplug pcmcia i2c_nforce2 >> > >> snd forcedeth sr_mod pcspkr parport i2c_core snd_page_alloc nvidia_agp >> > >> sl811_hcd pcmcia_core uhci_hcd ehci_hcd >> > >> [80577.151190] Pid: 11503, comm: cp Tainted: P ? ? ? ?W >> > >> ?2.6.28-sabayon-r10 #1 >> > >> [80577.151192] Call Trace: >> > >> [80577.151195] ?[] warn_on_slowpath+0x5f/0x90 >> > >> [80577.151203] ?[] rb_insert_color+0x77/0xe0 >> > >> [80577.151221] ?[] alloc_extent_buffer+0x1fe/0x300 [btrfs] >> > >> [80577.151238] ?[] clean_tree_block+0xa4/0xb0 [btrfs] >> > >> [80577.151253] ?[] btrfs_init_new_buffer+0x7d/0x130 [btrfs] >> > >> [80577.151269] ?[] btrfs_alloc_free_block+0x104/0x110 [btrfs] >> > >> [80577.151285] ?[] __btrfs_cow_block+0x22a/0x8b0 [btrfs] >> > >> [80577.151300] ?[] generic_bin_search+0x162/0x1c0 [btrfs] >> > >> [80577.151315] ?[] btrfs_cow_block+0x156/0x200 [btrfs] >> > >> [80577.151330] ?[] btrfs_search_slot+0x1a7/0x910 [btrfs] >> > >> [80577.151333] ?[] irq_exit+0x27/0x60 >> > >> [80577.151336] ?[] do_IRQ+0x6b/0x80 >> > >> [80577.151354] ?[] read_extent_buffer+0xd5/0x170 [btrfs] >> > >> [80577.151369] ?[] btrfs_insert_empty_items+0x6d/0x410 [btrfs] >> > >> [80577.151385] ?[] btrfs_find_block_group+0xff/0x1a0 [btrfs] >> > >> [80577.151402] ?[] btrfs_new_inode+0x18d/0x360 [btrfs] >> > >> [80577.151420] ?[] btrfs_create+0x189/0x2a0 [btrfs] >> > >> [80577.151423] ?[] security_capable+0x9/0x10 >> > >> [80577.151427] ?[] vfs_create+0xcd/0x160 >> > >> [80577.151430] ?[] do_filp_open+0x5af/0x7d0 >> > >> [80577.151433] ?[] cp_new_stat64+0xf9/0x110 >> > >> [80577.151436] ?[] do_sys_open+0x4e/0xe0 >> > >> [80577.151439] ?[] sys_open+0x2c/0x40 >> > >> [80577.151442] ?[] sysenter_do_call+0x12/0x21 >> > >> [80577.151444] ---[ end trace 79cdc48bc88dedf7 ]--- >> > >> >> > >> >> > >> On Tue, Feb 24, 2009 at 5:02 PM, Hugo Mills >> > >> wrote: >> > >> >> > >>> ? This is essentially a repost of a mail I made last week, to which I >> > >>> didn't get a reply. >> > >>> >> > >>> ? I'm getting huge numbers of kernel warnings whilst using >> > >>> btrfs. They're all "warn_slowpath", and all seem to be in >> > >>> fs/btrfs/disk-io.c. I've included one typical example at the end of >> > >>> this mail. >> > >>> >> > >>> ? Kernel versions are 2.6.29-rc2, -rc4 and -rc6. >> > >>> >> > >>> ? If I do lots of writes to my btrfs filesystem (e.g. video >> > >>> encoding), I end up with a syslog in the tens-of-megab
Re: btrfs: warn_slowpath in clean_tree_block and others
On Wed, Feb 25, 2009 at 11:05:58AM -0500, Lee Trager wrote: > But what are you doing to the filesystem when it crashes? How did you > mount it? In my case, it's mounted with this fstab entry: /dev/media/scratch /media/vlad/video/video btrfs noatime,nosuid,nodev 0 0 and I can trigger hundreds (literally) of these backtraces with a single "touch /media/vlad/video/video/foo". If I encode a video to the FS, the backtraces come in bursts at intervals of, say, 20 seconds (it's not perfectly regular). Hugo. > On Wed, Feb 25, 2009 at 08:03:01AM -0600, Mitch Harder (aka DontPanic) wrote: > > I've been creating a local git repository of full btrfs-unstable sources. > > > > I'll create a new branch off the master branch, and apply the patch > > supplied in the Feb. 11 message to the M/L. > > > > I then create a kernel module based on the results in /fs/btrfs/ > > > > I have also tried replicating the experimental branch, and merging the > > patch into that branch, but I get the same results. > > > > On Wed, Feb 25, 2009 at 12:26 AM, Lee Trager wrote: > > > Mitch, I haven't seen any problems using BTRFS and my patch on 2.6.28 or > > > 2.6.27, what are you doing to cause this error? Are you using the latest > > > sources from btrfs-unstable? > > > > > > Lee > > > > > > Mitch Harder (aka DontPanic) wrote: > > >> I have also been getting similar warnings filling up my logs. > > >> > > >> However, in my case, I have been experimenting with back-porting btrfs > > >> to a 2.6.28 kernel. ?So I've been waiting for the back-porting efforts > > >> to get a little further along. > > >> > > >> But I thought I'd respond in case this information helps. > > >> > > >> Here's an example of the warnings I've been seeing: > > >> > > >> [80577.151167] [ cut here ] > > >> [80577.151169] WARNING: at > > >> /var/tmp/portage/sys-fs/btrfs-9998/work/btrfs-9998/disk-io.c:860 > > >> clean_tree_block+0xa4/0xb0 [btrfs]() > > >> [80577.151172] Modules linked in: btrfs snd_pcm_oss snd_mixer_oss > > >> snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device > > >> ipv6 ppdev snd_intel8x0 snd_ac97_codec parport_pc nvidia(P) ac97_bus > > >> snd_pcm snd_timer ohci_hcd ssb shpchp pci_hotplug pcmcia i2c_nforce2 > > >> snd forcedeth sr_mod pcspkr parport i2c_core snd_page_alloc nvidia_agp > > >> sl811_hcd pcmcia_core uhci_hcd ehci_hcd > > >> [80577.151190] Pid: 11503, comm: cp Tainted: P ? ? ? ?W > > >> ?2.6.28-sabayon-r10 #1 > > >> [80577.151192] Call Trace: > > >> [80577.151195] ?[] warn_on_slowpath+0x5f/0x90 > > >> [80577.151203] ?[] rb_insert_color+0x77/0xe0 > > >> [80577.151221] ?[] alloc_extent_buffer+0x1fe/0x300 [btrfs] > > >> [80577.151238] ?[] clean_tree_block+0xa4/0xb0 [btrfs] > > >> [80577.151253] ?[] btrfs_init_new_buffer+0x7d/0x130 [btrfs] > > >> [80577.151269] ?[] btrfs_alloc_free_block+0x104/0x110 [btrfs] > > >> [80577.151285] ?[] __btrfs_cow_block+0x22a/0x8b0 [btrfs] > > >> [80577.151300] ?[] generic_bin_search+0x162/0x1c0 [btrfs] > > >> [80577.151315] ?[] btrfs_cow_block+0x156/0x200 [btrfs] > > >> [80577.151330] ?[] btrfs_search_slot+0x1a7/0x910 [btrfs] > > >> [80577.151333] ?[] irq_exit+0x27/0x60 > > >> [80577.151336] ?[] do_IRQ+0x6b/0x80 > > >> [80577.151354] ?[] read_extent_buffer+0xd5/0x170 [btrfs] > > >> [80577.151369] ?[] btrfs_insert_empty_items+0x6d/0x410 [btrfs] > > >> [80577.151385] ?[] btrfs_find_block_group+0xff/0x1a0 [btrfs] > > >> [80577.151402] ?[] btrfs_new_inode+0x18d/0x360 [btrfs] > > >> [80577.151420] ?[] btrfs_create+0x189/0x2a0 [btrfs] > > >> [80577.151423] ?[] security_capable+0x9/0x10 > > >> [80577.151427] ?[] vfs_create+0xcd/0x160 > > >> [80577.151430] ?[] do_filp_open+0x5af/0x7d0 > > >> [80577.151433] ?[] cp_new_stat64+0xf9/0x110 > > >> [80577.151436] ?[] do_sys_open+0x4e/0xe0 > > >> [80577.151439] ?[] sys_open+0x2c/0x40 > > >> [80577.151442] ?[] sysenter_do_call+0x12/0x21 > > >> [80577.151444] ---[ end trace 79cdc48bc88dedf7 ]--- > > >> > > >> > > >> On Tue, Feb 24, 2009 at 5:02 PM, Hugo Mills > > >> wrote: > > >> > > >>> ? This is essentially a repost of a mail I made last week, to which I > > >>> didn't get a reply. > > >>> > > >>> ? I'm getting huge numbers of kernel warnings whilst using > > >>> btrfs. They're all "warn_slowpath", and all seem to be in > > >>> fs/btrfs/disk-io.c. I've included one typical example at the end of > > >>> this mail. > > >>> > > >>> ? Kernel versions are 2.6.29-rc2, -rc4 and -rc6. > > >>> > > >>> ? If I do lots of writes to my btrfs filesystem (e.g. video > > >>> encoding), I end up with a syslog in the tens-of-megabytes range. This > > >>> makes logcheck an unhappy bunny... > > >>> > > >>> ? I don't know if this behaviour is expected, and everyone using > > >>> btrfs simply puts up with it for now, or if it's something unusual > > >>> that needs investigating. On the chance that it's the latter, I'm > > >>> reporting it here. > > >>> > > >>> ? Hugo. > > >>> > > >>> Feb 23 21:45:42 vlad kernel: [ cut
Re: btrfs: warn_slowpath in clean_tree_block and others
But what are you doing to the filesystem when it crashes? How did you mount it? Lee On Wed, Feb 25, 2009 at 08:03:01AM -0600, Mitch Harder (aka DontPanic) wrote: > I've been creating a local git repository of full btrfs-unstable sources. > > I'll create a new branch off the master branch, and apply the patch > supplied in the Feb. 11 message to the M/L. > > I then create a kernel module based on the results in /fs/btrfs/ > > I have also tried replicating the experimental branch, and merging the > patch into that branch, but I get the same results. > > On Wed, Feb 25, 2009 at 12:26 AM, Lee Trager wrote: > > Mitch, I haven't seen any problems using BTRFS and my patch on 2.6.28 or > > 2.6.27, what are you doing to cause this error? Are you using the latest > > sources from btrfs-unstable? > > > > Lee > > > > Mitch Harder (aka DontPanic) wrote: > >> I have also been getting similar warnings filling up my logs. > >> > >> However, in my case, I have been experimenting with back-porting btrfs > >> to a 2.6.28 kernel. ?So I've been waiting for the back-porting efforts > >> to get a little further along. > >> > >> But I thought I'd respond in case this information helps. > >> > >> Here's an example of the warnings I've been seeing: > >> > >> [80577.151167] [ cut here ] > >> [80577.151169] WARNING: at > >> /var/tmp/portage/sys-fs/btrfs-9998/work/btrfs-9998/disk-io.c:860 > >> clean_tree_block+0xa4/0xb0 [btrfs]() > >> [80577.151172] Modules linked in: btrfs snd_pcm_oss snd_mixer_oss > >> snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device > >> ipv6 ppdev snd_intel8x0 snd_ac97_codec parport_pc nvidia(P) ac97_bus > >> snd_pcm snd_timer ohci_hcd ssb shpchp pci_hotplug pcmcia i2c_nforce2 > >> snd forcedeth sr_mod pcspkr parport i2c_core snd_page_alloc nvidia_agp > >> sl811_hcd pcmcia_core uhci_hcd ehci_hcd > >> [80577.151190] Pid: 11503, comm: cp Tainted: P ? ? ? ?W > >> ?2.6.28-sabayon-r10 #1 > >> [80577.151192] Call Trace: > >> [80577.151195] ?[] warn_on_slowpath+0x5f/0x90 > >> [80577.151203] ?[] rb_insert_color+0x77/0xe0 > >> [80577.151221] ?[] alloc_extent_buffer+0x1fe/0x300 [btrfs] > >> [80577.151238] ?[] clean_tree_block+0xa4/0xb0 [btrfs] > >> [80577.151253] ?[] btrfs_init_new_buffer+0x7d/0x130 [btrfs] > >> [80577.151269] ?[] btrfs_alloc_free_block+0x104/0x110 [btrfs] > >> [80577.151285] ?[] __btrfs_cow_block+0x22a/0x8b0 [btrfs] > >> [80577.151300] ?[] generic_bin_search+0x162/0x1c0 [btrfs] > >> [80577.151315] ?[] btrfs_cow_block+0x156/0x200 [btrfs] > >> [80577.151330] ?[] btrfs_search_slot+0x1a7/0x910 [btrfs] > >> [80577.151333] ?[] irq_exit+0x27/0x60 > >> [80577.151336] ?[] do_IRQ+0x6b/0x80 > >> [80577.151354] ?[] read_extent_buffer+0xd5/0x170 [btrfs] > >> [80577.151369] ?[] btrfs_insert_empty_items+0x6d/0x410 [btrfs] > >> [80577.151385] ?[] btrfs_find_block_group+0xff/0x1a0 [btrfs] > >> [80577.151402] ?[] btrfs_new_inode+0x18d/0x360 [btrfs] > >> [80577.151420] ?[] btrfs_create+0x189/0x2a0 [btrfs] > >> [80577.151423] ?[] security_capable+0x9/0x10 > >> [80577.151427] ?[] vfs_create+0xcd/0x160 > >> [80577.151430] ?[] do_filp_open+0x5af/0x7d0 > >> [80577.151433] ?[] cp_new_stat64+0xf9/0x110 > >> [80577.151436] ?[] do_sys_open+0x4e/0xe0 > >> [80577.151439] ?[] sys_open+0x2c/0x40 > >> [80577.151442] ?[] sysenter_do_call+0x12/0x21 > >> [80577.151444] ---[ end trace 79cdc48bc88dedf7 ]--- > >> > >> > >> On Tue, Feb 24, 2009 at 5:02 PM, Hugo Mills > >> wrote: > >> > >>> ? This is essentially a repost of a mail I made last week, to which I > >>> didn't get a reply. > >>> > >>> ? I'm getting huge numbers of kernel warnings whilst using > >>> btrfs. They're all "warn_slowpath", and all seem to be in > >>> fs/btrfs/disk-io.c. I've included one typical example at the end of > >>> this mail. > >>> > >>> ? Kernel versions are 2.6.29-rc2, -rc4 and -rc6. > >>> > >>> ? If I do lots of writes to my btrfs filesystem (e.g. video > >>> encoding), I end up with a syslog in the tens-of-megabytes range. This > >>> makes logcheck an unhappy bunny... > >>> > >>> ? I don't know if this behaviour is expected, and everyone using > >>> btrfs simply puts up with it for now, or if it's something unusual > >>> that needs investigating. On the chance that it's the latter, I'm > >>> reporting it here. > >>> > >>> ? Hugo. > >>> > >>> Feb 23 21:45:42 vlad kernel: [ cut here ] > >>> Feb 23 21:45:42 vlad kernel: WARNING: at fs/btrfs/disk-io.c:815 > >>> clean_tree_block+0x9d/0xbb [btrfs]() > >>> Feb 23 21:45:42 vlad kernel: Hardware name: System Product Name > >>> Feb 23 21:45:42 vlad kernel: Modules linked in: tun ext3 jbd btrfs > >>> zlib_deflate tcp_diag inet_diag kqemu cpufreq_userspace ipv6 nfsd nfs > >>> lockd nfs_acl auth_rpcgss sunrpc af_packet bridge stp llc xfs exportfs > >>> it87 hwmon_vid powernow_k8 sbp2 ieee1394 ide_generic ide_gd_mod > >>> ide_cd_mod pcspkr evdev k8temp hwmon i2c_viapro i2c_core button dm_mirror > >>> dm_region_hash dm_log dm_snapshot dm_mod rai