Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Swâmi Petaramesh
Le 08/10/2012 00:47, Goffredo Baroncelli a écrit : Please could you clarify if you are using the autodefrag options when you have the performance problem ? I use autodefrag on all volumes systematically, except on volumes that I use for really big files that would always be defragmenting (i.e.

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Swâmi Petaramesh
Hi Martin, Le 07/10/2012 16:48, Martin Steigerwald a écrit : Where is this volume pool located on? On which drive(s)? All the concerned machines are laptops with a single physical HD... This could be 100 or more subvolumes / snapshots. Maybe slowness could be related to this one. That's a

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Swâmi Petaramesh
Le 07/10/2012 16:44, Martin Steigerwald a écrit : I think you need to backup, reformat and restore from backup for now. No way. 4 machines on each of which 2 to 4 different OSes are sharing the same BTRFS volume ! If I ever need to reformat/reinstall all this, the new format won't be BTRFS ! I

Re: [PATCH v2 2/2] btrfs-progs: limit the min value of total_bytes

2012-10-08 Thread Robin Dong
2012/10/1 David Sterba d...@jikos.cz: On Fri, Sep 28, 2012 at 11:02:40AM +0800, Robin Dong wrote: From: Robin Dong san...@taobao.com Using mkfs.btrfs like: mkfs.btrfs -b 1048576 /dev/sda will report error: mkfs.btrfs: volumes.c:796: btrfs_alloc_chunk: Assertion `!(ret)'

[PATCH v3 1/2] btrfs-progs: limit the max value of leafsize and nodesize

2012-10-08 Thread Robin Dong
From: Robin Dong san...@taobao.com Using mkfs.btrfs like: mkfs.btrfs -l 131072 /dev/sda will return no error, but after mount it, the dmesg will report: BTRFS: couldn't mount because metadata blocksize (131072) was too large The leafsize and nodesize are equal at present, so

[PATCH v3 2/2] btrfs-progs: limit the min value of total_bytes

2012-10-08 Thread Robin Dong
From: Robin Dong san...@taobao.com Using mkfs.btrfs like: mkfs.btrfs -b 1048576 /dev/sda will report error: mkfs.btrfs: volumes.c:796: btrfs_alloc_chunk: Assertion `!(ret)' failed. Aborted because the length of dev_extent is 4MB. For the single/single case it's 5MB

Re: btrfs send/receive review by vfs folks

2012-10-08 Thread Jan Schmidt
Hi Alex, On Thu, October 04, 2012 at 17:59 (+0200), Alex Lyakas wrote: as I promised, here is some code for you to look at. And quite a lot of it. I hadn't thought of such a big change when I wrote preferably in form of a patch. As a side note, your patch doesn't follow the general kernel

[PATCH v3 2/2] btrfs-progs: limit the min value of total_bytes

2012-10-08 Thread Robin Dong
From: Robin Dong san...@taobao.com Using mkfs.btrfs like: mkfs.btrfs -b 1048576 /dev/sda will report error: mkfs.btrfs: volumes.c:796: btrfs_alloc_chunk: Assertion `!(ret)' failed. Aborted because the length of dev_extent is 4MB. For the single/single case it's 5MB

Re: [PATCH v3 1/2] btrfs-progs: limit the max value of leafsize and nodesize

2012-10-08 Thread David Sterba
Hi, please let us know what changed from v2 - v3. The usual place for such comments is pointed below: On Mon, Oct 08, 2012 at 04:10:40PM +0800, Robin Dong wrote: [changelog ...] Signed-off-by: Robin Dong san...@taobao.com Reviewed-by: David Sterba d...@jikos.cz --- here ctree.h |6

Re: btrfs send/receive review by vfs folks

2012-10-08 Thread Alex Lyakas
Hi Jan, thanks for taking time to look at the code. On Mon, Oct 8, 2012 at 11:26 AM, Jan Schmidt list.bt...@jan-o-sch.net wrote: Hi Alex, On Thu, October 04, 2012 at 17:59 (+0200), Alex Lyakas wrote: as I promised, here is some code for you to look at. And quite a lot of it. I hadn't

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Goffredo Baroncelli
On Mon, Oct 8, 2012 at 8:08 AM, Swâmi Petaramesh sw...@petaramesh.org wrote: Le 08/10/2012 00:47, Goffredo Baroncelli a écrit : Please could you clarify if you are using the autodefrag options when you have the performance problem ? I use autodefrag on all volumes systematically, except on

Re: Wiki (scrub)

2012-10-08 Thread Goffredo Baroncelli
On Mon, Oct 8, 2012 at 12:51 AM, Alex a...@bpmit.com wrote: David Sterba dave at jikos.cz writes: Hi, On Sun, Oct 07, 2012 at 12:07:43PM +, Alex wrote: The official wiki seems to have lost references to scrub if not other commands. Sorry, I felt sure that scrub was listed on

working quota example?

2012-10-08 Thread matthieu Barthélemy
Hi all, I tried without success to get a working Btrfs+quota setup. I created a new Btrfs filesystem on a new partition, then activated quota management ('btrfs quota enable'), and created a few subvolumes. I created a qgroup (with id 100) with 'btrfs qgroup create', and tried to apply a quota on

Re: [PATCH 2/2 v3] Btrfs: snapshot-aware defrag

2012-10-08 Thread Liu Bo
On 10/03/2012 10:02 PM, Chris Mason wrote: On Tue, Sep 25, 2012 at 07:07:53PM -0600, Liu Bo wrote: On 09/26/2012 01:39 AM, Mitch Harder wrote: On Mon, Sep 17, 2012 at 4:58 AM, Liu Bo bo.li@oracle.com wrote: This comes from one of btrfs's project ideas, As we defragment files, we break any

Re: [PATCH 2/2 v3] Btrfs: snapshot-aware defrag

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 06:18:26AM -0600, Liu Bo wrote: On 10/03/2012 10:02 PM, Chris Mason wrote: On Tue, Sep 25, 2012 at 07:07:53PM -0600, Liu Bo wrote: On 09/26/2012 01:39 AM, Mitch Harder wrote: On Mon, Sep 17, 2012 at 4:58 AM, Liu Bo bo.li@oracle.com wrote: This comes from one of

[PATCH] Btrfs: remove repeated eb-pages check in, disk-io.c/csum_dirty_buffer

2012-10-08 Thread Wang Sheng-Hui
In csum_dirty_buffer, we first get eb from page-private. Then we check if the page is the first page of eb. Later we check it again. Remove the repeated check here. Signed-off-by: Wang Sheng-Hui shh...@gmail.com --- fs/btrfs/disk-io.c |8 +++- 1 files changed, 3 insertions(+), 5

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Swâmi Petaramesh
Le 08/10/2012 13:38, Goffredo Baroncelli a écrit : The autodefrag option is per filesystem not per subvolume. The settings of the first subvolueme is used also for the other ones. Uh ! So there is no interest in creating several subvols, some for which files should be autodefragged, and some

Re: btrfs send/receive review by vfs folks

2012-10-08 Thread Jan Schmidt
On Mon, October 08, 2012 at 13:38 (+0200), Alex Lyakas wrote: I realize this is a big change, and a new IOCTL has to be introduced in order not to break current user-kernel protocol. The pros as I see them: # One data-copy is avoided (no pipe). For WRITE commands two data-copies are avoided

Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
I'm tracking this bug here: https://bugzilla.redhat.com/show_bug.cgi?id=863978 Since approx. last week I'm seeing lots of failures in btrfs. The common factor seems to be that the filesystem is created (mkfs.btrfs /dev/sda1) and then it is immediately used -- eg. mounted or some btrfs subtool

fix btrfs-progs build

2012-10-08 Thread Christian Hesse
Hello everybody, man pages for btrfs-progs are compressed by gzip by default. In Makefile the variable GZIP is use, this evaluates to 'gzip gzip' on my system. From man gzip: The environment variable GZIP can hold a set of default options for gzip. These options are interpreted first and can

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: I'm tracking this bug here: https://bugzilla.redhat.com/show_bug.cgi?id=863978 Since approx. last week I'm seeing lots of failures in btrfs. The common factor seems to be that the filesystem is created (mkfs.btrfs

Re: fix btrfs-progs build

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 08:17:13AM -0600, Christian Hesse wrote: Hello everybody, man pages for btrfs-progs are compressed by gzip by default. In Makefile the variable GZIP is use, this evaluates to 'gzip gzip' on my system. From man gzip: The environment variable GZIP can hold a set of

Re: fix btrfs-progs build

2012-10-08 Thread Christian Hesse
Chris Mason chris.ma...@fusionio.com on Mon, 2012/10/08 10:29: On Mon, Oct 08, 2012 at 08:17:13AM -0600, Christian Hesse wrote: Hello everybody, man pages for btrfs-progs are compressed by gzip by default. In Makefile the variable GZIP is use, this evaluates to 'gzip gzip' on my system.

Re: fix btrfs-progs build

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 08:30:31AM -0600, Christian Hesse wrote: Chris Mason chris.ma...@fusionio.com on Mon, 2012/10/08 10:29: On Mon, Oct 08, 2012 at 08:17:13AM -0600, Christian Hesse wrote: Hello everybody, man pages for btrfs-progs are compressed by gzip by default. In Makefile

Re: Experiences: Why BTRFS had to yield for ZFS

2012-10-08 Thread Casper Bang
Thanks for taking the time to write this up follow through the thread. It's always interesting to hear situations where btrfs doesn't work well. There are three basic problems with the database workloads on btrfs. First is that we have higher latencies on writes because we are feeding

Re: fix btrfs-progs build

2012-10-08 Thread Christian Hesse
Chris Mason chris.ma...@fusionio.com on Mon, 2012/10/08 10:33: On Mon, Oct 08, 2012 at 08:30:31AM -0600, Christian Hesse wrote: Chris Mason chris.ma...@fusionio.com on Mon, 2012/10/08 10:29: On Mon, Oct 08, 2012 at 08:17:13AM -0600, Christian Hesse wrote: Hello everybody, man

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: I'm tracking this bug here: https://bugzilla.redhat.com/show_bug.cgi?id=863978 Since approx. last week I'm seeing lots of failures in btrfs. The

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: I'm tracking this bug here: https://bugzilla.redhat.com/show_bug.cgi?id=863978

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 11:04:19AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: I'm tracking this bug

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 09:15:14AM -0600, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at 11:04:19AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Swâmi Petaramesh
Hi again Goffredo, Le 08/10/2012 13:38, Goffredo Baroncelli a écrit : I fear that both the combination of autodefrag and the high number of snapshot could be the root-cause of the the bad performance. I've removed, on one of my machines, all snapshots but three per subvol (keeping the oldests

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Josef Bacik
On Sun, Oct 07, 2012 at 03:26:32AM -0600, Swâmi Petaramesh wrote: Hi, I have 4 machines, all converted to BTRFS about 6 months ago, now all running Ubuntu Quantal with kernel 3.5.0-17 The matter is that all these machines are now getting slower and slower everyday, every disk access

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Swâmi Petaramesh
Le 08/10/2012 18:09, Josef Bacik a écrit : Can you get sysrq+w when you are seeing slowness? Usually bootup slow times means you don't have space_cache enabled or your cache is being evicted for some reason, can you check dmesg after bootup for messages related to space cache? Thanks, I

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Josef Bacik
On Mon, Oct 08, 2012 at 10:15:51AM -0600, Swâmi Petaramesh wrote: Le 08/10/2012 18:09, Josef Bacik a écrit : Can you get sysrq+w when you are seeing slowness? Usually bootup slow times means you don't have space_cache enabled or your cache is being evicted for some reason, can you check

Re: BTRFS, getting darn slower everyday

2012-10-08 Thread Goffredo Baroncelli
On 10/08/2012 05:50 PM, Swâmi Petaramesh wrote: Hi again Goffredo, Le 08/10/2012 13:38, Goffredo Baroncelli a écrit : I fear that both the combination of autodefrag and the high number of snapshot could be the root-cause of the the bad performance. I've removed, on one of my machines, all

btrfs receive to subdirectory

2012-10-08 Thread Rory Campbell-Lange
I can send snapshots to volume, but not volume/dir. Please advise if what I am doing is incorrect. Rory Format usb3 disk and mount root@orchard:/bkp# mkfs.btrfs /dev/sdb1 WARNING! - Btrfs v0.20-rc1-37-g91d9eec IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org

Two Issues with Btrfs Delayed Cleaner Process (linux-next)

2012-10-08 Thread Mitch Harder
I've run across two issues with the delayed cleaner process running a kernel based on the 3.6.0 btrfs-next branch in Josef's git repository. (1) I'm getting an error when trying to list my subvolumes whenever the cleaner thread is running: # btrfs su li /mnt/benchmark/ ERROR: Failed to lookup

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 06:42:27PM +0200, David Sterba wrote: On Mon, Oct 08, 2012 at 04:15:14PM +0100, Richard W.M. Jones wrote: Unfortunately I'm struggling to reproduce this outside of our build system (Koji). I will keep you informed if I do manage to reproduce it locally. Adding

[PATCH 0/4] Resend: btrfs-progs: Some cosmetic changes (mainly) to btrfsck

2012-10-08 Thread Dieter Ries
Hi, I sent an earlier version of this patchset before, and didn't get any response, so here is the next try: This patch series implements some mainly cosmetic changes to. btrfs-progs, most in btrfsck. As this is my first contribution here, I'd kindly ask you for feedback, and if work like this

[PATCH 4/4] btrfs-progs: btrfsck: Remove binary error code output

2012-10-08 Thread Dieter Ries
This patch changes the output after checking a filesystem. Before, the default output was found x bytes used err is 0, where the last integer corresponds to the return value of check_root_refs(), which is either 1 or 0. Now this value is evaluated, and a message saying if errors were found or not

[PATCH 3/4] btrfs-progs: btrfsck: Print feedback about fscking to stdout.

2012-10-08 Thread Dieter Ries
Status reports of the checking process should be printed to stdout instead of stderr, as that is normal program output and not related to problems in btrfsck. This patch changes this behaviour and adds the output Done! after each of the parts. Signed-off-by: Dieter Ries m...@dieterries.net ---

[PATCH 1/4] btrfs-progs: Remove redundant Btrfs string from version string

2012-10-08 Thread Dieter Ries
In the first line of version.sh, $v was set to Btrfs vx.yy, and in the end Btrfs $v was echoed to the version.h file. This resulted in the version string Btrfs Btrfs vx.yy. This patch removes the second occurrence of Btrfs. Signed-off-by: Dieter Ries m...@dieterries.net --- version.sh |2 +-

Re: working quota example?

2012-10-08 Thread matthieu Barthélemy
On Mon, Oct 8, 2012 at 3:01 PM, Arne Jansen sensi...@gmx.net wrote: Please refer here http://sensille.com/qgroups.pdf for a discussion of the meaning of those values. Thanks, I already read your paper a few days ago, but it was a quick reading in order to try to understand the concepts.

Re: working quota example?

2012-10-08 Thread Arne Jansen
On 10/08/12 21:31, matthieu Barthélemy wrote: Are there any plan to maybe get a better 'btrfs quota show' output? Definitely. The first priority was to get the kernel part running, when that is settled, we can improve the user mode part. There's also still some work to do to make the tracking

Re: btrfs receive to subdirectory

2012-10-08 Thread Arne Jansen
On 10/08/12 18:30, Rory Campbell-Lange wrote: I can send snapshots to volume, but not volume/dir. Please advise if what I am doing is incorrect. Rory Format usb3 disk and mount root@orchard:/bkp# mkfs.btrfs /dev/sdb1 WARNING! - Btrfs v0.20-rc1-37-g91d9eec IS EXPERIMENTAL

Re: Experiences: Why BTRFS had to yield for ZFS

2012-10-08 Thread Avi Miller
Hi, On 09/10/2012, at 1:38 AM, Casper Bang casper.b...@gmail.com wrote: If you do have a suspicion or insight on the matter (perhaps work for Oracle, or know OUK?), of course we'd love a followup offline this list. I've sent an email to Casper to follow this up offline. Thanks, Avi --

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 04:15:14PM +0100, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at 11:04:19AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: On Mon, Oct 08, 2012 at

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 03:22:30PM -0600, Richard W.M. Jones wrote: I have now reproduced this bug locally. Adding sync() + fsync of each /dev/sd* device after the mkfs command does appear to fix the problem. However it's a little bit difficult to know for sure because I might just be

Re: [RFC][PATCH] Btrfs-progs, btrfsck: add block group check function

2012-10-08 Thread Miao Xie
On Tue, 2 Oct 2012 20:21:56 -0400, Chris Mason wrote: On Thu, Aug 02, 2012 at 04:14:37AM -0600, Miao Xie wrote: From: Chen Yang chenyang.f...@cn.fujitsu.com This patch adds the function to check correspondence between block group, chunk and device extent. Excellent, thank you. Could you