Re: Rebalancing RAID1
On Tue, 12 Feb 2013, Chris Murphy wrote:
On Feb 12, 2013, at 4:01 PM, Fredrik Tolf wrote:
mkfs.btrfs -d raid -m raid1 /dev/sd{d,e}1
Is that a typo? -d raid isn't valid.
Ah yes, sorry. That was a typo.
What do you get for:
btrfs fi df /mnt
$ sudo ./btrfs fi df /mnt
Data, RAID1: total=2.66TB, used=2.66TB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=388.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=4.00GB, used=3.66GB
Metadata: total=8.00MB, used=0.00
Please report the result for each drive:
smartctl -a /dev/sdX
As they're a bit long for mail, so see here:
There's not a whole lot to see, though.
smartctl -l scterc /dev/sdX
"Warning: device does not support SCT Error Recovery Control command"
Also, why does it say that the errors are occuring /dev/sdd1? Is it just
remembering the whole filesystem by that name since that's how I mounted it, or
is it still trying to access the old removed instance of that disk and is that,
then, why it's giving all these errors?
I suspect bad sectors at the moment.
Doesn't seem that way to me; partly because of the SMART data, and partly
because of the errors that were logged as the drive failed:
Feb 12 16:36:49 nerv kernel: [36769.546522] ata6.00: Ata error. fis:0x21
Feb 12 16:36:49 nerv kernel: [36769.550454] ata6: SError: { Handshk }
Feb 12 16:36:51 nerv kernel: [36769.554129] ata6.00: failed command: WRITE
FPDMA QUEUED
Feb 12 16:36:51 nerv kernel: [36769.559375] ata6.00: cmd
61/00:00:00:ec:2e/04:00:cd:00:00/40 tag 0 ncq 524288 out
Feb 12 16:36:51 nerv kernel: [36769.559375] res
41/84:d0:00:98:2e/84:00:cd:00:00/40 Emask 0x10 (ATA bus error)
Feb 12 16:36:51 nerv kernel: [36769.574831] ata6.00: status: { DRDY ERR }
Feb 12 16:36:52 nerv kernel: [36769.578867] ata6.00: error: { ICRC ABRT }
That's not typical for actual media problems, in my experience. :)
What kernel version?
Oh, sorry, it's 3.7.1. The system is otherwise a pretty much vanilla
Debian Squeeze (curreny Stable) that I've just compiled a newer kernel
(and btrfs-tools) for.
Thanks for replying!
--
Fredrik Tolf
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: check out if the swap device
Hi, All,
Thanks for advice.
On 2013/02/13 5:57, Zach Brown wrote:
So, I chose this one. (read /proc/swaps)
Sure, I think your change is good. I just think perhaps mkfs should also try
to open O_EXCL after all those other tests, as a last safety check.
I think mkfs should first try an O_EXCL open. If that works it doesn't
need to do any of this work to try and predict if the open will fail.
After it fails it can poke around a bit to try and give nice context for
why it failed. But it might not be able to because /proc/swaps is
fundamentally unreliable.
Then, how should we do?I have no idea...
file = av[optind++];
+ret = is_swap_device(file);
The input string is a CWD-realtive path. You'd at least want to use
realpath() to get it to a canonical name. So it's not full of junk like
"./" and "../../.." which won't be present in /proc/swaps.
+charbuf[1024];
Use PATH_MAX so it doesn't fail on absurd but allowed file names.
(Where on earth does 1024 come from?)
+if ((f = fopen("/proc/swaps", "r")) == NULL)
+return -errno;
As was pointed out, there's no reason this failure should stop mkfs.
/proc/swaps might not be available or allowed and I should still be able
to do an unpriviledged "./mkfs ./myfile; ./btrfs-debug-tree ./myfile".
+if (strcmp(file, buf) == 0) {
+ret = 1;
+break;
+}
The command line path that lead to the inode might not be the path for
the inode that is in /proc/swaps. Bind mounts, chroot, name spaces,
hard links, etc, all make it possible -- though unlikely -- that mkfs
simply won't be able to tell if the path names are related.
OK. I agree.
>
Also, /proc/swaps escapes whitespace in file names. So you could be
comparing "swap file" with "swap\040file".
I had forgotten this. Thank you for pointing it out.
+if ((stat(buf, &st_buf) == 0) && S_ISBLK(st_buf.st_mode) &&
+rdev == st_buf.st_rdev) {
+ret = 1;
+break;
+}
One possible alternative is to then try and open every swap file path to
see if it points to the same inode as the path mkfs was given. But you
might not have access to the paths and we're back to the unpriviledged
failure case.
I want to think about a new patch later. and, I will post it again if it
seems to make a sense.
Thanks,
Tsutomu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Integration branch of btrfs-progs 2013-02
Hi David, I really have concerns about the libification, in particular this commit: 6fc8b21 btrfs-progs: libify some parts of btrfs-progs The relevant snippets that concern me below: +libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \ + crc32c.h list.h kerncompat.h radix-tree.h extent-cache.h \ + extent_io.h ioctl.h ctree.h ... +headers = $(libbtrfs_headers) ... + $(INSTALL) -m644 $(headers) $(DESTDIR)$(incdir) I really don't think that all those headers should be exposed to the userspace. I think, to do it right, there should be a single or a few header files, like /usr/include/btrfs.h or /usr/include/libbtrfs.h or /usr/include/btrfs/xyz.h, that export only the parts that are really necessary for an application that wants to use libbtrfs. Does it really make sense to have btrfs-progs expose things like ctree.h or crc32c.h or even list.h to userspace?! Another reason of my concerns is that I've been trying to work on exporting the equivalent of ioctl.h, with the constants and structs needed to call btrfs-specific ioctls, from the kernel side, I already submitted a patch to export it from a Linux kernel build as /usr/include/linux/btrfs.h. I believe that's the right way to export that particular information. The other part of it is synchronizing the header files (and to some extent some source files, like ctree.c) between the kernel and btrfs-progs. The current version of those headers (and C source files) in btrfs-progs was a copy of the same files from the kernel tree that was edited to compile in userspace and then diverged from the copy of the same files in the kernel. We should try to unify those (I sent another patch with a suggestion of a script that would update those from the kernel git tree.) Exposing those to userspace now would only muddle that situation more. If you think you can build a new header file that is clean and contain only the prototypes and constants and structs that are strictly required to export functions in a libbtrfs, then I'd fully support this patchset. If it's going to export all the header files as it's doing now, then I wouldn't... Thanks, Filipe -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Rebalancing RAID1
On Feb 12, 2013, at 4:01 PM, Fredrik Tolf wrote:
>
> mkfs.btrfs -d raid -m raid1 /dev/sd{d,e}1
Is that a typo? -d raid isn't valid.
What do you get for:
btrfs fi df /mnt
Please report the result for each drive:
smartctl -a /dev/sdX
smartctl -l scterc /dev/sdX
>
> Also, why does it say that the errors are occuring /dev/sdd1? Is it just
> remembering the whole filesystem by that name since that's how I mounted it,
> or is it still trying to access the old removed instance of that disk and is
> that, then, why it's giving all these errors?
I suspect bad sectors at the moment. But it could be other things too. What
kernel version?
Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Rebalancing RAID1
Dear list,
I'm sorry if this is a dumb n3wb question, but I couldn't find anything
about it, so please bear with me.
I just decided to try BtrFS for the first time, to replace an old ReiserFS
data partition currently on a mdadm mirror. To do so, I'm using two 3 TB
disks that were initially detected as sdd and sde, on which I have a
single large GPT partition, so the devices I'm using for btrfs are sdd1
and sde1.
I created a filesystem on them using RAID1 from the start (mkfs.btrfs -d
raid -m raid1 /dev/sd{d,e}1), and started copying the data from the old
partition onto it during the night. As it happened, I immediately got
reason to try out BtrFS recovery because sometime during the copying
operation /dev/sdd had some kind of cable failure and was removed from the
system. A while later, however, it was apparently auto-redetected, this
time as /dev/sdi, and BtrFS seems to have inserted it back into the
filesystem somehow.
The current situation looks like this:
$ sudo ./btrfs fi show
Label: none uuid: 40d346bb-2c77-4a78-8803-1e441bf0aff7
Total devices 2 FS bytes used 1.64TB
devid1 size 2.73TB used 1.64TB path /dev/sdi1
devid2 size 2.73TB used 2.67TB path /dev/sde1
Btrfs v0.20-rc1-56-g6cd836d
As you can see, /dev/sdi1 has much less space used, which I can only
assume is because extents weren't allocated on it while it was off-line.
I'm now trying to remedy this, but I'm not sure if I'm doing it right.
What I'm doing is to run "btrfs fi bal start /mnt &", and it gives me a
ton of kernel messages that look like this:
Feb 12 22:57:16 nerv kernel: [59596.948464] btrfs: relocating block group
2879804932096 flags 17
Feb 12 22:57:45 nerv kernel: [59626.618280] btrfs_end_buffer_write_sync: 8
callbacks suppressed
Feb 12 22:57:45 nerv kernel: [59626.621893] lost page write due to I/O error on
/dev/sdd1
Feb 12 22:57:45 nerv kernel: [59626.621893] btrfs_dev_stat_print_on_error: 8
callbacks suppressed
Feb 12 22:57:45 nerv kernel: [59626.621893] btrfs: bdev /dev/sdd1 errs: wr
66339, rd 26, flush 1, corrupt 0, gen 0
Feb 12 22:57:45 nerv kernel: [59626.644110] lost page write due to I/O error on
/dev/sdd1
[Lots of the above, and occasionally a couple of lines like these]
Feb 12 22:57:48 nerv kernel: [59629.569278] btrfs: found 46 extents
Feb 12 22:57:50 nerv kernel: [59631.685067] btrfs_dev_stat_print_on_error: 5
callbacks suppressed
This barrage of messages combined with the fact that the rebalance is
going quite slowly (btrfs fi bal stat indicates about 1 extent per minute,
where an extent seems to be about 1 GB; which is several factors slower
than it took to copy the data onto the filesystem) leads me to think that
something is wrong. Is it, or should I just wait 2 days for it to
complete, ignoring the error?
Also, why does it say that the errors are occuring /dev/sdd1? Is it just
remembering the whole filesystem by that name since that's how I mounted
it, or is it still trying to access the old removed instance of that disk
and is that, then, why it's giving all these errors?
Thanks for reading!
--
Fredrik Tolf
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 13/21] btrfs-progs: build btrsfck to keep compatibility
The command 'btrfsck' is commonly used and we should build it by default. Signed-off-by: David Sterba --- Makefile | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 039f0dc..7a679d0 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,5 @@ CC = gcc +LN = ln AM_CFLAGS = -Wall -D_FILE_OFFSET_BITS=64 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 CFLAGS = -g -O1 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \ @@ -35,7 +36,7 @@ endif MAKEOPTS = --no-print-directory Q=$(Q) -progs = btrfsctl mkfs.btrfs btrfs-debug-tree btrfs-show btrfs-vol \ +progs = btrfsctl mkfs.btrfs btrfs-debug-tree btrfs-show btrfs-vol btrfsck \ btrfs btrfs-map-logical btrfs-image btrfs-zero-log btrfs-convert \ btrfs-find-root btrfs-restore btrfstune btrfs-show-super @@ -110,6 +111,11 @@ btrfs-show: $(objects) btrfs-show.o @echo "[LD] $@" $(Q)$(CC) $(CFLAGS) -o btrfs-show btrfs-show.o $(objects) $(LDFLAGS) $(LIBS) +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's named 'btrfsck' +btrfsck: btrfs + @echo "[LN] $@" + $(Q)$(LN) -f btrfs btrfsck + mkfs.btrfs: $(objects) mkfs.o @echo "[LD] $@" $(Q)$(CC) $(CFLAGS) -o mkfs.btrfs $(objects) mkfs.o $(LDFLAGS) $(LIBS) -lblkid @@ -171,7 +177,7 @@ install-man: clean : @echo "Cleaning" $(Q)rm -f $(progs) cscope.out *.o .*.d btrfs-convert btrfs-image btrfs-select-super \ - btrfs-zero-log btrfstune dir-test ioctl-test quick-test btrfs.static \ + btrfs-zero-log btrfstune dir-test ioctl-test quick-test btrfs.static btrfsck \ version.h $(Q)$(MAKE) $(MAKEOPTS) -C man $@ -- 1.8.0.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 17/21] btrfs-progs: libify some parts of btrfs-progs
External software wanting to use the functionality provided by the btrfs
send ioctl has a hard time doing so without replicating tons of work. Of
particular interest are functions like btrfs_read_and_process_send_stream()
and subvol_uuid_search(). As that functionality requires a bit more than
just send-stream.c and send-utils.c we have to pull in some other parts of
the progs package.
This patch adds code to the Makefile and headers to create a library,
libbtrfs which the btrfs command now links to.
Signed-off-by: Mark Fasheh
Signed-off-by: David Sterba
---
Makefile | 80 ++
btrfs-list.h | 4 +++
crc32c.h | 4 +++
ctree.h| 9 +++
extent-cache.h | 6 +
extent_io.h| 7 +
radix-tree.h | 4 +++
rbtree.h | 4 +++
send-utils.h | 5
9 files changed, 96 insertions(+), 27 deletions(-)
diff --git a/Makefile b/Makefile
index e83cf5b..1dc24e3 100644
--- a/Makefile
+++ b/Makefile
@@ -1,16 +1,20 @@
CC = gcc
LN = ln
-AM_CFLAGS = -Wall -D_FILE_OFFSET_BITS=64 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
+AM_CFLAGS = -Wall -D_FILE_OFFSET_BITS=64 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
-DBTRFS_FLAT_INCLUDES -fPIC
CFLAGS = -g -O1
objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
root-tree.o dir-item.o file-item.o inode-item.o \
- inode-map.o crc32c.o rbtree.o extent-cache.o extent_io.o \
+ inode-map.o extent-cache.o extent_io.o \
volumes.o utils.o btrfs-list.o btrfslabel.o repair.o \
- send-stream.o send-utils.o qgroup.o raid6.o
+ qgroup.o raid6.o
cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
cmds-restore.o
+libbtrfs_objects = send-stream.o send-utils.o rbtree.o btrfs-list.o crc32c.o
+libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \
+ crc32c.h list.h kerncompat.h radix-tree.h extent-cache.h \
+ extent_io.h ioctl.h ctree.h
CHECKFLAGS= -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise \
-Wuninitialized -Wshadow -Wundef
@@ -19,7 +23,10 @@ DEPFLAGS = -Wp,-MMD,$(@D)/.$(@F).d,-MT,$@
INSTALL = install
prefix ?= /usr/local
bindir = $(prefix)/bin
-LIBS=-luuid -lm -lz
+libdir = $(prefix)/lib
+incdir = $(prefix)/include/btrfs
+lib_LIBS = -luuid -lm -lz -L.
+LIBS = $(lib_LIBS) -lbtrfs
ifeq ("$(origin V)", "command line")
BUILD_VERBOSE = $(V)
@@ -50,6 +57,10 @@ STATIC_CFLAGS = $(CFLAGS) -ffunction-sections -fdata-sections
STATIC_LDFLAGS = -static -Wl,--gc-sections
STATIC_LIBS = $(LIBS) -lpthread
+libs = libbtrfs.so.0.1
+lib_links = libbtrfs.so.0 libbtrfs.so
+headers = $(libbtrfs_headers)
+
# make C=1 to enable sparse
ifdef C
check = sparse $(CHECKFLAGS)
@@ -72,38 +83,47 @@ all: version.h $(progs) manpages
# NOTE: For static compiles, you need to have all the required libs
# static equivalent available
#
-static: version.h btrfs.static
+static: version.h $(libs) btrfs.static
version.h:
$(Q)bash version.sh
-btrfs: $(objects) btrfs.o help.o $(cmds_objects)
+$(libs): $(libbtrfs_objects) $(lib_links) send.h
+ @echo "[LD] $@"
+ $(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(lib_LIBS) -shared
-Wl,-soname,libbtrfs.so -o libbtrfs.so.0.1
+
+$(lib_links):
+ @echo "[LN] $@"
+ $(Q)$(LN) -sf libbtrfs.so.0.1 libbtrfs.so.0
+ $(Q)$(LN) -sf libbtrfs.so.0.1 libbtrfs.so
+
+btrfs: $(objects) btrfs.o help.o $(cmds_objects) $(libs)
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) -o btrfs btrfs.o help.o $(cmds_objects) \
$(objects) $(LDFLAGS) $(LIBS) -lpthread
-btrfs.static: $(static_objects) btrfs.static.o help.static.o
$(static_cmds_objects)
+btrfs.static: $(static_objects) $(libs) btrfs.static.o help.static.o
$(static_cmds_objects)
@echo "[LD] $@"
$(Q)$(CC) $(STATIC_CFLAGS) -o btrfs.static btrfs.static.o help.static.o
$(static_cmds_objects) \
$(static_objects) $(STATIC_LDFLAGS) $(STATIC_LIBS)
-calc-size: $(objects) calc-size.o
+calc-size: $(objects) $(libs) calc-size.o
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) -o calc-size calc-size.o $(objects) $(LDFLAGS)
$(LIBS)
-btrfs-find-root: $(objects) find-root.o
+btrfs-find-root: $(objects) $(libs) find-root.o
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) -o btrfs-find-root find-root.o $(objects)
$(LDFLAGS) $(LIBS)
-btrfsctl: $(objects) btrfsctl.o
+btrfsctl: $(objects) $(libs) btrfsctl.o
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) -o btrfsctl btrfsctl.o $(objects) $(LDFLAGS) $(LIBS)
-btrfs-vol: $(objects) btrfs-vol.o
+btrfs-vol: $(objects) $(libs) btrfs-vol.o
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) -o btrfs-vol btrfs-vol.o $(obj
[PATCH 20/21] btrfs-progs: don't link binaries to a dynamic library
Linking 'btrfs' and other binaries against the dynamic library makes it
tedious to use directly from the git repo. This is useful for testing
various fixes, but now it'd need to also set LD_LIBRARY_PATH or install
the library to a known path.
Add a target for static library and use it for linking, the dynamic
library is to be used by external users.
Signed-off-by: David Sterba
---
Makefile | 15 +++
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/Makefile b/Makefile
index 6013fe2..84c39c0 100644
--- a/Makefile
+++ b/Makefile
@@ -1,5 +1,6 @@
CC = gcc
LN = ln
+AR = ar
AM_CFLAGS = -Wall -D_FILE_OFFSET_BITS=64 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
-DBTRFS_FLAT_INCLUDES -fPIC
CFLAGS = -g -O1
objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
@@ -26,7 +27,7 @@ bindir = $(prefix)/bin
libdir = $(prefix)/lib
incdir = $(prefix)/include/btrfs
lib_LIBS = -luuid -lm -lz -L.
-LIBS = $(lib_LIBS) -lbtrfs
+LIBS = $(lib_LIBS) $(libs_static)
ifeq ("$(origin V)", "command line")
BUILD_VERBOSE = $(V)
@@ -57,7 +58,9 @@ STATIC_CFLAGS = $(CFLAGS) -ffunction-sections -fdata-sections
STATIC_LDFLAGS = -static -Wl,--gc-sections
STATIC_LIBS = $(LIBS) -lpthread
-libs = libbtrfs.so.0.1
+libs_shared = libbtrfs.so.0.1
+libs_static = libbtrfs.a
+libs = $(libs_shared) $(libs_static)
lib_links = libbtrfs.so.0 libbtrfs.so
headers = $(libbtrfs_headers)
@@ -88,10 +91,14 @@ static: version.h $(libs) btrfs.static
version.h:
$(Q)bash version.sh
-$(libs): $(libbtrfs_objects) $(lib_links) send.h
+$(libs_shared): $(libbtrfs_objects) $(lib_links) send.h
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(lib_LIBS) -shared
-Wl,-soname,libbtrfs.so -o libbtrfs.so.0.1
+$(libs_static): $(libbtrfs_objects)
+ @echo "[AR] $@"
+ $(Q)$(AR) cru libbtrfs.a $(libbtrfs_objects)
+
$(lib_links):
@echo "[LN] $@"
$(Q)$(LN) -sf libbtrfs.so.0.1 libbtrfs.so.0
@@ -199,7 +206,7 @@ clean :
$(Q)rm -f $(progs) cscope.out *.o .*.d btrfs-convert btrfs-image
btrfs-select-super \
btrfs-zero-log btrfstune dir-test ioctl-test quick-test send-test
btrfs.static btrfsck \
version.h \
- $(libs) libbtrfs.so libbtrfs.so.0 libbtrfs.so.0.1
+ $(libs) libbtrfs.so libbtrfs.so.0 libbtrfs.so.0.1 libbtrfs.a
$(Q)$(MAKE) $(MAKEOPTS) -C man $@
install: $(libs) $(progs) install-man
--
1.8.0.2
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Integration branch of btrfs-progs 2013-02
Hi, a few more fixes and enhancements: check and restore subcommands, support for send mode NO_FILE_DATA and partial "libifying". The libify patchset porting was not straightforward (based on old sources) and I fixed it here up to my satisfaction. The changes were mostly small but there was quite a lot of them. I'll send patches as a followup to this mail for review in case I did non-trivial modifications. There's the patch converting build to autotools, but the conflict surface was too big so it's left out for now. (The patch backlog is not yet finished.) Thanks for testing, david --- Arvin Schnell (1): btrfs-progs: make libbtrfs usable from C++ David Sterba (2): btrfs-progs: build btrsfck to keep compatibility btrfs-progs: don't link binaries to a dynamic library Eric Sandeen (2): Btrfs-progs print more informative error when we fail to open a device btrfs-progs: remove unused bit-radix.[ch] files Ian Kumlien (6): Btrfs-progs: add static compile target Btrfs-progs: -U_FORTIFY_SOURCE before -D Btrfs-progs: Rename btrfsck.c -> cmds-check.c Btrfs-progs: add btrfsck functionality to btrfs Btrfs-progs: restore.c -> cmds-restore.c Btrfs-progs: add restore command to btrfs Ilya Dryomov (3): Btrfs-progs: move crc32c optimization init Btrfs-progs: add btrfsck name detection to btrfs Btrfs-progs: make 0 a valid usage filter argument Josef Bacik (3): Btrfs-progs: handle errors reading fs roots Btrfs-progs: fix double free of extent buffer Btrfs-progs: return an error if we can't find an fs root Mark Fasheh (3): btrfs-progs: Add support for BTRFS_SEND_FLAG_NO_FILE_DATA btrfs-progs: libify some parts of btrfs-progs btrfs-progs: add send-test Wang Sheng-Hui (1): btrfs-progs: code cleanup for root-tree.c/btrfs_del_root -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs
On 02/12/2013 11:52 PM, David Sterba wrote: > On Tue, Feb 12, 2013 at 07:01:33PM +0100, Goffredo Baroncelli wrote: >> On 02/12/2013 06:37 PM, Filipe Brandenburger wrote: >>> Hi, >>> >>> On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's named 'btrfsck' +btrfsck: btrfs + @echo "[CP] $@" + $(Q)cp btrfs btrfsck + >>> >>> I think the idea was that btrfsck becomes a link (either symbolic or >>> hardlink works) to btrfs... >>> >>> Maybe just replace cp with ln? >> >> I agree with Filipe, or even a script is reasonable. So we have only one >> binary to update, and we avoid the risk to have a version mismatch >> between btrfsck and btrfs. This could lead to a different behaviour >> when the user call btrfsck instead btrfs. Finally this could save some >> bytes of space. > > Ok, I'll replace it with a hardlink. A symlink is not reliable (cannot > be copied without breaking the path). ...mmm... the install command (invoked by the Makefile during the installation phase) doesn't seem to preserve both the hard-link and the soft-link: $ touch test $ ln test test2 $ ln -sf test lntest $ mkdir t3 $ install test2 t3/ $ install lntest t3/ $ ls -li lntest test test2 t3/test2 t3/lntest 3005857 lrwxrwxrwx 1 ghigo ghigo 4 Feb 13 00:03 lntest -> test 3005858 -rwxr-xr-x 1 ghigo ghigo 0 Feb 13 00:03 t3/lntest 3005854 -rwxr-xr-x 1 ghigo ghigo 0 Feb 13 00:00 t3/test2 3005852 -rw-r--r-- 2 ghigo ghigo 0 Feb 13 00:00 test 3005852 -rw-r--r-- 2 ghigo ghigo 0 Feb 13 00:00 test2 I think that a bash script is a better choice. >> Anyway my opinion would be to left this kind to decision to the >> distribution. We (as upstream) should only remove the old btrfsck and >> issue an WARNING/REMARK in the release note to notify this change. >> Unfortunately btrfsck is old; now we must provide an alternative file to >> overwrite this binary in order to avoid the mismatch above when the user >> is used to recompile the binary from the source. > > A warning is a good idea and it will start a deprecation period of > btrfsck as separate utility. Unil some point in future, I'd rather stay > conservative and let it exist. > > david > -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs
On Tue, Feb 12, 2013 at 07:01:33PM +0100, Goffredo Baroncelli wrote: > On 02/12/2013 06:37 PM, Filipe Brandenburger wrote: > > Hi, > > > > On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: > >> +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's > >> named 'btrfsck' > >> +btrfsck: btrfs > >> + @echo "[CP] $@" > >> + $(Q)cp btrfs btrfsck > >> + > > > > I think the idea was that btrfsck becomes a link (either symbolic or > > hardlink works) to btrfs... > > > > Maybe just replace cp with ln? > > I agree with Filipe, or even a script is reasonable. So we have only one > binary to update, and we avoid the risk to have a version mismatch > between btrfsck and btrfs. This could lead to a different behaviour > when the user call btrfsck instead btrfs. Finally this could save some > bytes of space. Ok, I'll replace it with a hardlink. A symlink is not reliable (cannot be copied without breaking the path). > Anyway my opinion would be to left this kind to decision to the > distribution. We (as upstream) should only remove the old btrfsck and > issue an WARNING/REMARK in the release note to notify this change. > Unfortunately btrfsck is old; now we must provide an alternative file to > overwrite this binary in order to avoid the mismatch above when the user > is used to recompile the binary from the source. A warning is a good idea and it will start a deprecation period of btrfsck as separate utility. Unil some point in future, I'd rather stay conservative and let it exist. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: accept zero for balance usage filter
On Tue, Feb 12, 2013 at 06:36:17PM +0200, Ilya Dryomov wrote: > Josef, please don't pull this one into btrfs-next, it's been deprecated > by "Btrfs: allow for selecting only completely empty chunks" from me. > David and I discussed this on IRC, he is OK with it. Ack. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC] fix async ordered operations flush deadlock
Hello,
So btrfs_commit_transaction does this
ret = btrfs_run_ordered_operations(root, 0);
which async flushes all inodes on the ordered operations list. The problem with
this is that we wait for this flushing to finish, so we end up with this
Task 1 Task 2 Task 3
start transaction
set trans_no_join
wait forever
commit
btrfs_run_ordered_operations
async flush inode cow_file_range
join_transaction
wait forever
wait forever
Task1 is waiting for the flushint to finish, task 2 is waiting for task 1 to
give up its num_writers, and task 3 is waiting to join the transaction. This
used to work fine because the flushing was done inline so we just took on the
current journal info of the guy who managed to race in and get a ref on the
transaction, but now we've gotten rid of that by doing it async. Here is a
basic bullshit patch that just moves the flushing below the "is somebody else
committing right now?" logic which will hopefully fix the problem, but it's a
shit patch but its 5:10 and I need to go make Liam dinner. I'll try to think of
a better solution between now and tomorrow, but I'm open to suggestions.
Thanks,
Josef
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 49c79b3..8c50495 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -1480,13 +1480,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle
*trans,
int should_grow = 0;
unsigned long now = get_seconds();
- ret = btrfs_run_ordered_operations(root, 0);
- if (ret) {
- btrfs_abort_transaction(trans, root, ret);
- btrfs_end_transaction(trans, root);
- return ret;
- }
-
/* Stop the commit early if ->aborted is set */
if (unlikely(ACCESS_ONCE(cur_trans->aborted))) {
ret = cur_trans->aborted;
@@ -1541,6 +1534,10 @@ int btrfs_commit_transaction(struct btrfs_trans_handle
*trans,
spin_unlock(&cur_trans->commit_lock);
wake_up(&root->fs_info->transaction_blocked_wait);
+ ret = btrfs_run_ordered_operations(root, 0);
+ if (ret)
+ goto cleanup_transaction;
+
spin_lock(&root->fs_info->trans_lock);
if (cur_trans->list.prev != &root->fs_info->trans_list) {
prev_trans = list_entry(cur_trans->list.prev,
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What can I do to make btrfs work?
On Tue, Feb 12, 2013 at 02:05:35PM -0700, Richard W.M. Jones wrote: > > Yes, this is inside a very recent KVM (qemu 1.3.0), using virtio-scsi > as the backing disk. Ok, can you please run this on your virtio device file? It will overwrite the first 256K, so don't do this on a file you care about. gcc -Wall -o vtest vtest.c ./vtest /dev/xxx I've attached vtest.c and gzip'd it just to make sure no mailers mess with my pretty code. -chris vtest.c.gz Description: Binary data
Re: btrfs and LVM snapshots (Re: kernel BUG at fs/btrfs/extent-tree.c:1772)
I can confirm that, even with a single-device btrfs filesystem. However I am curious why you want to use the lvm snapshot capability instead of the btrfs one. You can't use btrfs snapshots on a broken FS. LVM snapshots would be useful to save the original state, before any potentially destructive repair attempts, and for further analysis. Regards -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What can I do to make btrfs work?
On Tue, Feb 12, 2013 at 11:44:56AM -0800, Zach Brown wrote: > On Tue, Feb 12, 2013 at 06:54:49PM +, Richard W.M. Jones wrote: > > Btrfs has been broken for me for ages. I first reported it on this > > list 5 months ago[1]. Below is a very simple reproducer that anyone > > can run. > > The very simple reproducer doesn't fail over here on bare hardware for > me. > > # dmesg | grep -c 'device label TEST' > 808 > > (keeps it spinning) > > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > > unused block device! > > What block devices are you using? I ask because how it implements > caching affects this test. > > The theory, as I understand it, is that btrfs is issuing bio reads that > don't see the cached writes from mkfs. > > You'd never see this bug on loopback because it serves bio reads from > the cache that mkfs wrote to. > > I'm not seeing it on hardware because the filemap_write_and_wait() that > btrfs does in the kernel on mount is syncing the cached writes. The bio > reads then get the mkfs data from disk. > > If your block device has a cache that doesn't sync with btrfs calls > this, you'd see this problem.. but.. that'd be strange indeed. Chris' > test patch to sync from mkfs would probably help, but you'd still see > the problem if, say, you just wrote a btrfs image to the block device > and reasonably expected the kernel to find it on mount. > > So it'd be nice to know what devices you're using. Maybe some nutty > in-guest virt passthrough thing? It's virtio-scsi in a KVM (qemu 1.3.0) guest. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What can I do to make btrfs work?
On Tue, Feb 12, 2013 at 02:16:37PM -0500, Josef Bacik wrote: > On Tue, Feb 12, 2013 at 11:54:49AM -0700, Richard W.M. Jones wrote: > > Btrfs has been broken for me for ages. I first reported it on this > > list 5 months ago[1]. Below is a very simple reproducer that anyone > > can run. > > > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > > unused block device! > > > > -- > > #!/bin/sh - > > set -e > > while true; do > > parted -s -- /dev/sda mklabel msdos > > parted -s -- /dev/sda mkpart primary 64s -64s > > wipefs -a /dev/sda1 > > mkfs.btrfs --label TEST /dev/sda1 > > mount /dev/sda1 /sysroot > > touch /sysroot/foo > > mkdir /sysroot/bar > > umount /sysroot > > done > > -- > > > > On the latest 3.8.0 kernel, this fails immediately (at the mount), and > > on 3.7.x it usually fails after a very few iterations. I see a > > variety of errors, but the latest kernel error is: > > > > [8.474934] device label ROOT devid 1 transid 2 /dev/sda2 > > [8.570619] device label ROOT devid 1 transid 2 /dev/sda2 > > [8.581891] btrfs: disk space caching is enabled > > [8.594146] btrfs bad tree block start 0 4194304 > > [8.595144] btrfs: failed to read tree root on sda2 > > [8.605308] btrfs: open_ctree failed > > > > I would really like btrfs to work. What can I do? > > Been running this in a loop for 20 minutes with no issues, is this in a virt > guest or something? Thanks, Yes, this is inside a very recent KVM (qemu 1.3.0), using virtio-scsi as the backing disk. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: check out if the swap device
> >
> > So, I chose this one. (read /proc/swaps)
>
> Sure, I think your change is good. I just think perhaps mkfs should also try
> to open O_EXCL after all those other tests, as a last safety check.
I think mkfs should first try an O_EXCL open. If that works it doesn't
need to do any of this work to try and predict if the open will fail.
After it fails it can poke around a bit to try and give nice context for
why it failed. But it might not be able to because /proc/swaps is
fundamentally unreliable.
> >>> file = av[optind++];
> >>> +ret = is_swap_device(file);
The input string is a CWD-realtive path. You'd at least want to use
realpath() to get it to a canonical name. So it's not full of junk like
"./" and "../../.." which won't be present in /proc/swaps.
> >>> +charbuf[1024];
Use PATH_MAX so it doesn't fail on absurd but allowed file names.
(Where on earth does 1024 come from?)
> >>> +if ((f = fopen("/proc/swaps", "r")) == NULL)
> >>> +return -errno;
As was pointed out, there's no reason this failure should stop mkfs.
/proc/swaps might not be available or allowed and I should still be able
to do an unpriviledged "./mkfs ./myfile; ./btrfs-debug-tree ./myfile".
> >>> +if (strcmp(file, buf) == 0) {
> >>> +ret = 1;
> >>> +break;
> >>> +}
The command line path that lead to the inode might not be the path for
the inode that is in /proc/swaps. Bind mounts, chroot, name spaces,
hard links, etc, all make it possible -- though unlikely -- that mkfs
simply won't be able to tell if the path names are related.
Also, /proc/swaps escapes whitespace in file names. So you could be
comparing "swap file" with "swap\040file".
> >>> +if ((stat(buf, &st_buf) == 0) && S_ISBLK(st_buf.st_mode) &&
> >>> +rdev == st_buf.st_rdev) {
> >>> +ret = 1;
> >>> +break;
> >>> +}
One possible alternative is to then try and open every swap file path to
see if it points to the same inode as the path mkfs was given. But you
might not have access to the paths and we're back to the unpriviledged
failure case.
- z
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What can I do to make btrfs work?
On Tue, 12 Feb 2013 18:54:49 + "Richard W.M. Jones" wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > unused block device! I might be pointing out the most obvious here, but where does /dev/sda2 come from? Some more ideas for you to try dd if=/dev/zero of=/dev/sda bs=1M count=1 blockdev --rereadpt /dev/sda and then proceed again with your test loop. > -- > #!/bin/sh - > set -e > while true; do > parted -s -- /dev/sda mklabel msdos > parted -s -- /dev/sda mkpart primary 64s -64s > wipefs -a /dev/sda1 > mkfs.btrfs --label TEST /dev/sda1 > mount /dev/sda1 /sysroot ^^ sda1 > touch /sysroot/foo > mkdir /sysroot/bar > umount /sysroot > done > -- > > On the latest 3.8.0 kernel, this fails immediately (at the mount), and > on 3.7.x it usually fails after a very few iterations. I see a > variety of errors, but the latest kernel error is: > > [8.474934] device label ROOT devid 1 transid 2 /dev/sda2 > [8.570619] device label ROOT devid 1 transid 2 /dev/sda2 sda2? > [8.581891] btrfs: disk space caching is enabled > [8.594146] btrfs bad tree block start 0 4194304 > [8.595144] btrfs: failed to read tree root on sda2 > [8.605308] btrfs: open_ctree failed > > I would really like btrfs to work. What can I do? > > Rich. > > [1] http://article.gmane.org/gmane.comp.file-systems.btrfs/20257 > -- With respect, Roman signature.asc Description: PGP signature
Re: What can I do to make btrfs work?
On Tue, Feb 12, 2013 at 06:54:49PM +, Richard W.M. Jones wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. The very simple reproducer doesn't fail over here on bare hardware for me. # dmesg | grep -c 'device label TEST' 808 (keeps it spinning) > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > unused block device! What block devices are you using? I ask because how it implements caching affects this test. The theory, as I understand it, is that btrfs is issuing bio reads that don't see the cached writes from mkfs. You'd never see this bug on loopback because it serves bio reads from the cache that mkfs wrote to. I'm not seeing it on hardware because the filemap_write_and_wait() that btrfs does in the kernel on mount is syncing the cached writes. The bio reads then get the mkfs data from disk. If your block device has a cache that doesn't sync with btrfs calls this, you'd see this problem.. but.. that'd be strange indeed. Chris' test patch to sync from mkfs would probably help, but you'd still see the problem if, say, you just wrote a btrfs image to the block device and reasonably expected the kernel to find it on mount. So it'd be nice to know what devices you're using. Maybe some nutty in-guest virt passthrough thing? - z -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs-progs: move crc32c optimization init
Don't call crc32c_optimization_init() until we know that a command is
actually going to be invoked.
Signed-off-by: Ilya Dryomov
---
btrfs.c |4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/btrfs.c b/btrfs.c
index 687acec..7752bd6 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -261,8 +261,6 @@ int main(int argc, char **argv)
{
const struct cmd_struct *cmd;
- crc32c_optimization_init();
-
argc--;
argv++;
handle_options(&argc, &argv);
@@ -278,6 +276,8 @@ int main(int argc, char **argv)
handle_help_options_next_level(cmd, argc, argv);
+ crc32c_optimization_init();
+
fixup_argv0(argv, cmd->token);
exit(cmd->fn(argc, argv));
}
--
1.7.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What can I do to make btrfs work?
On Tue, Feb 12, 2013 at 11:54:49AM -0700, Richard W.M. Jones wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > unused block device! > > -- > #!/bin/sh - > set -e > while true; do > parted -s -- /dev/sda mklabel msdos > parted -s -- /dev/sda mkpart primary 64s -64s > wipefs -a /dev/sda1 > mkfs.btrfs --label TEST /dev/sda1 > mount /dev/sda1 /sysroot > touch /sysroot/foo > mkdir /sysroot/bar > umount /sysroot > done > -- > > On the latest 3.8.0 kernel, this fails immediately (at the mount), and > on 3.7.x it usually fails after a very few iterations. I see a > variety of errors, but the latest kernel error is: > > [8.474934] device label ROOT devid 1 transid 2 /dev/sda2 > [8.570619] device label ROOT devid 1 transid 2 /dev/sda2 > [8.581891] btrfs: disk space caching is enabled > [8.594146] btrfs bad tree block start 0 4194304 > [8.595144] btrfs: failed to read tree root on sda2 > [8.605308] btrfs: open_ctree failed > > I would really like btrfs to work. What can I do? Been running this in a loop for 20 minutes with no issues, is this in a virt guest or something? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: relax the block group size limit for bitmaps
Dave pointed out that xfstests 273 will tell you that it failed to load the space cache for a block group when it remounts. This is because we run out of space writing out the block group cache. This is ok and is working as it should, but let's try to be a bit nicer. This happens because the block group was 100mb, but bitmap entries cover 128mb, so we were only getting extent entries for this block group, which ended up being too many to fit in the free space cache. So relax the bitmap size requirements to block groups that are at least half the size a bitmap will cover or larger, that way we can still keep the amount of space used in the free space cache low enough to be able to write it out. With this patch I no longer fail to write out the free space cache. Thanks, Reported-by: David Sterba Signed-off-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 12 +--- 1 files changed, 9 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 0be7a87..c8090f1 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -1356,6 +1356,8 @@ static void recalculate_thresholds(struct btrfs_free_space_ctl *ctl) u64 bytes_per_bg = BITS_PER_BITMAP * ctl->unit; int max_bitmaps = div64_u64(size + bytes_per_bg - 1, bytes_per_bg); + max_bitmaps = max(max_bitmaps, 1); + BUG_ON(ctl->total_bitmaps > max_bitmaps); /* @@ -1636,10 +1638,14 @@ static bool use_bitmap(struct btrfs_free_space_ctl *ctl, } /* -* some block groups are so tiny they can't be enveloped by a bitmap, so -* don't even bother to create a bitmap for this +* The original block groups from mkfs can be really small, like 8 +* megabytes, so don't bother with a bitmap for those entries. However +* some block groups can be smaller than what a bitmap would cover but +* are still large enough that they could overflow the 32k memory limit, +* so allow those block groups to still be allowed to have a bitmap +* entry. */ - if (BITS_PER_BITMAP * ctl->unit > block_group->key.offset) + if (((BITS_PER_BITMAP * ctl->unit) >> 1) > block_group->key.offset) return false; return true; -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What can I do to make btrfs work?
On Tue, Feb 12, 2013 at 11:54:49AM -0700, Richard W.M. Jones wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > unused block device! > > -- > #!/bin/sh - > set -e > while true; do > parted -s -- /dev/sda mklabel msdos > parted -s -- /dev/sda mkpart primary 64s -64s > wipefs -a /dev/sda1 > mkfs.btrfs --label TEST /dev/sda1 > mount /dev/sda1 /sysroot > touch /sysroot/foo > mkdir /sysroot/bar > umount /sysroot > done > -- > > On the latest 3.8.0 kernel, this fails immediately (at the mount), and > on 3.7.x it usually fails after a very few iterations. I see a > variety of errors, but the latest kernel error is: > > [8.474934] device label ROOT devid 1 transid 2 /dev/sda2 > [8.570619] device label ROOT devid 1 transid 2 /dev/sda2 > [8.581891] btrfs: disk space caching is enabled > [8.594146] btrfs bad tree block start 0 4194304 > [8.595144] btrfs: failed to read tree root on sda2 > [8.605308] btrfs: open_ctree failed > > I would really like btrfs to work. What can I do? Hi Rich, Can you try the btrfs-progs raid56-experimental branch. It has this patch which was fixing things for me: https://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs.git;a=commit;h=8fe354744cd7b5c4f7a3314dcdbb5095192a032f I'm not 100% sure I've reproduced your exact problem, but I hope this is it. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
What can I do to make btrfs work?
Btrfs has been broken for me for ages. I first reported it on this list 5 months ago[1]. Below is a very simple reproducer that anyone can run. *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an unused block device! -- #!/bin/sh - set -e while true; do parted -s -- /dev/sda mklabel msdos parted -s -- /dev/sda mkpart primary 64s -64s wipefs -a /dev/sda1 mkfs.btrfs --label TEST /dev/sda1 mount /dev/sda1 /sysroot touch /sysroot/foo mkdir /sysroot/bar umount /sysroot done -- On the latest 3.8.0 kernel, this fails immediately (at the mount), and on 3.7.x it usually fails after a very few iterations. I see a variety of errors, but the latest kernel error is: [8.474934] device label ROOT devid 1 transid 2 /dev/sda2 [8.570619] device label ROOT devid 1 transid 2 /dev/sda2 [8.581891] btrfs: disk space caching is enabled [8.594146] btrfs bad tree block start 0 4194304 [8.595144] btrfs: failed to read tree root on sda2 [8.605308] btrfs: open_ctree failed I would really like btrfs to work. What can I do? Rich. [1] http://article.gmane.org/gmane.comp.file-systems.btrfs/20257 -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: check out if the swap device
On 02/12/2013 02:25 AM, Tsutomu Itoh wrote:
> Currently, the following commands succeed.
>
> # cat /proc/swaps
> FilenameTypeSizeUsed
> Priority
> /dev/sda3 partition 8388604 0 -1
> /dev/sdc8 partition 9765884 0 -2
> # mkfs.btrfs /dev/sdc8
>
> WARNING! - Btrfs v0.20-rc1-165-g82ac345 IS EXPERIMENTAL
> WARNING! - see http://btrfs.wiki.kernel.org before using
>
> fs created label (null) on /dev/sdc8
> nodesize 4096 leafsize 4096 sectorsize 4096 size 9.31GB
> Btrfs v0.20-rc1-165-g82ac345
> # btrfs fi sh /dev/sdc8
> Label: none uuid: fc0bdbd0-7eed-460f-b4e9-131273b66df2
> Total devices 1 FS bytes used 28.00KB
> devid1 size 9.31GB used 989.62MB path /dev/sdc8
>
> Btrfs v0.20-rc1-165-g82ac345
> #
>
> But we should check out the swap device. So fixed it.
>
> Signed-off-by: Tsutomu Itoh
> ---
> (this patch is based on Chris's raid56-experimental branch)
> ---
> mkfs.c | 18 ++
> utils.c | 49 +
> utils.h | 1 +
> 3 files changed, 68 insertions(+)
>
> diff --git a/mkfs.c b/mkfs.c
> index 2d3c2af..fdc3373 100644
> --- a/mkfs.c
> +++ b/mkfs.c
> @@ -1366,6 +1366,15 @@ int main(int ac, char **av)
>
> if (source_dir == 0) {
> file = av[optind++];
> + ret = is_swap_device(file);
> + if (ret < 0) {
> + fprintf(stderr, "error checking %s status\n", file);
> + exit(1);
> + }
The fact that it is not possible to perform a check shouldn't prohibit
to run a mkfs.btrfs.
It is possible to add a switch to bypass this kind of checks ? We should
allow the user to be not limited by the fact that the check fails. I am
thinking to a "rescue" scenario like boot in a single mode where not al
filesystem are mounted.
I am referring to all the "safety" check not this one only.
BR
G.Baroncelli
[...]
--
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs
On 02/12/2013 06:37 PM, Filipe Brandenburger wrote: > Hi, > > On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: >> +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's >> named 'btrfsck' >> +btrfsck: btrfs >> + @echo "[CP] $@" >> + $(Q)cp btrfs btrfsck >> + > > I think the idea was that btrfsck becomes a link (either symbolic or > hardlink works) to btrfs... > > Maybe just replace cp with ln? I agree with Filipe, or even a script is reasonable. So we have only one binary to update, and we avoid the risk to have a version mismatch between btrfsck and btrfs. This could lead to a different behaviour when the user call btrfsck instead btrfs. Finally this could save some bytes of space. Anyway my opinion would be to left this kind to decision to the distribution. We (as upstream) should only remove the old btrfsck and issue an WARNING/REMARK in the release note to notify this change. Unfortunately btrfsck is old; now we must provide an alternative file to overwrite this binary in order to avoid the mismatch above when the user is used to recompile the binary from the source. BR Goffredo -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: check out if the swap device
On 2/11/13 11:50 PM, Tsutomu Itoh wrote:
> Hi, Eric,
>
> Thanks for your comment.
>
> On 2013/02/12 13:22, Eric Sandeen wrote:
>> On 2/11/13 7:25 PM, Tsutomu Itoh wrote:
>>> Currently, the following commands succeed.
>>>
>>> # cat /proc/swaps
>>> FilenameTypeSizeUsed
>>> Priority
>>> /dev/sda3 partition 8388604 0 -1
>>> /dev/sdc8 partition 9765884 0 -2
>>> # mkfs.btrfs /dev/sdc8
>>>
>>> WARNING! - Btrfs v0.20-rc1-165-g82ac345 IS EXPERIMENTAL
>>> WARNING! - see http://btrfs.wiki.kernel.org before using
>>>
>>> fs created label (null) on /dev/sdc8
>>> nodesize 4096 leafsize 4096 sectorsize 4096 size 9.31GB
>>> Btrfs v0.20-rc1-165-g82ac345
>>> # btrfs fi sh /dev/sdc8
>>> Label: none uuid: fc0bdbd0-7eed-460f-b4e9-131273b66df2
>>> Total devices 1 FS bytes used 28.00KB
>>> devid1 size 9.31GB used 989.62MB path /dev/sdc8
>>>
>>> Btrfs v0.20-rc1-165-g82ac345
>>> #
>>>
>>> But we should check out the swap device. So fixed it.
>>
>> I guess it's nice to parse /proc/swaps to be able to offer the
>
>> helpful error message in this case. (though I wonder how long
>> /proc/swaps will be available, and in this format? Does it count
>> as ABI?)
>
> Umm, I don't know how long /proc/swaps will be available, too...
I guess it is good enough for e2fsprogs :)
>>
>> Your implementation looks just like the one in e2fsprogs, so it should
>> work fine.
>
> Yes.
>
>>
>> But I also wonder if overall it would be safest to open the device O_EXCL,
>> which would fail with EBUSY if it were in use for swap, or mounted,
>> or opened O_EXCL by another process for any other reason:
>
> But details of the error cannot be notified when O_EXCL is used. and,
> after is_swap_device(), check_mounted() check state of the mount or not.
>
> So, I chose this one. (read /proc/swaps)
Sure, I think your change is good. I just think perhaps mkfs should also try
to open O_EXCL after all those other tests, as a last safety check.
I can take a look at the code & send that patch if it seems to make sense.
Thanks,
-Eric
> Thanks,
> Tsutomu
>
>>
>> [root@host tmp]# cat /proc/swaps
>> FilenameTypeSizeUsedPriority
>> /dev/sda3 partition2048280822616-1
>>
>> [root@host tmp]# strace -e open ./test
>> open("/etc/ld.so.cache", O_RDONLY) = 3
>> open("/lib64/libc.so.6", O_RDONLY) = 3
>> open("/dev/sda3", O_RDWR|O_EXCL)= -1 EBUSY (Device or resource busy)
>> open: Device or resource busy
>>
>> -Eric
>>
>>> Signed-off-by: Tsutomu Itoh
>>> ---
>>> (this patch is based on Chris's raid56-experimental branch)
>>> ---
>>> mkfs.c | 18 ++
>>> utils.c | 49 +
>>> utils.h | 1 +
>>> 3 files changed, 68 insertions(+)
>>>
>>> diff --git a/mkfs.c b/mkfs.c
>>> index 2d3c2af..fdc3373 100644
>>> --- a/mkfs.c
>>> +++ b/mkfs.c
>>> @@ -1366,6 +1366,15 @@ int main(int ac, char **av)
>>>
>>> if (source_dir == 0) {
>>> file = av[optind++];
>>> +ret = is_swap_device(file);
>>> +if (ret < 0) {
>>> +fprintf(stderr, "error checking %s status\n", file);
>>> +exit(1);
>>> +}
>>> +if (ret == 1) {
>>> +fprintf(stderr, "%s is a swap device\n", file);
>>> +exit(1);
>>> +}
>>> ret = check_mounted(file);
>>> if (ret < 0) {
>>> fprintf(stderr, "error checking %s mount status\n", file);
>>> @@ -1461,6 +1470,15 @@ int main(int ac, char **av)
>>> int old_mixed = mixed;
>>>
>>> file = av[optind++];
>>> +ret = is_swap_device(file);
>>> +if (ret < 0) {
>>> +fprintf(stderr, "error checking %s status\n", file);
>>> +exit(1);
>>> +}
>>> +if (ret == 1) {
>>> +fprintf(stderr, "%s is a swap device\n", file);
>>> +exit(1);
>>> +}
>>> ret = check_mounted(file);
>>> if (ret < 0) {
>>> fprintf(stderr, "error checking %s mount status\n",
>>> diff --git a/utils.c b/utils.c
>>> index f9ee812..0c551a0 100644
>>> --- a/utils.c
>>> +++ b/utils.c
>>> @@ -1386,3 +1386,52 @@ int get_fs_info(int fd, char *path, struct
>>> btrfs_ioctl_fs_info_args *fi_args,
>>>
>>> return 0;
>>> }
>>> +
>>> +/*
>>> + * Checks if the swap device or not.
>>> + * Returns 1 if the swap device, < 0 on error or 0 if not the swap device.
>>> + */
>>> +int is_swap_device(const char *file)
>>> +{
>>> +FILE*f;
>>> +struct statst_buf;
>>> +charbuf[1024];
>>> +char*cp;
>>> +dev_trdev;
>>> +intret = 0;
>>> +
>>> +if (stat(file, &st_buf) < 0)
>>> +return -errno;
>>> +if (!S_ISBLK(st_buf.st_mode))
>>> +return 0;
>>> +
>>> +rdev =
Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs
Hi, On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: > +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's > named 'btrfsck' > +btrfsck: btrfs > + @echo "[CP] $@" > + $(Q)cp btrfs btrfsck > + I think the idea was that btrfsck becomes a link (either symbolic or hardlink works) to btrfs... Maybe just replace cp with ln? Cheers, Filipe -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs-progs: make 0 a valid usage filter argument
This is a progs counterpart to a "Btrfs: allow for selecting only
completely empty chunks". usage=0 now means "select only only
completely empty chunks and nothing else".
Signed-off-by: Ilya Dryomov
---
cmds-balance.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/cmds-balance.c b/cmds-balance.c
index 38a7426..1f888db 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -159,7 +159,7 @@ static int parse_filters(char *filters, struct
btrfs_balance_args *args)
return 1;
}
if (parse_u64(value, &args->usage) ||
- args->usage < 1 || args->usage > 100) {
+ args->usage > 100) {
fprintf(stderr, "Invalid usage argument: %s\n",
value);
return 1;
--
1.7.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs
On Fri, Feb 08, 2013 at 01:36:58AM +0100, Ian Kumlien wrote: > -btrfsck: $(objects) btrfsck.o > - @echo "[LD] $@" > - $(Q)$(CC) $(CFLAGS) -o btrfsck btrfsck.o $(objects) $(LDFLAGS) $(LIBS) > - Do we want to get rid of the btrfsck binary completely? Using the term 'btrsfck' is common and anybody compiling sources from git needs to cp or ln btrfs -> btrfsck. Let's make it automatic, I'll add a makefile rule: --- a/Makefile +++ b/Makefile @@ -42,7 +42,7 @@ endif MAKEOPTS = --no-print-directory Q=$(Q) -progs = btrfsctl mkfs.btrfs btrfs-debug-tree btrfs-show btrfs-vol \ +progs = btrfsctl mkfs.btrfs btrfs-debug-tree btrfs-show btrfs-vol btrfsck \ btrfs btrfs-map-logical btrfs-image btrfs-zero-log btrfs-convert \ btrfs-find-root btrfstune btrfs-show-super @@ -125,6 +125,11 @@ btrfs-show: $(objects) $(libs) btrfs-show.o @echo "[LD] $@" $(Q)$(CC) $(CFLAGS) -o btrfs-show btrfs-show.o $(objects) $(LDFLAGS) $(LIBS) +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's named 'btrfsck' +btrfsck: btrfs + @echo "[CP] $@" + $(Q)cp btrfs btrfsck + mkfs.btrfs: $(objects) $(libs) mkfs.o @echo "[LD] $@" $(Q)$(CC) $(CFLAGS) -o mkfs.btrfs $(objects) mkfs.o $(LDFLAGS) $(LIBS) -lblkid @@ -186,7 +191,7 @@ install-man: clean : @echo "Cleaning" $(Q)rm -f $(progs) cscope.out *.o .*.d btrfs-convert btrfs-image btrfs-select-super \ - btrfs-zero-log btrfstune dir-test ioctl-test quick-test btrfs.static \ + btrfs-zero-log btrfstune dir-test ioctl-test quick-test btrfs.static btrfsck \ version.h \ $(libs) libbtrfs.so libbtrfs.so.0 libbtrfs.so.0.1 $(Q)$(MAKE) $(MAKEOPTS) -C man $@ --- david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: accept zero for balance usage filter
On Mon, Feb 11, 2013 at 01:33:52PM +0100, David Sterba wrote: > The condition can be relaxed to accept also 0 which will delete > unoccupied chunks and does not need space for the actual data > relocation. Until there is an automatic empty chunk reclaim, we > can use this as a last resort option under enospc. > > CC: Ilya Dryomov > Signed-off-by: David Sterba Josef, please don't pull this one into btrfs-next, it's been deprecated by "Btrfs: allow for selecting only completely empty chunks" from me. David and I discussed this on IRC, he is OK with it. Thanks, Ilya -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: allow for selecting only completely empty chunks
Enhance balance usage filter by making it possible to balance out only completely empty chunks. Today, usage filter properly acts on values from 1 to 99 inclusive, usage=100 selects all chunks, and usage=0 selects no chunks. This commit changes the usage=0 case: the new meaning is to restripe only completely empty chunks and nothing else. Suggested-by: David Sterba Signed-off-by: Ilya Dryomov --- fs/btrfs/volumes.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 15f6efd..7e41619 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -2615,7 +2615,7 @@ static int chunk_usage_filter(struct btrfs_fs_info *fs_info, u64 chunk_offset, chunk_used = btrfs_block_group_used(&cache->item); if (bargs->usage == 0) - user_thresh = 0; + user_thresh = 1; else if (bargs->usage > 100) user_thresh = cache->key.offset; else -- 1.7.9.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: eliminate a use-after-free in btrfs_balance()
Commit 5af3e8cc introduced a use-after-free at volumes.c:3139: bctl is freed
above in __cancel_balance() in all cases except for balance pause. Fix this
by moving the offending check a couple statements above, the meaning of the
check is preserved.
Reported-by: Chris Mason
Signed-off-by: Ilya Dryomov
---
fs/btrfs/volumes.c | 10 +-
1 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 15f6efd..5d25aed 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3126,6 +3126,11 @@ int btrfs_balance(struct btrfs_balance_control *bctl,
mutex_lock(&fs_info->balance_mutex);
atomic_dec(&fs_info->balance_running);
+ if (bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) {
+ fs_info->num_tolerated_disk_barrier_failures =
+ btrfs_calc_num_tolerated_disk_barrier_failures(fs_info);
+ }
+
if (bargs) {
memset(bargs, 0, sizeof(*bargs));
update_ioctl_balance_args(fs_info, 0, bargs);
@@ -3136,11 +3141,6 @@ int btrfs_balance(struct btrfs_balance_control *bctl,
__cancel_balance(fs_info);
}
- if (bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) {
- fs_info->num_tolerated_disk_barrier_failures =
- btrfs_calc_num_tolerated_disk_barrier_failures(fs_info);
- }
-
wake_up(&fs_info->balance_wait_q);
return ret;
--
1.7.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
On Tue, Feb 12, 2013 at 08:16:49AM -0700, Kaspar Schleiser wrote: > Hey Chris, > > On 02/02/2013 05:02 PM, Chris Mason wrote: > > Btrfs -- 604MB/s > > MD-- 162MB/s > > > > > > MD -- 800MB/s very little system time > > Btrfs -- 3.8GB/s one CPU mostly pegged > > > Btrfs -- 380MB/s seen by fio > > MD-- 174MB/s seen by fio > > > Creating 12 million files on Btrfs raid5 took 226 seconds, vs 485 > > seconds on MD. > > Do I read these numbers incorrectly, or does even this first iteration > of btrfs' raid5/6 code run circles around MD? Yes and no. Most of the differences were on flash, and really it just looks like MD needs tuning for IO latency and concurrency. There are some MD patches for this recently to add more threads for parity calculations, and these solve some throughput problems. But one thing that we've proven with btrfs is that helper threads mean more IO latencies. So the MD code probably needs some short cuts to do the parity inline as well. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/6] Btrfs-progs: add the rescue section to btrfs
On Fri, Feb 08, 2013 at 01:37:02AM +0100, Ian Kumlien wrote: > the btrfs command now lists: > btrfs rescue select-super -s > Select a superblock > btrfs rescue dump-super > Dump a superblock to disk > btrfs rescue debug-tree [options] > Debug the filesystem The only candidate into rescue section I currently see is 'zero-log'. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
Hey Chris, On 02/02/2013 05:02 PM, Chris Mason wrote: > Btrfs -- 604MB/s > MD-- 162MB/s > > > MD -- 800MB/s very little system time > Btrfs -- 3.8GB/s one CPU mostly pegged > Btrfs -- 380MB/s seen by fio > MD-- 174MB/s seen by fio > Creating 12 million files on Btrfs raid5 took 226 seconds, vs 485 > seconds on MD. Do I read these numbers incorrectly, or does even this first iteration of btrfs' raid5/6 code run circles around MD? Thanks for all the work! Kaspar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: check out if the swap device
On Tue, Feb 12, 2013 at 10:25:23AM +0900, Tsutomu Itoh wrote: > Currently, the following commands succeed. > > # cat /proc/swaps > FilenameTypeSizeUsed > Priority > /dev/sda3 partition 8388604 0 -1 > /dev/sdc8 partition 9765884 0 -2 > # mkfs.btrfs /dev/sdc8 if a swap device is backed by a file, mkfs succeeds: FilenameTypeSizeUsed Priority /dev/sda15 partition 4232016 0 -2 /mnt/swap file10236 0 -3 $ mkfs.btrfs /mnt/swap WARNING! - Btrfs 0.20 IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata chunk of size 1048576 fs created label (null) on /mnt/swap nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs 0.20 --- Please add check for this as well. thanks, david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: fix the deadlock between the transaction attach and commit
On Thu, Feb 07, 2013 at 11:55:51PM -0700, Miao Xie wrote: > Here is the whole story: > Trans_Attach_Task Trans_Commit_Task > btrfs_commit_transaction() >|->wait writers to be 1 > btrfs_attach_transaction() | > btrfs_commit_transaction() | >| |->set trans_no_join to 1 >| | (close join transaction) >|->btrfs_run_ordered_operations | > (Those ordered operations| >are added when releasing| >file) | >|->btrfs_join_transaction() | > |->wait_commit() | >|->wait writers to be 1 I'm just dropping this patch, the way you describe this deadlock can't happen since the second btrfs_join_transaction() would see theres already a transaction in current->journal_info and use that and not do wait_commit(). If you observed a deadlock like this then look at it again, there is something else going wrong. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: -U_FORTIFY_SOURCE before -D
On Sat, Feb 09, 2013 at 09:02:06PM +0300, Sergei Trofimovich wrote: > On Sat, 9 Feb 2013 00:30:21 +0100 > Ian Kumlien wrote: > > > My builds are cluttered with: > > :0:0: warning: "_FORTIFY_SOURCE" redefined [enabled by > > default] > > > > Which makes it hard to tell if something breaks or not. > > > I don't know about you, but bilding with > > GCC 4.7.2 on gentoo, this is a issue. > > Unfortunately it's a gentoo specific gcc bug. I'd suggest reporting > it to bugs.gentoo.org. That -D by default breaks linux's perf as well > (due to -D_FOTIFY_SOURCE=2 -Werror in perf sources). > > Current workaround is to pass AM_CFLAGS to make: > make AM_CFLAGS=stuff_w/o_fortify_source > It's what gentoo's live ebuild does. The fortify flag was added without a special reason, it's good to have it. I'm ok with adding the forced undefine to keep the build quiet and working as specified in the Makefile. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: add static compile target
On Sat, Feb 09, 2013 at 12:19:29AM +0100, Ian Kumlien wrote: > +# Define static compilation flags > +STATIC_CFLAGS = $(CFLAGS) -static -ffunction-sections -fdata-sections > +STATIC_LDFLAGS = -Wl,--gc-sections > +STATIC_LIBS = $(LIBS) -lpthread > +btrfs.static: $(static_objects) btrfs.static.o help.static.o > $(static_cmds_objects) > + @echo "[LD] $@" > + $(Q)$(CC) $(STATIC_CFLAGS) -o btrfs.static btrfs.static.o help.static.o > $(static_cmds_objects) \ > + $(static_objects) $(STATIC_LDFLAGS) $(STATIC_LIBS) -static belongs into LDFLAGS, it's propagated to the link phase through CFLAGS which is not IMO clean. As the point is to make the static version as small as possible I think it's ok to separate the built objects and name the binary differently from the normal one. Also, from distro side, I can build both targets from within one package without nonstandard build steps (make, copy, clean, make, copy/rename). I'll fix the -static flag locally and add this patch to integration. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: fix the deadlock between the transaction attach and commit
On Mon, Feb 11, 2013 at 03:35:37PM -0500, Josef Bacik wrote: > or something like that. Me and kdave reproduced by running 274 in a loop, it > happpened pretty quick. I'd fix it myself but I have to leave my house for > people to come look at it. If you haven't fixed this by tomorrow I'll fix it > up. Thanks, I found 224 stuck with this [ 3840.176147] INFO: task btrfs-transacti:3692 blocked for more than 480 seconds. [ 3840.176157] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3840.176162] btrfs-transacti D 0 3692 2 0x [ 3840.176172] 880103a3bd98 0046 880103a3a010 [ 3840.176183] 880103a3a000 880103a3a010 880103a3a000 880103a3a000 [ 3840.176190] 880103a3a000 880103a3bfd8 880102814800 81a14420 [ 3840.176200] Call Trace: [ 3840.176218] [] ? enqueue_entity+0x229/0xa40 [ 3840.176230] [] schedule+0x24/0x70 [ 3840.176296] [] wait_current_trans+0xc5/0x110 [btrfs] [ 3840.176306] [] ? wake_up_bit+0x40/0x40 [ 3840.176361] [] start_transaction+0x228/0x4c0 [btrfs] [ 3840.176400] [] btrfs_attach_transaction+0x12/0x20 [btrfs] [ 3840.176434] [] transaction_kthread+0x167/0x220 [btrfs] [ 3840.176476] [] ? __btree_submit_bio_start+0x10/0x10 [btrfs] [ 3840.176494] [] kthread+0xc6/0xd0 [ 3840.176506] [] ? kthread_freezable_should_stop+0x70/0x70 [ 3840.176514] [] ret_from_fork+0x7c/0xb0 [ 3840.176522] [] ? kthread_freezable_should_stop+0x70/0x70 [ 3840.176528] INFO: task rm:15763 blocked for more than 480 seconds. [ 3840.176531] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3840.176534] rm D 0002 0 15763 3517 0x [ 3840.176544] 8801071b7c88 0086 0002000280da 8801071b6010 [ 3840.176551] 8801071b6000 8801071b6010 8801071b6000 8801071b6000 [ 3840.176558] 8801071b6000 8801071b7fd8 8800be5762c0 88012af063c0 [ 3840.176567] Call Trace: [ 3840.176578] [] ? __alloc_pages_nodemask+0x162/0x250 [ 3840.176608] [] ? reserve_metadata_bytes+0x378/0x490 [btrfs] [ 3840.176622] [] schedule+0x24/0x70 [ 3840.176664] [] wait_current_trans+0xc5/0x110 [btrfs] [ 3840.176671] [] ? wake_up_bit+0x40/0x40 [ 3840.176721] [] start_transaction+0x228/0x4c0 [btrfs] [ 3840.176769] [] btrfs_start_transaction+0x13/0x20 [btrfs] [ 3840.176812] [] __unlink_start_trans+0x7e/0x490 [btrfs] [ 3840.176832] [] ? __d_lookup+0xa7/0x170 [ 3840.176839] [] ? handle_pte_fault+0x91/0x200 [ 3840.176849] [] ? security_inode_permission+0x19/0x20 [ 3840.176893] [] btrfs_unlink+0x36/0xd0 [btrfs] [ 3840.176899] [] vfs_unlink+0xb4/0x110 [ 3840.176905] [] do_unlinkat+0x233/0x240 [ 3840.176912] [] sys_unlinkat+0x1d/0x40 [ 3840.176918] [] system_call_fastpath+0x16/0x1b mounted with noatime,space_cache david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] btrfs-progs: introduce kernelsrc directory and script to update it
The kernelsrc directory will contain a copy of the Btrfs-related files from the Linux kernel source tree (mainly the files in fs/btrfs/ directory.) This commit introduces an initial filelist (generated by comparing the list of files in both directories) and a script to fetch the files from a local clone of the kernel git directory into the kernelsrc directory. When updating the directory, a git tag needs to be specified, which is recorded in a local file, in a way that it is possible to track down to which kernel release the kernelsrc files originally belong. A README file was also added to explain the idea behind it and to document the procedure to update the kernelsrc directory. Signed-off-by: Filipe Brandenburger --- kernelsrc/README.kernelsrc | 108 kernelsrc/update_kernelsrc.filelist | 23 ++ kernelsrc/update_kernelsrc.py | 158 3 files changed, 289 insertions(+) create mode 100644 kernelsrc/README.kernelsrc create mode 100644 kernelsrc/update_kernelsrc.filelist create mode 100755 kernelsrc/update_kernelsrc.py diff --git a/kernelsrc/README.kernelsrc b/kernelsrc/README.kernelsrc new file mode 100644 index 000..0aed4bb --- /dev/null +++ b/kernelsrc/README.kernelsrc @@ -0,0 +1,108 @@ + +The kernelsrc directory tree + + +The `kernelsrc' directory in the btrfs-progs tree contains a copy of some of the +files in the Linux kernel tree, mainly the files in fs/btrfs/, that are shared +between the implementation of the filesystem kernel module and some of the +command-line utilities. + +These files should never be edited on the btrfs-progs tree. Changes to these +files should be committed and pushed to the Linux kernel git tree (optionally +using #define's and #ifdef's to make code compatible with both kernel and user +spaces) and then it should be updated here from the kernel tree. + +In order to update the `kernelsrc' directory, you need to have a checkout of +both the btrfs-progs git tree and the Linux kernel git tree. + +Additionally to the kernel source files, the `kernelsrc' directory tree also +contains the scripts and control files used to update this same tree. + + +Updating the kernelsrc tree with update_kernelsrc.py + + +In order to update the `kernelsrc' tree to include the source code files from a +newer version of the Linux kernel tree, these are the steps that should be +followed: + +1. Make sure that you are in the root of the btrfs-progs git tree and that + the tree is clean (i.e., run `git status' and check its output.) + +2. Make sure you have a clone of the Linux kernel git tree. Let's assume + for this example that your kernel git tree is in directory ../linux/ + +3. Make sure you have know to which tag of the Linux kernel you want to + upgrade the `kernelsrc' tree to (e.g. v3.7 or v3.8-rc7) + +4. (Optional:) You may change the list of files that will be copied from + the Linux kernel tree. In order to do that, edit the file + `kernelsrc/update_kernelsrc.filelist' and either add or remove entries + from that file. + +5. Run the following command from the root of the btrfs-progs git tree: + + $ kernelsrc/update_kernelsrc.py ../linux/ v3.7 + + (assuming Linux kernel git tree at ../linux/ and upgrading to tag v3.7) + +6. Check whether the update matches your expectations. In particular, the + commands `git status' and `git diff --cached' may be helpful. + +7. Build btrfs-progs (using `make') and test them to make sure they are + still compatible with the new kernel source tree. Any needed adjustments + may be done by editing files *OUTSIDE* of the `kernelsrc' tree. Once + everything seems to be in place and working correctly, you may add those + files to the same git commit using the `git add' command. + +8. Commit the change, using the `git commit' command. You may use the + suggested commit command from the script output, with a standard commit + message. The editor will open anyways, so you can add more details + (particularly about files outside the `kernelsrc' tree that needed + changes.) + + +Migration to kernelsrc +== + +Using the `kernelsrc' tree allows for a gradual migration from the current +situation, where copies of the original source files from the kernel tree were +made in btrfs-progs, adjustments were made to allow them to compile in user +space, and they diverged further over time. + +First, it allows checking out the files from the kernel tree and adding them to +the btrfs-progs without actually using them initially. Then, it allows gradually +migrating the files one by one. One way to do it would be replacing the file in +the btrfs-progs root directory with a symbolic link to the same file in the +`kernelsrc' tree. Another way would be integrating t
[PATCH 1/2] btrfs-progs: anchor gitignore patterns on the root of the tree
This is needed in order to allow checking out fs/btrfs/, otherwise the ignore rule for the btrfs binary would conflict with it. Signed-off-by: Filipe Brandenburger --- .gitignore | 32 +++- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/.gitignore b/.gitignore index 0e560d5..3d360b6 100644 --- a/.gitignore +++ b/.gitignore @@ -1,15 +1,21 @@ *.o .*.o.d -version.h -man/*.gz -btrfs -btrfs-debug-tree -btrfs-map-logical -btrfs-show -btrfs-vol -btrfsck -btrfsctl -find-root -mkfs.btrfs -repair -restore +/version.h +/man/*.gz +/btrfs +/btrfsck +/btrfs-convert +/btrfsctl +/btrfs-debug-tree +/btrfs-find-root +/btrfs-image +/btrfs-map-logical +/btrfs-restore +/btrfs-show +/btrfstune +/btrfs-vol +/btrfs-zero-log +/find-root +/mkfs.btrfs +/repair +/restore -- 1.8.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC][PATCH 0/2] btrfs-progs: introduce kernelsrc directory to update it
Hello, This is a proposal on a way to keep btrfs-progs in sync with kernel code. https://btrfs.wiki.kernel.org/index.php/Cleanup_ideas#Use_the_kernel_code_in_user_mode For now, the patch introduces a method to import and update the kernel source files into the btrfs-progs tree. Further work will be needed to update the kernel source code so to make it generic (i.e. add #define's and #ifdef's) so that it can be compiled in both kernel and user space. The Python script is just a draft, it's already functional but error handling (and reporting) could definitely be improved, the code could be cleaned up, it could cache the kernel source directory and allow a --force parameter to allow moving to an tag that's not a descendant, but it's already fully functional and it's probably a good start... To see it in action, try this after applying the commits: $ kernelsrc/update_kernelsrc.py /path/to/linux v3.6 $ git status $ git commit -m "btrfs-progs: initial checkout of kernelsrc v3.6" $ vim kernelsrc/update_kernelsrc.filelist (add/delete a line) $ kernelsrc/update_kernelsrc.py ../linux-btrfs/ v3.7 $ git status $ git commit -m "btrfs-progs: upgrade kernelsrc from v3.6 to v3.7" Let me know what you think and if you think it's worth to continue on this path. Cheers, Filipe Filipe Brandenburger (2): btrfs-progs: anchor gitignore patterns on root btrfs-progs: introduce kernelsrc directory and script to update it .gitignore | 32 +--- kernelsrc/README.kernelsrc | 108 kernelsrc/update_kernelsrc.filelist | 23 ++ kernelsrc/update_kernelsrc.py | 158 4 files changed, 308 insertions(+), 13 deletions(-) create mode 100644 kernelsrc/README.kernelsrc create mode 100644 kernelsrc/update_kernelsrc.filelist create mode 100755 kernelsrc/update_kernelsrc.py -- 1.8.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
