Re: [PATCH 13/16] btrfs-progs: check btrfs_scan_one_device in btrfs_scan_lblkid()

[Anand Jain]

reviewed-by: Anand Jain anand.j...@oracle.com


On 11/07/2013 07:15 AM, Eric Sandeen wrote:

Even if it's definitely btrfs at this point,
btrfs_scan_one_device could fail for other reasons.

Check the return value, warn if it fails, and skip
the device register.

Resolves-Coverity-CID: 1125925
Signed-off-by: Eric Sandeen sand...@redhat.com
---
  utils.c |9 -
  1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/utils.c b/utils.c
index 8471148..ecacc29 100644
--- a/utils.c
+++ b/utils.c
@@ -1937,6 +1937,7 @@ int test_skip_this_disk(char *path)
  int btrfs_scan_lblkid(int update_kernel)
  {
int fd = -1;
+   int ret;
u64 num_devices;
struct btrfs_fs_devices *tmp_devices;
blkid_dev_iterate iter = NULL;
@@ -1965,8 +1966,14 @@ int btrfs_scan_lblkid(int update_kernel)
printf(ERROR: could not open %s\n, path);
continue;
}
-   btrfs_scan_one_device(fd, path, tmp_devices,
+   ret = btrfs_scan_one_device(fd, path, tmp_devices,
num_devices, BTRFS_SUPER_INFO_OFFSET);
+   if (ret) {
+   printf(ERROR: could not scan %s\n, path);
+   close (fd);
+   continue;
+   }
+
close(fd);
if (update_kernel)
btrfs_register_one_device(path);


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: unable to delete files after kernel upgrade from 3.8.10 to 3.12

[Stefan Behrens]

On Wed, 6 Nov 2013 21:53:25 +, Bartosz Kulicki wrote:
 As per subject. Seems UUID tree creation failed after upgrade. I could
 not mount filesystem under 3.12. Going back to 3.8.10 allowed me to
 mount fs but I could no longer perform any deletes, writes etc.
 
 I've opened a bug report here.
 https://bugzilla.kernel.org/show_bug.cgi?id=64461
 
 I've included link to image captured with btrfs-image.
 
 Please CC me if further information is needed as I'm not subscribed to
 the mailing list.
 

ENOSPC means you're out of disk space. A copy-on-write filesystem needs
disk space for delete operations, that's not a bug.

What does 'btrfs fi df /mountpoint' say?
How much disk space do you have, how much is allocated?

And try the procedure that is described in the wiki:

https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#I_get_.22No_space_left_on_device.22_errors.2C_but_df_says_I.27ve_got_lots_of_space


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 02/17] btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue

[Stefan Behrens]

On Thu, 7 Nov 2013 13:51:52 +0800, Qu Wenruo wrote:
 Use kernel workqueue to implement a new btrfs_workqueue_struct, which
 has the ordering execution feature like the btrfs_worker.
 
 The func is executed in a concurrency way, and the
 ordred_func/ordered_free is executed in the sequence them are queued
 after the corresponding func is done.
 The new btrfs_workqueue use 2 workqueues to implement the original
 btrfs_worker, one for the normal work and one for ordered work.
 
 At this patch, high priority work queue or thresholding is not added yet.
 The high priority feature and thresholding will be added in the following 
 patches.
 
 Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
[...]
 diff --git a/fs/btrfs/async-thread.h b/fs/btrfs/async-thread.h
 index 1f26792..eee6709 100644
 --- a/fs/btrfs/async-thread.h
 +++ b/fs/btrfs/async-thread.h
 @@ -1,5 +1,6 @@
  /*
   * Copyright (C) 2007 Oracle.  All rights reserved.
 + * Copyright (C) 2013 Fujitsu.  All rights reserved.
   *
   * This program is free software; you can redistribute it and/or
   * modify it under the terms of the GNU General Public
 @@ -118,4 +119,47 @@ void btrfs_init_workers(struct btrfs_workers *workers, 
 char *name, int max,
   struct btrfs_workers *async_starter);
  void btrfs_requeue_work(struct btrfs_work *work);
  void btrfs_set_work_high_prio(struct btrfs_work *work);
 +
 +struct btrfs_workqueue_struct {
 + struct workqueue_struct *normal_wq;
 + struct workqueue_struct *ordered_wq;
 +
 + /*
 +  * Spinlock to ensure that both ordered and normal work can
 +  * be inserted to each workqueue at the same sequance,
 +  * which will reduce the ordered_work waiting time and disk head moves.
 +  */
 + spinlock_t insert_lock;
 +};
 +
 +struct btrfs_work_struct {
 + void (*func)(struct btrfs_work_struct *arg);
 + void (*ordered_func)(struct btrfs_work_struct *arg);
 + void (*ordered_free)(struct btrfs_work_struct *arg);
 +
 + /* Don't touch things below */
 + struct work_struct normal_work;
 + struct work_struct ordered_work;
 + struct completion normal_completion;
 +};

If you compare the Btrfs sources before applying your patchset and after
applying all 17 patches, one change is this:
-struct btrfs_work {
+struct btrfs_work_struct {

Which causes changes s/struct btrfs_work/struct btrfs_work_struct/ like
in patch 16/17:
-   struct btrfs_work   work;
+   struct btrfs_work_struct
+   work;

-static void scrub_bio_end_io_worker(struct btrfs_work *work);
+static void scrub_bio_end_io_worker(struct btrfs_work_struct *work);

I just don't see any good reason for renaming 'struct foo' to 'struct
foo_struct'.


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] btrfs-progs: lblkid wouldn't find non mapper path input v3

[Anand Jain]

A new test case when disk is unmounted and if the non mapper
disk path is given as the argument to the btrfs filesystem show arg
we still need this to work but lblkid will pull only mapper disks,
it won't match. So this will normalize the input to find btrfs
by fsid and pass it to the search.

v2: accepts Josef suggested
v2: accepts Josef suggested

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 cmds-filesystem.c |   59 +---
 1 files changed, 55 insertions(+), 4 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index d237989..72ffe46 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -37,6 +37,7 @@
 #include version.h
 #include commands.h
 #include list_sort.h
+#include disk-io.h
 
 static const char * const filesystem_cmd_group_usage[] = {
btrfs filesystem [group] command [args],
@@ -414,6 +415,39 @@ static int btrfs_scan_kernel(void *search)
return 0;
 }
 
+static int dev_to_fsid(char *dev, __u8 *fsid)
+{
+   struct btrfs_super_block *disk_super;
+   char *buf;
+   int ret;
+   int fd;
+
+   buf = malloc(4096);
+   if (!buf)
+   return -ENOMEM;
+
+   fd = open(dev, O_RDONLY);
+   if (fd  0) {
+   ret = -errno;
+   free(buf);
+   return ret;
+   }
+
+   disk_super = (struct btrfs_super_block *)buf;
+   ret = btrfs_read_dev_super(fd, disk_super,
+   BTRFS_SUPER_INFO_OFFSET);
+   if (ret)
+   goto out;
+
+   memcpy(fsid, disk_super-fsid, BTRFS_FSID_SIZE);
+   ret = 0;
+
+out:
+   close(fd);
+   free(buf);
+   return ret;
+}
+
 static const char * const cmd_show_usage[] = {
btrfs filesystem show [options] [path|uuid|device|label],
Show the structure of a filesystem,
@@ -434,6 +468,8 @@ static int cmd_show(int argc, char **argv)
int type = 0;
char mp[BTRFS_PATH_NAME_MAX + 1];
char path[PATH_MAX];
+   __u8 fsid[BTRFS_FSID_SIZE];
+   char uuid_buf[37];
 
while (1) {
int long_index;
@@ -466,6 +502,10 @@ static int cmd_show(int argc, char **argv)
if (strlen(search) == 0)
usage(cmd_show_usage);
type = check_arg_type(search);
+   /* needs spl handling if input arg is block dev
+* And if input arg is mount-point just print it
+* right away
+*/
if (type == BTRFS_ARG_BLKDEV) {
if (where == BTRFS_SCAN_DEV) {
/* we need to do this because
@@ -477,14 +517,25 @@ static int cmd_show(int argc, char **argv)
} else {
ret = get_btrfs_mount(search,
mp, sizeof(mp));
-   if (!ret)
+   if (!ret) {
/* given block dev is mounted*/
search = mp;
-   else
+   type = BTRFS_ARG_MNTPOINT;
+   } else {
+   ret = dev_to_fsid(search, fsid);
+   if (ret) {
+   fprintf(stderr,
+   ERROR: No btrfs on 
%s\n,
+   search);
+   return 1;
+   }
+   uuid_unparse(fsid, uuid_buf);
+   search = uuid_buf;
+   type = BTRFS_ARG_UUID;
goto devs_only;
+   }
}
-   }
-   if (type == BTRFS_ARG_MNTPOINT) {
+   } else if (type == BTRFS_ARG_MNTPOINT) {
char label[BTRFS_LABEL_SIZE];
if (get_label_mounted(search, label))
return 1;
-- 
1.7.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/4 v2] btrfs-progs: mechanism to fetch fsinfo from btrfs-control

[Anand Jain]

Thanks sent out v3.

On 11/07/13 04:13 AM, Josef Bacik wrote:

On Mon, Nov 04, 2013 at 11:45:43AM +0800, Anand Jain wrote:

need fsinfo from btrfs-control that is when mount path is
not known.
current method of going through each mount points isn't
efficient, and multiple subvol of a fsid could be mounted
means extra logic to handle that. Further this will help
to revamp check_mounted() (planned)

check_mounted is heavily used in the btrfs-progs, it
does full scan of all the disks in the system to confirm
if a multi-disk btrfs is mounted it doesn't scalable well
with few hundreds luns, check_mounted for sure needs a
revamp. using this it can be done easily. which is planned.

v2: commit reword

Signed-off-by: Anand Jainanand.j...@oracle.com
---
  ioctl.h |   19 +++
  utils.c |   80 +++
  utils.h |1 +
  3 files changed, 100 insertions(+), 0 deletions(-)

diff --git a/ioctl.h b/ioctl.h
index d21413f..29575d8 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -506,6 +506,23 @@ static inline char *btrfs_err_str(enum btrfs_err_code 
err_code)
}
  }

+/* fs flags */
+#define BTRFS_FS_MOUNTED   (1LLU  0)
+
+struct btrfs_ioctl_fslist {
+   __u64 self_sz;  /* in/out */
+   __u8 fsid[BTRFS_FSID_SIZE]; /* out */
+   __u64 num_devices;
+   __u64 missing_devices;
+   __u64 total_devices;
+   __u64 flags;
+};
+
+struct btrfs_ioctl_fslist_args {
+   __u64 self_sz;  /* in/out */
+   __u64 count;/* out */
+};
+
  #define BTRFS_IOC_SNAP_CREATE _IOW(BTRFS_IOCTL_MAGIC, 1, \
   struct btrfs_ioctl_vol_args)
  #define BTRFS_IOC_DEFRAG _IOW(BTRFS_IOCTL_MAGIC, 2, \
@@ -604,6 +621,8 @@ struct btrfs_ioctl_clone_range_args {
struct btrfs_ioctl_dev_replace_args)
  #define BTRFS_IOC_DEDUP_CTL _IOWR(BTRFS_IOCTL_MAGIC, 55, \
  struct btrfs_ioctl_dedup_args)
+#define BTRFS_IOC_GET_FSLIST _IOWR(BTRFS_IOCTL_MAGIC, 56, \
+   struct btrfs_ioctl_fslist_args)
  #ifdef __cplusplus
  }
  #endif
diff --git a/utils.c b/utils.c
index 5bedd97..1798a7c 100644
--- a/utils.c
+++ b/utils.c
@@ -2087,3 +2087,83 @@ int lookup_ino_rootid(int fd, u64 *rootid)

return 0;
  }
+
+/* scans for fsid(s) in the kernel using the btrfs-control
+ * interface.
+ */
+int get_fslist(struct btrfs_ioctl_fslist **out_fslist, int *out_count)
+{
+   int ret, fd, e;
+   struct btrfs_ioctl_fslist_args *fsargs;
+   struct btrfs_ioctl_fslist *fslist;
+   struct btrfs_ioctl_fslist *fslist_tmp;
+   u64 sz;
+   int count;
+
+   fd = open(/dev/btrfs-control, O_RDWR);
+   e = errno;
+   if (fd  0) {
+   perror(failed to open /dev/btrfs-control);
+   return -e;
+   }
+
+   /* space to hold 512 fsids, doesn't matter if small
+* it would fail and return count so then we try again
+*/
+   count = 512;
+again:
+   sz = sizeof(*fsargs) + sizeof(*fslist) * count;
+
+   fsargs = (struct btrfs_ioctl_fslist_args *) malloc(sz);


No need to cast the return value of malloc.


+   memset(fsargs, 0, sz);
+
+   if (!fsargs) {
+   close(fd);
+   return -ENOMEM;
+   }


Should check !fsargs before memsetting it.


+   fsargs-count = count;
+
+   ret = ioctl(fd, BTRFS_IOC_GET_FSLIST, fsargs);
+   e = errno;
+   if (ret == 1) {
+   /* out of size so reallocate */
+   count = fsargs-count;
+   free(fsargs);
+   goto again;
+   } else if (ret  0) {
+   printf(ERROR: scan_fsid ioctl failed - %s\n,
+   strerror(e));
+   ret = -e;
+   goto out;
+   }
+
+   /* ioctl returns fsid count in count parameter*/
+
+   *out_count = count = fsargs-count;
+   if (count == 0) {
+   *out_fslist = NULL;
+   ret = 0;
+   goto out;
+   }
+
+   fslist = (struct btrfs_ioctl_fslist *) (fsargs +
+   sizeof(*fsargs));
+
+   fslist_tmp = *out_fslist = (struct btrfs_ioctl_fslist *)
+   malloc(sizeof(*fslist) * count);


No need to cast.  Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] btrfs-progs: fs show should handle if subvol(s) mounted v3

[Anand Jain]

as of now with out this patch user would see
fsinfo per btrfs mount path but which mean multiple
entry if more than one subvol is mounted of the same
fsid. so this patch will handle that nicely.

v3: accepts Josef suggested and fix git screwup
v2: accepts Zach suggested

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 cmds-filesystem.c |   90 +
 utils.c   |   60 +++
 utils.h   |1 +
 3 files changed, 117 insertions(+), 34 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index d2cad81..d237989 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -317,6 +317,29 @@ static int print_one_fs(struct btrfs_ioctl_fs_info_args 
*fs_info,
return 0;
 }
 
+static void handle_print(char *mnt, char *label)
+{
+   int fd;
+   struct btrfs_ioctl_fs_info_args fs_info_arg;
+   struct btrfs_ioctl_dev_info_args *dev_info_arg = NULL;
+   struct btrfs_ioctl_space_args *space_info_arg;
+
+   if (get_fs_info(mnt, fs_info_arg, dev_info_arg)) {
+   fprintf(stdout, ERROR: get_fs_info failed\n);
+   return;
+   }
+
+   fd = open(mnt, O_RDONLY);
+   if (fd != -1  !get_df(fd, space_info_arg)) {
+   print_one_fs(fs_info_arg, dev_info_arg,
+   space_info_arg, label, mnt);
+   kfree(space_info_arg);
+   }
+   if (fd != -1)
+   close(fd);
+   kfree(dev_info_arg);
+}
+
 /* This function checks if the given input parameter is
  * an uuid or a path
  * return -1: some error in the given input
@@ -350,47 +373,39 @@ static int check_arg_type(char *input)
 
 static int btrfs_scan_kernel(void *search)
 {
-   int ret = 0, fd;
-   FILE *f;
-   struct mntent *mnt;
-   struct btrfs_ioctl_fs_info_args fs_info_arg;
-   struct btrfs_ioctl_dev_info_args *dev_info_arg = NULL;
-   struct btrfs_ioctl_space_args *space_info_arg;
+   int ret = 0;
char label[BTRFS_LABEL_SIZE];
-
-   f = setmntent(/proc/self/mounts, r);
-   if (f == NULL)
-   return 1;
-
-   memset(label, 0, sizeof(label));
-   while ((mnt = getmntent(f)) != NULL) {
-   if (strcmp(mnt-mnt_type, btrfs))
+   char mnt[BTRFS_PATH_NAME_MAX + 1];
+   struct btrfs_ioctl_fslist *fslist;
+   struct btrfs_ioctl_fslist *fslist_saved;
+   u64 cnt_fs;
+   int cnt_mnt;
+   __u8 *fsid;
+   __u64 flags;
+
+   ret = get_fslist(fslist, cnt_fs);
+   if (ret)
+   return ret;
+   fslist_saved = fslist;
+   while (cnt_fs--) {
+   fsid = fslist-fsid;
+   flags = fslist-flags;
+   fslist++;
+   if (!(flags  BTRFS_FS_MOUNTED))
continue;
-   ret = get_fs_info(mnt-mnt_dir, fs_info_arg,
-   dev_info_arg);
+   memset(mnt, 0, BTRFS_PATH_NAME_MAX + 1);
+   memset(label, 0, sizeof(label));
+   ret = fsid_to_mntpt(fsid, mnt, cnt_mnt);
if (ret)
return ret;
-
-   if (get_label_mounted(mnt-mnt_dir, label)) {
-   kfree(dev_info_arg);
+   if (get_label_mounted(mnt, label))
return 1;
-   }
-   if (search  !match_search_item_kernel(fs_info_arg.fsid,
-   mnt-mnt_dir, label, search)) {
-   kfree(dev_info_arg);
+
+   if (search  !match_search_item_kernel(fsid,
+   mnt, label, search))
continue;
-   }
 
-   fd = open(mnt-mnt_dir, O_RDONLY);
-   if (fd  0  !get_df(fd, space_info_arg)) {
-   print_one_fs(fs_info_arg, dev_info_arg,
-   space_info_arg, label, mnt-mnt_dir);
-   kfree(space_info_arg);
-   memset(label, 0, sizeof(label));
-   }
-   if (fd  0)
-   close(fd);
-   kfree(dev_info_arg);
+   handle_print(mnt, label);
if (search)
return 0;
}
@@ -469,6 +484,13 @@ static int cmd_show(int argc, char **argv)
goto devs_only;
}
}
+   if (type == BTRFS_ARG_MNTPOINT) {
+   char label[BTRFS_LABEL_SIZE];
+   if (get_label_mounted(search, label))
+   return 1;
+   handle_print(search, label);
+   return 0;
+   }
}
 
if (where == BTRFS_SCAN_DEV)
diff --git a/utils.c b/utils.c
index 4b90026..6126af4 100644
--- a/utils.c
+++ b/utils.c
@@ -2163,3 +2163,63 @@ out:
close(fd);

[PATCH 1/3] btrfs-progs: mechanism to fetch fsinfo from btrfs-control v3

[Anand Jain]

need fsinfo from btrfs-control that is when mount path is
not known.
current method of going through each mount points isn't
efficient, and multiple subvol of a fsid could be mounted
means extra logic to handle that. Further this will help
to revamp check_mounted() (planned)

check_mounted is heavily used in the btrfs-progs, it
does full scan of all the disks in the system to confirm
if a multi-disk btrfs is mounted it doesn't scalable well
with few hundreds luns, check_mounted for sure needs a
revamp. using this it can be done easily. which is planned.

v3: accepts Josef suggested and fix git screwup
v2: commit reword

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 ioctl.h |   19 +++
 utils.c |   76 +++
 utils.h |1 +
 3 files changed, 96 insertions(+), 0 deletions(-)

diff --git a/ioctl.h b/ioctl.h
index d21413f..29575d8 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -506,6 +506,23 @@ static inline char *btrfs_err_str(enum btrfs_err_code 
err_code)
}
 }
 
+/* fs flags */
+#define BTRFS_FS_MOUNTED   (1LLU  0)
+
+struct btrfs_ioctl_fslist {
+   __u64 self_sz;  /* in/out */
+   __u8 fsid[BTRFS_FSID_SIZE]; /* out */
+   __u64 num_devices;
+   __u64 missing_devices;
+   __u64 total_devices;
+   __u64 flags;
+};
+
+struct btrfs_ioctl_fslist_args {
+   __u64 self_sz;  /* in/out */
+   __u64 count;/* out */
+};
+
 #define BTRFS_IOC_SNAP_CREATE _IOW(BTRFS_IOCTL_MAGIC, 1, \
   struct btrfs_ioctl_vol_args)
 #define BTRFS_IOC_DEFRAG _IOW(BTRFS_IOCTL_MAGIC, 2, \
@@ -604,6 +621,8 @@ struct btrfs_ioctl_clone_range_args {
struct btrfs_ioctl_dev_replace_args)
 #define BTRFS_IOC_DEDUP_CTL _IOWR(BTRFS_IOCTL_MAGIC, 55, \
  struct btrfs_ioctl_dedup_args)
+#define BTRFS_IOC_GET_FSLIST _IOWR(BTRFS_IOCTL_MAGIC, 56, \
+   struct btrfs_ioctl_fslist_args)
 #ifdef __cplusplus
 }
 #endif
diff --git a/utils.c b/utils.c
index 60d2c3a..4b90026 100644
--- a/utils.c
+++ b/utils.c
@@ -47,6 +47,7 @@
 #include utils.h
 #include volumes.h
 #include ioctl.h
+#include btrfs-list.h
 
 #ifndef BLKDISCARD
 #define BLKDISCARD _IO(0x12,119)
@@ -2087,3 +2088,78 @@ int lookup_ino_rootid(int fd, u64 *rootid)
 
return 0;
 }
+
+/* scans for fsid(s) in the kernel using the btrfs-control
+ * interface.
+ */
+int get_fslist(struct btrfs_ioctl_fslist **out_fslist, u64 *out_count)
+{
+   int ret, fd, e;
+   struct btrfs_ioctl_fslist_args *fsargs;
+   struct btrfs_ioctl_fslist_args *fsargs_saved = NULL;
+   struct btrfs_ioctl_fslist *fslist;
+   u64 sz;
+   int count;
+
+   fd = open(/dev/btrfs-control, O_RDWR);
+   e = errno;
+   if (fd  0) {
+   perror(failed to open /dev/btrfs-control);
+   return -e;
+   }
+
+   /* space to hold 512 fsids, doesn't matter if small
+* it would fail and return count so then we try again
+*/
+   count = 512;
+again:
+   sz = sizeof(*fsargs) + sizeof(*fslist) * count;
+
+   fsargs = malloc(sz);
+   if (!fsargs) {
+   close(fd);
+   return -ENOMEM;
+   }
+
+   memset(fsargs, 0, sz);
+   fsargs-count = count;
+
+   ret = ioctl(fd, BTRFS_IOC_GET_FSLIST, fsargs);
+   e = errno;
+   if (ret == 1) {
+   /* out of size so reallocate */
+   count = fsargs-count;
+   free(fsargs);
+   goto again;
+   } else if (ret  0) {
+   printf(ERROR: scan_fsid ioctl failed - %s\n,
+   strerror(e));
+   ret = -e;
+   goto out;
+   }
+
+   /* ioctl returns fsid count in count parameter*/
+
+   *out_count = fsargs-count;
+   if (*out_count == 0) {
+   *out_fslist = NULL;
+   ret = 0;
+   goto out;
+   }
+
+   fsargs_saved = fsargs;
+   fslist = (struct btrfs_ioctl_fslist *) (++fsargs);
+
+   sz = sizeof(*fslist) * *out_count;
+   *out_fslist = malloc(sz);
+   if (*out_fslist == NULL) {
+   ret = -ENOMEM;
+   goto out;
+   }
+   memcpy(*out_fslist, fslist, sz);
+   ret = 0;
+out:
+   free(fsargs_saved);
+   close(fd);
+   return 0;
+}
diff --git a/utils.h b/utils.h
index 6f4b10c..a14a887 100644
--- a/utils.h
+++ b/utils.h
@@ -94,5 +94,6 @@ int ask_user(char *question);
 int lookup_ino_rootid(int fd, u64 *rootid);
 int btrfs_scan_lblkid(int update_kernel);
 int get_btrfs_mount(const char *dev, char *mp, size_t mp_size);
+int get_fslist(struct btrfs_ioctl_fslist **out_fslist, u64 *out_count);
 
 #endif
-- 
1.7.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  

Re: [PATCH 4/4] btrfs-progs: lblkid wouldn't find non mapper path input

[Anand Jain]

Done.


On 11/07/13 04:20 AM, Josef Bacik wrote:

On Mon, Nov 04, 2013 at 11:45:45AM +0800, Anand Jain wrote:

A new test case when disk is unmounted and if the non mapper
disk path is given as the argument to the btrfs filesystem showarg
we still need this to work but lblkid will pull only mapper disks,
it won't match. So this will normalize the input to find btrfs
by fsid and pass it to the search.

v2: accepts Josef suggested

Signed-off-by: Anand Jainanand.j...@oracle.com
---
  cmds-filesystem.c |   59 +---
  1 files changed, 55 insertions(+), 4 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index f8e8475..f40178a 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -37,6 +37,7 @@
  #include version.h
  #include commands.h
  #include list_sort.h
+#include disk-io.h

  static const char * const filesystem_cmd_group_usage[] = {
btrfs filesystem [group]command  [args],
@@ -414,6 +415,39 @@ static int btrfs_scan_kernel(void *search)
return 0;
  }

+static int dev_to_fsid(char *dev, __u8 *fsid)
+{
+   struct btrfs_super_block *disk_super;
+   char *buf;
+   int ret;
+   int fd;
+
+   buf = malloc(4096);
+   if (!buf)
+   return -ENOMEM;
+
+   fd = open(dev, O_RDONLY);
+   if (fd  0) {
+   ret = -errno;
+   free(buf);
+   return ret;
+   }
+
+   disk_super = (struct btrfs_super_block *)buf;
+   ret = btrfs_read_dev_super(fd, disk_super,
+   BTRFS_SUPER_INFO_OFFSET);
+   if (ret)
+   goto out;
+
+   memcpy(fsid, disk_super-fsid, BTRFS_FSID_SIZE);
+   ret = 0;
+
+out:
+   close(fd);
+   free(buf);
+   return ret;
+}
+
  static const char * const cmd_show_usage[] = {
btrfs filesystem show [options] [path|uuid|device|label],
Show the structure of a filesystem,
@@ -434,6 +468,8 @@ static int cmd_show(int argc, char **argv)
int type = 0;
char mp[BTRFS_PATH_NAME_MAX + 1];
char path[PATH_MAX];
+   __u8 fsid[BTRFS_FSID_SIZE];
+   char uuid_buf[37];

while (1) {
int long_index;
@@ -466,6 +502,10 @@ static int cmd_show(int argc, char **argv)
if (strlen(search) == 0)
usage(cmd_show_usage);
type = check_arg_type(search);
+   /*needs spl handling if input arg is block dev
+*And if input arg is mount-point just print it
+*right away
+   */


Format screwup.  Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/4 v2] btrfs-progs: fs show should handle if subvol(s) mounted

[Anand Jain]

Thanks sent out v3


On 11/07/13 04:19 AM, Josef Bacik wrote:

On Mon, Nov 04, 2013 at 11:45:44AM +0800, Anand Jain wrote:

as of now with out this patch user would see
fsinfo per btrfs mount path but which mean multiple
entry if more than one subvol is mounted of the same
fsid. so this patch will handle that nicely.

v2: accepts Zach suggested

Signed-off-by: Anand Jainanand.j...@oracle.com
---
  cmds-filesystem.c |   90 +
  utils.c   |   88 ++-
  utils.h   |3 +-
  3 files changed, 130 insertions(+), 51 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index d2cad81..f8e8475 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -317,6 +317,29 @@ static int print_one_fs(struct btrfs_ioctl_fs_info_args 
*fs_info,
return 0;
  }

+static void handle_print(char *mnt, char *label)
+{
+   int fd;
+   struct btrfs_ioctl_fs_info_args fs_info_arg;
+   struct btrfs_ioctl_dev_info_args *dev_info_arg = NULL;
+   struct btrfs_ioctl_space_args *space_info_arg;
+
+   if (get_fs_info(mnt,fs_info_arg,dev_info_arg)) {
+   fprintf(stdout, ERROR: get_fs_info failed\n);
+   return;
+   }
+
+   fd = open(mnt, O_RDONLY);
+   if (fd  0  !get_df(fd,space_info_arg)) {


0 is a valid fd.  Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] btrfs-progs: lblkid wouldn't find non mapper path input v3

[Anand Jain]

A new test case when disk is unmounted and if the non mapper
disk path is given as the argument to the btrfs filesystem show arg
we still need this to work but lblkid will pull only mapper disks,
it won't match. So this will normalize the input to find btrfs
by fsid and pass it to the search.

v3: accepts Josef suggested
v2: accepts Josef suggested

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 cmds-filesystem.c |   59 +---
 1 files changed, 55 insertions(+), 4 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index d237989..72ffe46 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -37,6 +37,7 @@
 #include version.h
 #include commands.h
 #include list_sort.h
+#include disk-io.h
 
 static const char * const filesystem_cmd_group_usage[] = {
btrfs filesystem [group] command [args],
@@ -414,6 +415,39 @@ static int btrfs_scan_kernel(void *search)
return 0;
 }
 
+static int dev_to_fsid(char *dev, __u8 *fsid)
+{
+   struct btrfs_super_block *disk_super;
+   char *buf;
+   int ret;
+   int fd;
+
+   buf = malloc(4096);
+   if (!buf)
+   return -ENOMEM;
+
+   fd = open(dev, O_RDONLY);
+   if (fd  0) {
+   ret = -errno;
+   free(buf);
+   return ret;
+   }
+
+   disk_super = (struct btrfs_super_block *)buf;
+   ret = btrfs_read_dev_super(fd, disk_super,
+   BTRFS_SUPER_INFO_OFFSET);
+   if (ret)
+   goto out;
+
+   memcpy(fsid, disk_super-fsid, BTRFS_FSID_SIZE);
+   ret = 0;
+
+out:
+   close(fd);
+   free(buf);
+   return ret;
+}
+
 static const char * const cmd_show_usage[] = {
btrfs filesystem show [options] [path|uuid|device|label],
Show the structure of a filesystem,
@@ -434,6 +468,8 @@ static int cmd_show(int argc, char **argv)
int type = 0;
char mp[BTRFS_PATH_NAME_MAX + 1];
char path[PATH_MAX];
+   __u8 fsid[BTRFS_FSID_SIZE];
+   char uuid_buf[37];
 
while (1) {
int long_index;
@@ -466,6 +502,10 @@ static int cmd_show(int argc, char **argv)
if (strlen(search) == 0)
usage(cmd_show_usage);
type = check_arg_type(search);
+   /* needs spl handling if input arg is block dev
+* And if input arg is mount-point just print it
+* right away
+*/
if (type == BTRFS_ARG_BLKDEV) {
if (where == BTRFS_SCAN_DEV) {
/* we need to do this because
@@ -477,14 +517,25 @@ static int cmd_show(int argc, char **argv)
} else {
ret = get_btrfs_mount(search,
mp, sizeof(mp));
-   if (!ret)
+   if (!ret) {
/* given block dev is mounted*/
search = mp;
-   else
+   type = BTRFS_ARG_MNTPOINT;
+   } else {
+   ret = dev_to_fsid(search, fsid);
+   if (ret) {
+   fprintf(stderr,
+   ERROR: No btrfs on 
%s\n,
+   search);
+   return 1;
+   }
+   uuid_unparse(fsid, uuid_buf);
+   search = uuid_buf;
+   type = BTRFS_ARG_UUID;
goto devs_only;
+   }
}
-   }
-   if (type == BTRFS_ARG_MNTPOINT) {
+   } else if (type == BTRFS_ARG_MNTPOINT) {
char label[BTRFS_LABEL_SIZE];
if (get_label_mounted(search, label))
return 1;
-- 
1.7.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs-progs: mechanism to fetch fsinfo from btrfs-control v3

[Anand Jain]

need fsinfo from btrfs-control that is when mount path is
not known.
current method of going through each mount points isn't
efficient, and multiple subvol of a fsid could be mounted
means extra logic to handle that. Further this will help
to revamp check_mounted() (planned)

check_mounted is heavily used in the btrfs-progs, it
does full scan of all the disks in the system to confirm
if a multi-disk btrfs is mounted it doesn't scalable well
with few hundreds luns, check_mounted for sure needs a
revamp. using this it can be done easily. which is planned.

v3: accepts Josef suggested and fix git screwup
v2: commit reword

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 ioctl.h |   19 +++
 utils.c |   76 +++
 utils.h |1 +
 3 files changed, 96 insertions(+), 0 deletions(-)

diff --git a/ioctl.h b/ioctl.h
index d21413f..29575d8 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -506,6 +506,23 @@ static inline char *btrfs_err_str(enum btrfs_err_code 
err_code)
}
 }
 
+/* fs flags */
+#define BTRFS_FS_MOUNTED   (1LLU  0)
+
+struct btrfs_ioctl_fslist {
+   __u64 self_sz;  /* in/out */
+   __u8 fsid[BTRFS_FSID_SIZE]; /* out */
+   __u64 num_devices;
+   __u64 missing_devices;
+   __u64 total_devices;
+   __u64 flags;
+};
+
+struct btrfs_ioctl_fslist_args {
+   __u64 self_sz;  /* in/out */
+   __u64 count;/* out */
+};
+
 #define BTRFS_IOC_SNAP_CREATE _IOW(BTRFS_IOCTL_MAGIC, 1, \
   struct btrfs_ioctl_vol_args)
 #define BTRFS_IOC_DEFRAG _IOW(BTRFS_IOCTL_MAGIC, 2, \
@@ -604,6 +621,8 @@ struct btrfs_ioctl_clone_range_args {
struct btrfs_ioctl_dev_replace_args)
 #define BTRFS_IOC_DEDUP_CTL _IOWR(BTRFS_IOCTL_MAGIC, 55, \
  struct btrfs_ioctl_dedup_args)
+#define BTRFS_IOC_GET_FSLIST _IOWR(BTRFS_IOCTL_MAGIC, 56, \
+   struct btrfs_ioctl_fslist_args)
 #ifdef __cplusplus
 }
 #endif
diff --git a/utils.c b/utils.c
index 60d2c3a..4b90026 100644
--- a/utils.c
+++ b/utils.c
@@ -47,6 +47,7 @@
 #include utils.h
 #include volumes.h
 #include ioctl.h
+#include btrfs-list.h
 
 #ifndef BLKDISCARD
 #define BLKDISCARD _IO(0x12,119)
@@ -2087,3 +2088,78 @@ int lookup_ino_rootid(int fd, u64 *rootid)
 
return 0;
 }
+
+/* scans for fsid(s) in the kernel using the btrfs-control
+ * interface.
+ */
+int get_fslist(struct btrfs_ioctl_fslist **out_fslist, u64 *out_count)
+{
+   int ret, fd, e;
+   struct btrfs_ioctl_fslist_args *fsargs;
+   struct btrfs_ioctl_fslist_args *fsargs_saved = NULL;
+   struct btrfs_ioctl_fslist *fslist;
+   u64 sz;
+   int count;
+
+   fd = open(/dev/btrfs-control, O_RDWR);
+   e = errno;
+   if (fd  0) {
+   perror(failed to open /dev/btrfs-control);
+   return -e;
+   }
+
+   /* space to hold 512 fsids, doesn't matter if small
+* it would fail and return count so then we try again
+*/
+   count = 512;
+again:
+   sz = sizeof(*fsargs) + sizeof(*fslist) * count;
+
+   fsargs = malloc(sz);
+   if (!fsargs) {
+   close(fd);
+   return -ENOMEM;
+   }
+
+   memset(fsargs, 0, sz);
+   fsargs-count = count;
+
+   ret = ioctl(fd, BTRFS_IOC_GET_FSLIST, fsargs);
+   e = errno;
+   if (ret == 1) {
+   /* out of size so reallocate */
+   count = fsargs-count;
+   free(fsargs);
+   goto again;
+   } else if (ret  0) {
+   printf(ERROR: scan_fsid ioctl failed - %s\n,
+   strerror(e));
+   ret = -e;
+   goto out;
+   }
+
+   /* ioctl returns fsid count in count parameter*/
+
+   *out_count = fsargs-count;
+   if (*out_count == 0) {
+   *out_fslist = NULL;
+   ret = 0;
+   goto out;
+   }
+
+   fsargs_saved = fsargs;
+   fslist = (struct btrfs_ioctl_fslist *) (++fsargs);
+
+   sz = sizeof(*fslist) * *out_count;
+   *out_fslist = malloc(sz);
+   if (*out_fslist == NULL) {
+   ret = -ENOMEM;
+   goto out;
+   }
+   memcpy(*out_fslist, fslist, sz);
+   ret = 0;
+out:
+   free(fsargs_saved);
+   close(fd);
+   return 0;
+}
diff --git a/utils.h b/utils.h
index 6f4b10c..a14a887 100644
--- a/utils.h
+++ b/utils.h
@@ -94,5 +94,6 @@ int ask_user(char *question);
 int lookup_ino_rootid(int fd, u64 *rootid);
 int btrfs_scan_lblkid(int update_kernel);
 int get_btrfs_mount(const char *dev, char *mp, size_t mp_size);
+int get_fslist(struct btrfs_ioctl_fslist **out_fslist, u64 *out_count);
 
 #endif
-- 
1.7.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  

Re: [PATCH 1/3] btrfs-progs: mechanism to fetch fsinfo from btrfs-control v3

[Stefan Behrens]

On Thu,  7 Nov 2013 18:01:41 +0800, Anand Jain wrote:
 + *out_fslist = malloc(sz);
 + if (*out_fslist == NULL) {
 + ret = -ENOMEM;
 + goto out;
 + }
 + memcpy(*out_fslist, fslist, sz);
 + ret = 0;
 +out:
 + free(fsargs_saved);
 + close(fd);
 + return 0;
 +}

return ret;


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs-progs: mechanism to fetch fsinfo from btrfs-control v4

[Anand Jain]

need fsinfo from btrfs-control that is when mount path is
not known.
current method of going through each mount points isn't
efficient, and multiple subvol of a fsid could be mounted
means extra logic to handle that. Further this will help
to revamp check_mounted() (planned)

check_mounted is heavily used in the btrfs-progs, it
does full scan of all the disks in the system to confirm
if a multi-disk btrfs is mounted it doesn't scalable well
with few hundreds luns, check_mounted for sure needs a
revamp. using this it can be done easily. which is planned.

Signed-off-by: Anand Jain anand.j...@oracle.com

v4: bug fix and accepts Stefan suggested
v3: accepts Josef suggested
v2: commit reword
---
 ioctl.h |   19 
 utils.c |   75 +++
 utils.h |1 +
 3 files changed, 95 insertions(+), 0 deletions(-)

diff --git a/ioctl.h b/ioctl.h
index d21413f..29575d8 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -506,6 +506,23 @@ static inline char *btrfs_err_str(enum btrfs_err_code 
err_code)
}
 }
 
+/* fs flags */
+#define BTRFS_FS_MOUNTED   (1LLU  0)
+
+struct btrfs_ioctl_fslist {
+   __u64 self_sz;  /* in/out */
+   __u8 fsid[BTRFS_FSID_SIZE]; /* out */
+   __u64 num_devices;
+   __u64 missing_devices;
+   __u64 total_devices;
+   __u64 flags;
+};
+
+struct btrfs_ioctl_fslist_args {
+   __u64 self_sz;  /* in/out */
+   __u64 count;/* out */
+};
+
 #define BTRFS_IOC_SNAP_CREATE _IOW(BTRFS_IOCTL_MAGIC, 1, \
   struct btrfs_ioctl_vol_args)
 #define BTRFS_IOC_DEFRAG _IOW(BTRFS_IOCTL_MAGIC, 2, \
@@ -604,6 +621,8 @@ struct btrfs_ioctl_clone_range_args {
struct btrfs_ioctl_dev_replace_args)
 #define BTRFS_IOC_DEDUP_CTL _IOWR(BTRFS_IOCTL_MAGIC, 55, \
  struct btrfs_ioctl_dedup_args)
+#define BTRFS_IOC_GET_FSLIST _IOWR(BTRFS_IOCTL_MAGIC, 56, \
+   struct btrfs_ioctl_fslist_args)
 #ifdef __cplusplus
 }
 #endif
diff --git a/utils.c b/utils.c
index 60d2c3a..9983db2 100644
--- a/utils.c
+++ b/utils.c
@@ -47,6 +47,7 @@
 #include utils.h
 #include volumes.h
 #include ioctl.h
+#include btrfs-list.h
 
 #ifndef BLKDISCARD
 #define BLKDISCARD _IO(0x12,119)
@@ -2087,3 +2088,77 @@ int lookup_ino_rootid(int fd, u64 *rootid)
 
return 0;
 }
+
+/* scans for fsid(s) in the kernel using the btrfs-control
+ * interface.
+ */
+int get_fslist(struct btrfs_ioctl_fslist **out_fslist, u64 *out_count)
+{
+   int ret, fd, e;
+   struct btrfs_ioctl_fslist_args *fsargs;
+   struct btrfs_ioctl_fslist_args *fsargs_saved = NULL;
+   struct btrfs_ioctl_fslist *fslist;
+   u64 sz;
+   int count;
+
+   fd = open(/dev/btrfs-control, O_RDWR);
+   e = errno;
+   if (fd  0) {
+   perror(failed to open /dev/btrfs-control);
+   return -e;
+   }
+
+   /* space to hold 512 fsids, doesn't matter if small
+* it would fail and return count so then we try again
+*/
+   count = 512;
+again:
+   sz = sizeof(*fsargs) + sizeof(*fslist) * count;
+
+   fsargs_saved = fsargs = malloc(sz);
+   if (!fsargs) {
+   close(fd);
+   return -ENOMEM;
+   }
+
+   memset(fsargs, 0, sz);
+   fsargs-count = count;
+
+   ret = ioctl(fd, BTRFS_IOC_GET_FSLIST, fsargs);
+   e = errno;
+   if (ret == 1) {
+   /* out of size so reallocate */
+   count = fsargs-count;
+   free(fsargs);
+   goto again;
+   } else if (ret  0) {
+   printf(ERROR: scan_fsid ioctl failed - %s\n,
+   strerror(e));
+   ret = -e;
+   goto out;
+   }
+
+   /* ioctl returns fsid count in count parameter*/
+
+   *out_count = fsargs-count;
+   if (*out_count == 0) {
+   *out_fslist = NULL;
+   ret = 0;
+   goto out;
+   }
+
+   fslist = (struct btrfs_ioctl_fslist *) (++fsargs);
+
+   sz = sizeof(*fslist) * *out_count;
+   *out_fslist = malloc(sz);
+   if (*out_fslist == NULL) {
+   ret = -ENOMEM;
+   goto out;
+   }
+   memcpy(*out_fslist, fslist, sz);
+   ret = 0;
+out:
+   free(fsargs_saved);
+   close(fd);
+   return ret;
+}
diff --git a/utils.h b/utils.h
index 6f4b10c..a14a887 100644
--- a/utils.h
+++ b/utils.h
@@ -94,5 +94,6 @@ int ask_user(char *question);
 int lookup_ino_rootid(int fd, u64 *rootid);
 int btrfs_scan_lblkid(int update_kernel);
 int get_btrfs_mount(const char *dev, char *mp, size_t mp_size);
+int get_fslist(struct btrfs_ioctl_fslist **out_fslist, u64 *out_count);
 
 #endif
-- 
1.7.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  

Re: [PATCH 1/3] btrfs-progs: mechanism to fetch fsinfo from btrfs-control v3

[Anand Jain]

On 11/07/2013 06:04 PM, Stefan Behrens wrote:

On Thu,  7 Nov 2013 18:01:41 +0800, Anand Jain wrote:

+   *out_fslist = malloc(sz);
+   if (*out_fslist == NULL) {
+   ret = -ENOMEM;
+   goto out;
+   }
+   memcpy(*out_fslist, fslist, sz);
+   ret = 0;
+out:
+   free(fsargs_saved);
+   close(fd);
+   return 0;
+}


return ret;


 nice catch. Thanks.
 Also fsargs_saved wasn't initialized along with one of early goto out.
 I have fixed it as well in v4 which was sent out.

Thanks, Anand

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2] Btrfs: don't wait for ordered data outside desired range

[Filipe David Borba Manana]

In btrfs_wait_ordered_range(), if we found an extent to the left
of the start of our desired wait range and the last byte of that
extent is 1 less than the desired range's start, we would would
wait for the IO completion of that extent and all other extents
to the left of that extent, unnecessarily.

Signed-off-by: Filipe David Borba Manana fdman...@gmail.com
---

V2: Updated commit message.

 fs/btrfs/ordered-data.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index 25a8f38..c37124b 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -803,7 +803,7 @@ int btrfs_wait_ordered_range(struct inode *inode, u64 
start, u64 len)
btrfs_put_ordered_extent(ordered);
break;
}
-   if (ordered-file_offset + ordered-len  start) {
+   if (ordered-file_offset + ordered-len = start) {
btrfs_put_ordered_extent(ordered);
break;
}
-- 
1.7.9.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/9] block: Convert various code to bio_for_each_segment()

[Jan Kara]

On Mon 04-11-13 15:36:19, Kent Overstreet wrote:
 With immutable biovecs we don't want code accessing bi_io_vec directly -
 the uses this patch changes weren't incorrect since they all own the
 bio, but it makes the code harder to audit for no good reason - also,
 this will help with multipage bvecs later.
  I think you've missed the code in fs/ext4/page-io.c in the conversion
(likely because it was added relatively recently).

Honza
 
 Signed-off-by: Kent Overstreet k...@daterainc.com
 Cc: Jens Axboe ax...@kernel.dk
 Cc: Alexander Viro v...@zeniv.linux.org.uk
 Cc: Chris Mason chris.ma...@fusionio.com
 Cc: Jaegeuk Kim jaegeuk@samsung.com
 Cc: Joern Engel jo...@logfs.org
 Cc: Prasad Joshi prasadjoshi.li...@gmail.com
 Cc: Trond Myklebust trond.mykleb...@netapp.com
 ---
  fs/btrfs/compression.c   | 10 --
  fs/btrfs/disk-io.c   | 11 ---
  fs/btrfs/extent_io.c | 37 ++---
  fs/btrfs/inode.c | 15 ++-
  fs/f2fs/data.c   | 13 +
  fs/f2fs/segment.c| 12 +---
  fs/logfs/dev_bdev.c  | 18 +++---
  fs/mpage.c   | 17 -
  fs/nfs/blocklayout/blocklayout.c | 34 +-
  9 files changed, 66 insertions(+), 101 deletions(-)
 
 diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
 index 06ab821..52e7848 100644
 --- a/fs/btrfs/compression.c
 +++ b/fs/btrfs/compression.c
 @@ -203,18 +203,16 @@ csum_failed:
   if (cb-errors) {
   bio_io_error(cb-orig_bio);
   } else {
 - int bio_index = 0;
 - struct bio_vec *bvec = cb-orig_bio-bi_io_vec;
 + int i;
 + struct bio_vec *bvec;
  
   /*
* we have verified the checksum already, set page
* checked so the end_io handlers know about it
*/
 - while (bio_index  cb-orig_bio-bi_vcnt) {
 + bio_for_each_segment_all(bvec, cb-orig_bio, i)
   SetPageChecked(bvec-bv_page);
 - bvec++;
 - bio_index++;
 - }
 +
   bio_endio(cb-orig_bio, 0);
   }
  
 diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
 index 62176ad..733182e 100644
 --- a/fs/btrfs/disk-io.c
 +++ b/fs/btrfs/disk-io.c
 @@ -850,20 +850,17 @@ int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, 
 struct inode *inode,
  
  static int btree_csum_one_bio(struct bio *bio)
  {
 - struct bio_vec *bvec = bio-bi_io_vec;
 - int bio_index = 0;
 + struct bio_vec *bvec;
   struct btrfs_root *root;
 - int ret = 0;
 + int i, ret = 0;
  
 - WARN_ON(bio-bi_vcnt = 0);
 - while (bio_index  bio-bi_vcnt) {
 + bio_for_each_segment_all(bvec, bio, i) {
   root = BTRFS_I(bvec-bv_page-mapping-host)-root;
   ret = csum_dirty_buffer(root, bvec-bv_page);
   if (ret)
   break;
 - bio_index++;
 - bvec++;
   }
 +
   return ret;
  }
  
 diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
 index 0df176a..ea5a08b 100644
 --- a/fs/btrfs/extent_io.c
 +++ b/fs/btrfs/extent_io.c
 @@ -2014,7 +2014,7 @@ int repair_io_failure(struct btrfs_fs_info *fs_info, 
 u64 start,
   }
   bio-bi_bdev = dev-bdev;
   bio_add_page(bio, page, length, start - page_offset(page));
 - btrfsic_submit_bio(WRITE_SYNC, bio);
 + btrfsic_submit_bio(WRITE_SYNC, bio); /* XXX: submit_bio_wait() */
   wait_for_completion(compl);
  
   if (!test_bit(BIO_UPTODATE, bio-bi_flags)) {
 @@ -2340,12 +2340,13 @@ int end_extent_writepage(struct page *page, int err, 
 u64 start, u64 end)
   */
  static void end_bio_extent_writepage(struct bio *bio, int err)
  {
 - struct bio_vec *bvec = bio-bi_io_vec + bio-bi_vcnt - 1;
 + struct bio_vec *bvec;
   struct extent_io_tree *tree;
   u64 start;
   u64 end;
 + int i;
  
 - do {
 + bio_for_each_segment_all(bvec, bio, i) {
   struct page *page = bvec-bv_page;
   tree = BTRFS_I(page-mapping-host)-io_tree;
  
 @@ -2363,14 +2364,11 @@ static void end_bio_extent_writepage(struct bio *bio, 
 int err)
   start = page_offset(page);
   end = start + bvec-bv_offset + bvec-bv_len - 1;
  
 - if (--bvec = bio-bi_io_vec)
 - prefetchw(bvec-bv_page-flags);
 -
   if (end_extent_writepage(page, err, start, end))
   continue;
  
   end_page_writeback(page);
 - } while (bvec = bio-bi_io_vec);
 + }
  
   bio_put(bio);
  }
 @@ -2400,9 +2398,8 @@ endio_readpage_release_extent(struct extent_io_tree 
 *tree, u64 start, u64 len,
   */
  static void end_bio_extent_readpage(struct bio *bio, int err)
  {
 + struct bio_vec *bvec;
  

Re: Fwd: unable to delete files after kernel upgrade from 3.8.10 to 3.12

[Bartosz Kulicki]

Hi Stefan,

Yes, I realise that.
As I pointed out in bug report I have tried workarounds suggested in
wiki and more. To be precise: partial balancing didn't work (completed
without error but no change), balacing fs failed with ENOSPC,
clobbering a file didn't work (ENOSPC), btrfs-zero-log completed but
no change.

The filesystem was quite full 98% but I could still write to it before
rebooting with new kernel.
I had to nuke the fs since then and restore from backup but I captured
image with btrfs-image before running btrfs-zero-log.


regards,
Bart


On Thu, Nov 7, 2013 at 8:49 AM, Stefan Behrens
sbehr...@giantdisaster.de wrote:
 On Wed, 6 Nov 2013 21:53:25 +, Bartosz Kulicki wrote:
 As per subject. Seems UUID tree creation failed after upgrade. I could
 not mount filesystem under 3.12. Going back to 3.8.10 allowed me to
 mount fs but I could no longer perform any deletes, writes etc.

 I've opened a bug report here.
 https://bugzilla.kernel.org/show_bug.cgi?id=64461

 I've included link to image captured with btrfs-image.

 Please CC me if further information is needed as I'm not subscribed to
 the mailing list.


 ENOSPC means you're out of disk space. A copy-on-write filesystem needs
 disk space for delete operations, that's not a bug.

 What does 'btrfs fi df /mountpoint' say?
 How much disk space do you have, how much is allocated?

 And try the procedure that is described in the wiki:

 https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#I_get_.22No_space_left_on_device.22_errors.2C_but_df_says_I.27ve_got_lots_of_space


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: unable to delete files after kernel upgrade from 3.8.10 to 3.12

[Bartosz Kulicki]

FWIW - just before nuking the fs I have added a 3GB loopback device to btrfs.

This restored ability to delete the files but I could not remove the
loopback after deleting some large files (if I remember correctly
error I got was block device required)

cheers,
Bart
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] Btrfs: change ioctl number of BTRFS_IOC_START_SYNC to 21

[Liu Bo]

On Wed, Nov 06, 2013 at 02:24:25PM +0100, David Sterba wrote:
 On Wed, Nov 06, 2013 at 10:42:35AM +0100, Stefan Behrens wrote:
  Sharing the ioctl value seems to have happened by mistake and is
  uncommon, but IMHO now it's too late to change this interface just for
  esthetical reasons.
 
 Agreed, it's part of ABI, ioctl.h is append only.

Ok, I think that's why I tagged it as 'RFC' :-)

-liubo
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: unable to delete files after kernel upgrade from 3.8.10 to 3.12

[Duncan]

Bartosz Kulicki posted on Thu, 07 Nov 2013 11:43:15 + as excerpted:

 FWIW - just before nuking the fs I have added a 3GB loopback device to
 btrfs.
 
 This restored ability to delete the files but I could not remove the
 loopback after deleting some large files (if I remember correctly error
 I got was block device required)

Having read the wiki and the list but not having (yet) gotten into that 
sort of situation myself, I'm following this with some interest, and
was just about to ask if you'd tried that.

OK, so you added the loop device and were then able to do deletes but 
afterward could not delete the device...

The device thus obviously added some room for the filesystem to work, so 
you could delete files.  After that file-delete, did you then try a 
balance again?

Because as documented, and as demonstrated by the initial now allowed 
file-delete, that should have given you just enough extra metadata space 
to get you out of the jam, allowing a balance to finish, which in theory 
at least would have freed up enough space due to the compaction of the 
balance, to then finally do a proper btrfs device delete on the temporary 
loop device.

But without that last balance attempt after the loop add and successful 
file delete, we'll never know if the temporary loop device could have 
then been successfully btrfs device deleted or not.

Which is a bit frustrating here, since that's exactly the bit omitted 
from your posting what you tried, but what I'm counting on to get me out 
of a similar jam, should I ever get in it...

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

How to fix/remove csum failed ino error

[Anatol Pomozov]

Hi

I use Linux Arch, kernel 3.11.6.

Recently I had a disk crash and number of my files got corrupted. To
avoid this situation again I added more disks I trying to convert the
data to raid1:

# btrfs balance start -dconvert=raid1 -mconvert=raid1 /

But unfortunately it fails with IO erro. In dmesg I see

[ 5374.216320] BTRFS info (device sda3): csum failed ino 362 off
4993024 csum 1283121890 private 3720296651
[ 5374.219656] BTRFS info (device sda3): csum failed ino 362 off
5242880 csum 857237386 private 2562492866
[ 5374.222628] BTRFS info (device sda3): csum failed ino 362 off
5767168 csum 645194099 private 3149624654
[ 5374.223068] BTRFS info (device sda3): csum failed ino 362 off
4993024 csum 1283121890 private 3720296651

I looks like some files are corrupted. I would like either
fix/regenerate those files (e.g. reinstall from packages) or remove
them (as they corrupted anyway).

But I need to know what are these files. ino 362 mentioned in the
message does not exist on the file system:

# find / -mount -inum 362
finds nothing.

So I assume this ino is some internal identifier. I checked function
btrfs_ino() from btrfs_inode.h and the output value can be either
BTRFS_I(inode)-location.objectid
or
inode-i_ino

I believe 362 is BTRFS_I(inode)-location.objectid

So my question is how to find a file that has this id corresponding?
How to remove this object and finally make the raid1 conversion? Also
is it possible to improve the error message so users can find failing
objects (e.g. include the real inode number)?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Warning Your Mailbox Has Exceeded Quota Limit

[Kell, Todd]

Dear user,

Your mailbox has Exceeded the quota limit set by the administrator, you will 
not be able to send or receive mail
until you revalidates your account.

Please click the link below or copy paste to your browser to validate your 
mailbox.
http://tinylink.net/quotalimit

Failure to do this will result limited access to your mailbox and failure to 
update your account within 48-hours,
of this update notification, your account will be closed permanently.

Thanks
System Administrator.--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 02/17] btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue

[David Sterba]

On Thu, Nov 07, 2013 at 10:33:32AM +0100, Stefan Behrens wrote:
  +struct btrfs_work_struct {
  +   void (*func)(struct btrfs_work_struct *arg);
  +   void (*ordered_func)(struct btrfs_work_struct *arg);
  +   void (*ordered_free)(struct btrfs_work_struct *arg);
  +
  +   /* Don't touch things below */
  +   struct work_struct normal_work;
  +   struct work_struct ordered_work;
  +   struct completion normal_completion;
  +};
 
 If you compare the Btrfs sources before applying your patchset and after
 applying all 17 patches, one change is this:
 -struct btrfs_work {
 +struct btrfs_work_struct {
 
 Which causes changes s/struct btrfs_work/struct btrfs_work_struct/ like
 in patch 16/17:
 - struct btrfs_work   work;
 + struct btrfs_work_struct
 + work;
 
 -static void scrub_bio_end_io_worker(struct btrfs_work *work);
 +static void scrub_bio_end_io_worker(struct btrfs_work_struct *work);
 
 I just don't see any good reason for renaming 'struct foo' to 'struct
 foo_struct'.

It seems to be meaningfull only though out this patchset. The old
contents of btrfs_work is different from btrfs_work_struct, I agree it's
right to have the name without _struct suffix. But then the change to
new worker structs would have to be done in one single patch, while
there are 10+ patches converting each worker type.

I suggest to add one more patch to the end that removes the _struct
suffix again, so the series does not have to be redone.

david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 03/17] btrfs: Add high priority workqueue support for btrfs_workqueue_struct

[David Sterba]

On Thu, Nov 07, 2013 at 01:51:53PM +0800, Qu Wenruo wrote:
 @@ -753,6 +754,19 @@ struct btrfs_workqueue_struct 
 *btrfs_alloc_workqueue(char *name,
   }
   }
  
 + if (high_name) {
 + ret-high_wq = alloc_workqueue(high_name,
 +flags | WQ_HIGHPRI,
 +max_active);

I'd really like to add the btrfs- prefix of the workqueue. Quoting our
previous discussion:

  * the thread names lost the btrfs- prefix, this makes it hard to
 identify the processes and we want this, either debugging or
 performance monitoring
 Yes, that's right.
 But the problem is, even I added btrfs- prefix to the wq,
 the real work executor is kernel workers without any prefix.
 Still hard to debugging due to the workqueue mechanism.

AFAICS the string passed down to alloc_workqueue ends up in the process
name, this is what I'm expecing and don't understand what do you mean.


david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to fix/remove csum failed ino error

[Frank Holton]

Hey Anatol,

I just checked and on my filesystem inode number 362 corresponds to
part of the free space cache. You can check this yourself by running
(as root)

btrfs-debug-tree /dev/sdb | grep (362  -A 3 -B 1

where /dev/sdb is one of the devices from your filesystem.

It printed the following for me, note the location key (362
INODE_ITEM) under the FREE_SPACE key. Yours might be different but if
you see FREE_SPACE that points to the free space cache.

item 100 key (362 INODE_ITEM 0) itemoff 21857 itemsize 160
inode generation 2004 transid 2004 size 262144 block
group 0 mode 100600 links 1
item 101 key (362 EXTENT_DATA 0) itemoff 21804 itemsize 53
extent data disk byte 41903296512 nr 262144
extent data offset 0 nr 262144 ram 262144
extent compression 0
--
item 148 key (FREE_SPACE UNTYPED 113845993472) itemoff 23807 itemsize 41
location key (362 INODE_ITEM 0)
cache generation 2004 entries 2 bitmaps 0

I would try to mount the filesystem with the clear_cache option, which
should clear the current space cache and force a rebuild.

Hope that helps,
Frank



On Thu, Nov 7, 2013 at 9:42 AM, Anatol Pomozov anatol.pomo...@gmail.com wrote:
 Hi

 I use Linux Arch, kernel 3.11.6.

 Recently I had a disk crash and number of my files got corrupted. To
 avoid this situation again I added more disks I trying to convert the
 data to raid1:

 # btrfs balance start -dconvert=raid1 -mconvert=raid1 /

 But unfortunately it fails with IO erro. In dmesg I see

 [ 5374.216320] BTRFS info (device sda3): csum failed ino 362 off
 4993024 csum 1283121890 private 3720296651
 [ 5374.219656] BTRFS info (device sda3): csum failed ino 362 off
 5242880 csum 857237386 private 2562492866
 [ 5374.222628] BTRFS info (device sda3): csum failed ino 362 off
 5767168 csum 645194099 private 3149624654
 [ 5374.223068] BTRFS info (device sda3): csum failed ino 362 off
 4993024 csum 1283121890 private 3720296651

 I looks like some files are corrupted. I would like either
 fix/regenerate those files (e.g. reinstall from packages) or remove
 them (as they corrupted anyway).

 But I need to know what are these files. ino 362 mentioned in the
 message does not exist on the file system:

 # find / -mount -inum 362
 finds nothing.

 So I assume this ino is some internal identifier. I checked function
 btrfs_ino() from btrfs_inode.h and the output value can be either
 BTRFS_I(inode)-location.objectid
 or
 inode-i_ino

 I believe 362 is BTRFS_I(inode)-location.objectid

 So my question is how to find a file that has this id corresponding?
 How to remove this object and finally make the raid1 conversion? Also
 is it possible to improve the error message so users can find failing
 objects (e.g. include the real inode number)?
 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 01/17] btrfs: Cleanup the unused struct async_sched.

[Josef Bacik]

On Thu, Nov 07, 2013 at 01:51:51PM +0800, Qu Wenruo wrote:
 The struct async_sched is not used by any codes and can be removed.
 
 Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com

Reviewed-by: Josef Bacik jba...@fusionio.com

Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 00/17] Replace btrfs_workers with kernel workqueue based btrfs_workqueue_struct

[David Sterba]

On Thu, Nov 07, 2013 at 01:51:50PM +0800, Qu Wenruo wrote:
 Qu Wenruo (17):
   btrfs: Cleanup the unused struct async_sched.
   btrfs: Added btrfs_workqueue_struct implemented ordered execution
 based on kernel workqueue
   btrfs: Add high priority workqueue support for btrfs_workqueue_struct
   btrfs: Add threshold workqueue based on kernel workqueue
   btrfs: Replace fs_info-workers with btrfs_workqueue.
   btrfs: Replace fs_info-delalloc_workers with btrfs_workqueue
   btrfs: Replace fs_info-submit_workers with btrfs_workqueue.
   btrfs: Replace fs_info-flush_workers with btrfs_workqueue.
   btrfs: Replace fs_info-endio_* workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-rmw_workers workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-cache_workers workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-readahead_workers workqueue with
 btrfs_workqueue.
   btrfs: Replace fs_info-fixup_workers workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-delayed_workers workqueue with
 btrfs_workqueue.
   btrfs: Replace fs_info-qgroup_rescan_worker workqueue with
 btrfs_workqueue.
   btrfs: Replace fs_info-scrub_* workqueue with btrfs_workqueue.
   btrfs: Cleanup the old btrfs_worker.

One comment to all patches: please try to stick to the kernel coding
style, you're changing code that will hopefully stay untouched for a
long time, please get the formatting right from the beginning.

- comment formatting
- blank line after declaration block

The patches need functional review, I'm not happy to point out such
things, but please understand that it distracts (and sometimes hurts
eyes) during review.

david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 00/17] Replace btrfs_workers with kernel workqueue based btrfs_workqueue_struct

[Chris Mason]

Quoting Qu Wenruo (2013-11-07 00:51:50)
 Add a new btrfs_workqueue_struct which use kernel workqueue to implement
 most of the original btrfs_workers, to replace btrfs_workers.
 
 With this patchset, redundant workqueue codes are replaced with kernel
 workqueue infrastructure, which not only reduces the code size but also the
 effort to maintain it.
 
 More performace tests are ongoing, the result from sysbench shows minor
 improvement on the following server:
 CPU: two-way Xeon X5660
 RAM: 4G 
 HDD: SAS HDD, 150G total, 40G partition for btrfs test
 
 Test result:
 Mode|Num_threads|block size|extra flags|performance change vs 3.11 kernel
 rndrd   1   4K  none+1.22%
 rndrd   1   32K none+1.00%
 rndrd   8   32K sync+1.35%
 seqrd   8   4K  direct  +5.56%
 seqwr   8   4K  none-1.26%
 seqwr   8   32K sync+1.20%
 
 Changes below 1% are not mentioned.
 Overall the patchset doesn't change the performance on HDD.
 
 Since more tests are needed, more test result are welcomed.

Thanks for working on this, it's really good to move toward a single set
of workqueues in the kernel.

Have you benchmarked with compression on?  Especially on modern
hardware, the crcs don't exercise the workqueues very much.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 02/17] btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue

[Josef Bacik]

On Thu, Nov 07, 2013 at 01:51:52PM +0800, Qu Wenruo wrote:
 Use kernel workqueue to implement a new btrfs_workqueue_struct, which
 has the ordering execution feature like the btrfs_worker.
 
 The func is executed in a concurrency way, and the
 ordred_func/ordered_free is executed in the sequence them are queued
 after the corresponding func is done.
 The new btrfs_workqueue use 2 workqueues to implement the original
 btrfs_worker, one for the normal work and one for ordered work.
 
 At this patch, high priority work queue or thresholding is not added yet.
 The high priority feature and thresholding will be added in the following 
 patches.
 
 Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com

Reviewed-by: Josef Bacik jba...@fusionio.com

Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 02/17] btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue

[Josef Bacik]

On Thu, Nov 07, 2013 at 01:08:26PM -0500, Josef Bacik wrote:
 On Thu, Nov 07, 2013 at 01:51:52PM +0800, Qu Wenruo wrote:
  Use kernel workqueue to implement a new btrfs_workqueue_struct, which
  has the ordering execution feature like the btrfs_worker.
  
  The func is executed in a concurrency way, and the
  ordred_func/ordered_free is executed in the sequence them are queued
  after the corresponding func is done.
  The new btrfs_workqueue use 2 workqueues to implement the original
  btrfs_worker, one for the normal work and one for ordered work.
  
  At this patch, high priority work queue or thresholding is not added yet.
  The high priority feature and thresholding will be added in the following 
  patches.
  
  Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
 
 Reviewed-by: Josef Bacik jba...@fusionio.com
 

Keep in mind I agree with Stefan and Dave's comments so please make those
changes, but after that you can add my reviewed by.  Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfsck errors is it save to fix?

[Hendrik Friedel]

Hello again,

can someone please help me on this?

Regards,
Hendrik

Am 06.11.2013 07:45, schrieb Hendrik Friedel:

Hello,

sorry, I was totally unaware still being on 3.11rc2.

I re-ran btrfsck with the same result:
./btrfs-progs/btrfsck /dev/sdc1
Checking filesystem on /dev/sdc1
UUID: 989306aa-d291-4752-8477-0baf94f8c42f
checking extents
checking free space cache
checking fs roots
root 256 inode 9579 errors 100
root 256 inode 9580 errors 100
root 256 inode 14258 errors 100
root 256 inode 14259 errors 100
root  inode 9579 errors 100
root  inode 9580 errors 100
root  inode 14258 errors 100
root  inode 14259 errors 100
found 1992865028914 bytes used err is 1
total csum bytes: 3207847732
total tree bytes: 3902865408
total fs tree bytes: 38875136
total extent tree bytes: 135864320
btree space waste bytes: 411665032
file data blocks allocated: 3426722545664
  referenced 3426000965632
Btrfs v0.20-rc1-358-g194aa4a


Now dmesg and the syslog stay clear of entries relatet to btrfs.
But I think, that might also be a coincidence: I ran the old Kernel for
weeks until this error came, whereas I ran this kernel merely 12h.

Now:
Does it make sense to futher try to find a possible bug, or do we
suspect it is fixed? If so: How can I help?

And:
Can I fix these Problems safely with btrfsck?

Regards,
Hendrik

Am 05.11.2013 03:03, schrieb cwillu:

On Mon, Nov 4, 2013 at 3:14 PM, Hendrik Friedel
hend...@friedels.name wrote:

Hello,

the list was quite full with patches, so this might have been hidden.
Here the complete Stack.
Does this help? Is this what you needed?

[95764.899294] CPU: 1 PID: 21798 Comm: umount Tainted: GFCIO
3.11.0-031100rc2-generic #201307211535


Can you reproduce the problem under the released 3.11 or 3.12?  An
-rc2 is still pretty early in the release cycle, and I wouldn't be at
all surprised if it was a bug added and fixed in a later rc.







--
Hendrik Friedel
Auf dem Brink 12
28844 Weyhe
Mobil 0178 1874363
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/9] block: Convert various code to bio_for_each_segment()

[Kent Overstreet]

On Thu, Nov 07, 2013 at 12:26:30PM +0100, Jan Kara wrote:
 On Mon 04-11-13 15:36:19, Kent Overstreet wrote:
  With immutable biovecs we don't want code accessing bi_io_vec directly -
  the uses this patch changes weren't incorrect since they all own the
  bio, but it makes the code harder to audit for no good reason - also,
  this will help with multipage bvecs later.
   I think you've missed the code in fs/ext4/page-io.c in the conversion
 (likely because it was added relatively recently).

Thanks!
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel BUG at fs/btrfs/ctree.c:2964!

[Ilari Stenroth]

On 2.11.2013 3.34, Josef Bacik wrote:
 
 On this box can you run
 
 gdb btrfs.ko
 list *(__btrfs_drop_extents+0x59d)
 
 I want to know where exactly this is coming from so I can start trying to 
 figure
 out how it's happening.  Thanks,

Hi,

I've got the same issue. Btrfs FS crashes on my box running Linux 3.11.7.

It seems that the crash happens when I start collectd daemon. Maybe a
corrupted RRD file somewhere.
I don't have time for debugging this issue further so I'm going to
re-format this box to ext4. Btrfs is statically compiled in kernel so
I'm unable to gdb btrfs.ko. No debug info in kernel image. Sorry not
being able to help.

/dev/sda2 on / type btrfs (rw,noatime,nodiratime,ssd,space_cache)

Mount options used to have also autodefrag but I removed it while hoping
it would make the box more stable.

# btrfs fi show
Label: 'root'  uuid: 87943e37-a134-4b8d-bcf2-989334d23b75
Total devices 2 FS bytes used 6.22GB
devid2 size 10.00GB used 9.98GB path /dev/sdb2
devid1 size 10.00GB used 10.00GB path /dev/sda2


# btrfs fi df /
Data, RAID1: total=8.97GB, used=5.99GB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=234.92MB
Metadata: total=8.00MB, used=0.00

A screen grab from the console after the crash happened:
http://s3-eu-west-1.amazonaws.com/ilari.stenroth/btrfs-crash-201311/Screen+Shot+2013-11-07+at+9.19.27.png

--
Ilari Stenroth


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Question about btrfs as root filesystem

[Michael Göhler]

Hi,

I'm a contributor of the Arch Linux package mkinitcpio-btrfs [1]. The 
goal of this hook is to provide Btrfs rollback support for root 
filesystems directly from initrd.


Technically we are using a subvolume to store the root filesystem. The 
user can snapshot it entirely and boot from this snapshot. In case of 
rollback our hook snapshots the snapshot again, to keep its original 
unchanged. The boot subvolume is then set with 'btrfs subvolume 
set-default' and mounted without subvol/subvolid option by Arch's 
default mount handler. That way, we ensure the best compatibility and 
lowest maintenance, as we don't overwrite default init functions.


Assuming we have the following setup:

# btrfs su li -p /
ID 256 gen 86 parent 5 top level 5 path root
ID 259 gen 86 parent 256 top level 256 path var
ID 260 gen 86 parent 256 top level 256 path usr

The use case for that is to set quotas for the child subvolumes.

Now, if we snapshots the root subvolume, the child subvolumes are not 
snapshoted with it. There is no back reference which would allow Btrfs 
to auto-mount the original child subvolumes when we mount the snapshot 
as new root filesystem. Of cause we could snapshot the childs separately 
into their desired directories. But this would not help, because our 
hook snapshots the snapshot again, to keep it's original untouched while 
rolling back. And we don't have fstab to find out the correct mount 
points at this early boot stage.


Atm. all scenarios results in /usr/bin/init not found.

So here comes my question:
Wouldn't it be helpful to add a --recursive option to 'btrfs subvolume 
snapshot' to snapshot child subvolumes together with their parent?
Or maybe it is possible to add some functionality to reference the child 
subvolumes on the snapshots fs-tree to allow auto-mounting?


I appreciate other ideas or opinions too.

Thanks,
Michael

[1] https://aur.archlinux.org/packages/mkinitcpio-btrfs/
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 02/17] btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue

[Qu Wenruo]

On thu, 7 Nov 2013 17:05:29 +0100, David Sterba wrote:

On Thu, Nov 07, 2013 at 10:33:32AM +0100, Stefan Behrens wrote:

+struct btrfs_work_struct {
+   void (*func)(struct btrfs_work_struct *arg);
+   void (*ordered_func)(struct btrfs_work_struct *arg);
+   void (*ordered_free)(struct btrfs_work_struct *arg);
+
+   /* Don't touch things below */
+   struct work_struct normal_work;
+   struct work_struct ordered_work;
+   struct completion normal_completion;
+};

If you compare the Btrfs sources before applying your patchset and after
applying all 17 patches, one change is this:
-struct btrfs_work {
+struct btrfs_work_struct {

Which causes changes s/struct btrfs_work/struct btrfs_work_struct/ like
in patch 16/17:
-   struct btrfs_work   work;
+   struct btrfs_work_struct
+   work;

-static void scrub_bio_end_io_worker(struct btrfs_work *work);
+static void scrub_bio_end_io_worker(struct btrfs_work_struct *work);

I just don't see any good reason for renaming 'struct foo' to 'struct
foo_struct'.

It seems to be meaningfull only though out this patchset. The old
contents of btrfs_work is different from btrfs_work_struct, I agree it's
right to have the name without _struct suffix. But then the change to
new worker structs would have to be done in one single patch, while
there are 10+ patches converting each worker type.

I suggest to add one more patch to the end that removes the _struct
suffix again, so the series does not have to be redone.

david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Yes, the _struct suffix is mostly used to distinguishfrom the original
btrfs_workers, and also try to use the kernel workqueue naming, which has
the _struct suffix.
And itis true that the long struct name makes some original codes ugly.

I'll add a new patchafter all these patches to remove the long suffix.

Thanks
Qu

--
-
Qu Wenruo
Development Dept.I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No. 6 Wenzhu Road, Nanjing, 210012, China
TEL: +86+25-86630566-8526
COINS: 7998-8526
FAX: +86+25-83317685
MAIL: quwen...@cn.fujitsu.com
-

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [OPW kernel] [PATCH v2] btrfs:check-integrity.c: replace kmalloc with kmalloc_array and kzalloc with kzalloc_array.

[Rusty Russell]

Himangi Saraogi himangi...@gmail.com writes:
 This patch replaces kmalloc(size * nr, ) with kmalloc_array(nr, size)
 as kmalloc_array() is preferred because it can check that the
 calculation doesn't wrap and won't return a smaller allocation.
 Also kzalloc(size * nr) was replaced with kzalloc_array().

 Signed-off-by: Himangi Saraogi himangi...@gmail.com

Reviewed-by: Rusty Russell ru...@rustcorp.com.au

Cheers,
Rusty.

 ---
  fs/btrfs/check-integrity.c |   12 +++-
  1 file changed, 7 insertions(+), 5 deletions(-)

 diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c
 index 1c47be1..925a52b 100644
 --- a/fs/btrfs/check-integrity.c
 +++ b/fs/btrfs/check-integrity.c
 @@ -1660,9 +1660,10 @@ static int btrfsic_read_block(struct btrfsic_state 
 *state,
  
   num_pages = (block_ctx-len + (u64)PAGE_CACHE_SIZE - 1) 
   PAGE_CACHE_SHIFT;
 - block_ctx-mem_to_free = kzalloc((sizeof(*block_ctx-datav) +
 -   sizeof(*block_ctx-pagev)) *
 -  num_pages, GFP_NOFS);
 + block_ctx-mem_to_free = kzalloc_array(num_pages,
 + (sizeof(*block_ctx-datav) +
 +  sizeof(*block_ctx-pagev)),
 + GFP_NOFS);
   if (!block_ctx-mem_to_free)
   return -1;
   block_ctx-datav = block_ctx-mem_to_free;
 @@ -3031,8 +3032,9 @@ void btrfsic_submit_bio(int rw, struct bio *bio)
  (unsigned long long)bio-bi_sector, dev_bytenr,
  bio-bi_bdev);
  
 - mapped_datav = kmalloc(sizeof(*mapped_datav) * bio-bi_vcnt,
 -GFP_NOFS);
 + mapped_datav = kmalloc_array(bio-bi_vcnt,
 +  sizeof(*mapped_datav),
 +  GFP_NOFS);
   if (!mapped_datav)
   goto leave;
   for (i = 0; i  bio-bi_vcnt; i++) {
 -- 
 1.7.9.5

 -- 
 You received this message because you are subscribed to the Google Groups 
 opw-kernel group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to opw-kernel+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 03/17] btrfs: Add high priority workqueue support for btrfs_workqueue_struct

[Qu Wenruo]

On Thu, 7 Nov 2013 17:41:34 +0100, David Sterba wrote:

On Thu, Nov 07, 2013 at 01:51:53PM +0800, Qu Wenruo wrote:

@@ -753,6 +754,19 @@ struct btrfs_workqueue_struct *btrfs_alloc_workqueue(char 
*name,
}
}
  
+	if (high_name) {

+   ret-high_wq = alloc_workqueue(high_name,
+  flags | WQ_HIGHPRI,
+  max_active);

I'd really like to add the btrfs- prefix of the workqueue. Quoting our
previous discussion:

Sorry, I forgot to add the btrfs- prefix.
But on the other hand, I prefer to add the prefix outside the 
alloc_btrfs_workqueue,

which means change the strings passed to alloc_btrfs_workqueue.
(Maybe add a small macro?)

Which way do you prefer?



* the thread names lost the btrfs- prefix, this makes it hard to
identify the processes and we want this, either debugging or
performance monitoring

Yes, that's right.
But the problem is, even I added btrfs- prefix to the wq,
the real work executor is kernel workers without any prefix.
Still hard to debugging due to the workqueue mechanism.

AFAICS the string passed down to alloc_workqueue ends up in the process
name, this is what I'm expecing and don't understand what do you mean.


david


Sorry for the misunderstanding words.
What I mean is that, since we use the kernel workqueue,
there will be no kthread named the btrfs-worker or something like it.
(Only when WQ_MEM_RECLAIM is set, the rescuer kthread will have the name,
See kernel/workqueue.c if(flags  WQ_MEM_RECLAIM) para).

The real executor will be something like kworker/u8:6(Unbound workqueue),
so even the prefix is added, it's still harder than the original way to 
debug

workqueue.

What makes it worse is that, to simulate the thresholding and ordering,
I added serveral helper functions, which even makes kernel tracing not so
meaningful(all function queued to workqueue will all be normal_helper or
ordered_helper).

Neither way, the prefix will be added, but I consider this may not really
help to debug or monitor.

Qu.



--
-
Qu Wenruo
Development Dept.I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No. 6 Wenzhu Road, Nanjing, 210012, China
TEL: +86+25-86630566-8526
COINS: 7998-8526
FAX: +86+25-83317685
MAIL: quwen...@cn.fujitsu.com
-

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 00/17] Replace btrfs_workers with kernel workqueue based btrfs_workqueue_struct

[Qu Wenruo]

On thu, 7 Nov 2013 18:52:14 +0100, David Sterba wrote:

On Thu, Nov 07, 2013 at 01:51:50PM +0800, Qu Wenruo wrote:

Qu Wenruo (17):
   btrfs: Cleanup the unused struct async_sched.
   btrfs: Added btrfs_workqueue_struct implemented ordered execution
 based on kernel workqueue
   btrfs: Add high priority workqueue support for btrfs_workqueue_struct
   btrfs: Add threshold workqueue based on kernel workqueue
   btrfs: Replace fs_info-workers with btrfs_workqueue.
   btrfs: Replace fs_info-delalloc_workers with btrfs_workqueue
   btrfs: Replace fs_info-submit_workers with btrfs_workqueue.
   btrfs: Replace fs_info-flush_workers with btrfs_workqueue.
   btrfs: Replace fs_info-endio_* workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-rmw_workers workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-cache_workers workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-readahead_workers workqueue with
 btrfs_workqueue.
   btrfs: Replace fs_info-fixup_workers workqueue with btrfs_workqueue.
   btrfs: Replace fs_info-delayed_workers workqueue with
 btrfs_workqueue.
   btrfs: Replace fs_info-qgroup_rescan_worker workqueue with
 btrfs_workqueue.
   btrfs: Replace fs_info-scrub_* workqueue with btrfs_workqueue.
   btrfs: Cleanup the old btrfs_worker.

One comment to all patches: please try to stick to the kernel coding
style, you're changing code that will hopefully stay untouched for a
long time, please get the formatting right from the beginning.

- comment formatting
- blank line after declaration block

The patches need functional review, I'm not happy to point out such
things, but please understand that it distracts (and sometimes hurts
eyes) during review.

david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Thanks for pointing out the coding style problem.
I'll check them again to fix the problem.

Thanks.

Qu

--
-
Qu Wenruo
Development Dept.I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No. 6 Wenzhu Road, Nanjing, 210012, China
TEL: +86+25-86630566-8526
COINS: 7998-8526
FAX: +86+25-83317685
MAIL: quwen...@cn.fujitsu.com
-

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 00/17] Replace btrfs_workers with kernel workqueue based btrfs_workqueue_struct

[Qu Wenruo]

On Thu, 7 Nov 2013 12:54:56 -0500, Chris Mason wrote:

Quoting Qu Wenruo (2013-11-07 00:51:50)

Add a new btrfs_workqueue_struct which use kernel workqueue to implement
most of the original btrfs_workers, to replace btrfs_workers.

With this patchset, redundant workqueue codes are replaced with kernel
workqueue infrastructure, which not only reduces the code size but also the
effort to maintain it.

More performace tests are ongoing, the result from sysbench shows minor
improvement on the following server:
CPU: two-way Xeon X5660
RAM: 4G
HDD: SAS HDD, 150G total, 40G partition for btrfs test

Test result:
Mode|Num_threads|block size|extra flags|performance change vs 3.11 kernel
rndrd   1   4K  none+1.22%
rndrd   1   32K none+1.00%
rndrd   8   32K sync+1.35%
seqrd   8   4K  direct  +5.56%
seqwr   8   4K  none-1.26%
seqwr   8   32K sync+1.20%

Changes below 1% are not mentioned.
Overall the patchset doesn't change the performance on HDD.

Since more tests are needed, more test result are welcomed.

Thanks for working on this, it's really good to move toward a single set
of workqueues in the kernel.

Have you benchmarked with compression on?  Especially on modern
hardware, the crcs don't exercise the workqueues very much.

-chris


Iwill benchmarked on SSD and coompression both before next version of patch.

Thanks.

Qu

--
-
Qu Wenruo
Development Dept.I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No. 6 Wenzhu Road, Nanjing, 210012, China
TEL: +86+25-86630566-8526
COINS: 7998-8526
FAX: +86+25-83317685
MAIL: quwen...@cn.fujitsu.com
-

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 02/17] btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue

[Qu Wenruo]

On Thu, 7 Nov 2013 13:09:56 -0500, Josef Bacik wrote:

On Thu, Nov 07, 2013 at 01:08:26PM -0500, Josef Bacik wrote:

On Thu, Nov 07, 2013 at 01:51:52PM +0800, Qu Wenruo wrote:

Use kernel workqueue to implement a new btrfs_workqueue_struct, which
has the ordering execution feature like the btrfs_worker.

The func is executed in a concurrency way, and the
ordred_func/ordered_free is executed in the sequence them are queued
after the corresponding func is done.
The new btrfs_workqueue use 2 workqueues to implement the original
btrfs_worker, one for the normal work and one for ordered work.

At this patch, high priority work queue or thresholding is not added yet.
The high priority feature and thresholding will be added in the following 
patches.

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com

Reviewed-by: Josef Bacik jba...@fusionio.com


Keep in mind I agree with Stefan and Dave's comments so please make those
changes, but after that you can add my reviewed by.  Thanks,

Josef


I'll apply the Stefan and Dave's comments in the next versionwith
more benchmarks.

Thanks.

Qu

--
-
Qu Wenruo
Development Dept.I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No. 6 Wenzhu Road, Nanjing, 210012, China
TEL: +86+25-86630566-8526
COINS: 7998-8526
FAX: +86+25-83317685
MAIL: quwen...@cn.fujitsu.com
-

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question about btrfs as root filesystem

[Chris Murphy]

On Nov 7, 2013, at 4:45 PM, Michael Göhler vi...@myjm.de wrote:
  The boot subvolume is then set with 'btrfs subvolume set-default' and 
 mounted without subvol/subvolid option by Arch's default mount handler.

I'm unconvinced it's a good idea for it to be used behind the scenes for the 
described purpose. Consider the following:

1. btrfs subvolume set-default uses a user space tool and in my view it's 
primarily a user domain behavior modifier for the mount command.

2. It makes the actual mounted subvolume obscured, cat /proc/self/mountinfo 
doesn't show what subvolume is mounted unless subvol= is explicitly used in 
/etc/fstab.

3. Grub2 (and I'm pretty sure grubby) do not use the set-default. The GRUB 
intent for the prefix to search an absolute path, not one relative to the 
default subvolume. There's a bug that should very recently (week) be fixed, 
where GRUB fails to find prefix if set-default is changed. This maybe isn't 
affecting the particular layout you describe where only rootfs is on btrfs, 
rather than /boot being on its own subvolume.

 That way, we ensure the best compatibility and lowest maintenance, as we 
 don't overwrite default init functions.

I'm sympathetic to the alternative problem, which is that you need to alter 
grub.cfg to use the proper rootflags=subvol= to explicitly use the proper 
snapshot, and also it would mean altering the /etc/fstab within that snapshot.

 
 Now, if we snapshots the root subvolume, the child subvolumes are not 
 snapshoted with it. There is no back reference which would allow Btrfs to 
 auto-mount the original child subvolumes when we mount the snapshot as new 
 root filesystem. Of cause we could snapshot the childs separately into their 
 desired directories. But this would not help, because our hook snapshots the 
 snapshot again, to keep it's original untouched while rolling back. And we 
 don't have fstab to find out the correct mount points at this early boot 
 stage.

The fact of the matter is, we don't have the necessary metadata support in 
btrfs to understand the relationships between snapshots/subvolumes. There is a 
need for this, not least of which is the use case you're describing. This has 
come up with Fedora also for their offline updates rollback they want to 
eventually do. And it's also an issue with distribution installers which see 
these snapshots as wholly unique instances of existing installations, rather 
than as related snapshots.


 
 Atm. all scenarios results in /usr/bin/init not found.
 
 So here comes my question:
 Wouldn't it be helpful to add a --recursive option to 'btrfs subvolume 
 snapshot' to snapshot child subvolumes together with their parent?
 Or maybe it is possible to add some functionality to reference the child 
 subvolumes on the snapshots fs-tree to allow auto-mounting?

Well and it raises the problem of nested subvolumes making the parent  
subvolume undeletable. So I'd question the significant benefit of making nested 
subvolume in particular /var. It's complicating how the OS is to be put back 
together again.


Chris Murphy--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to fix/remove csum failed ino error

[Anatol Pomozov]

Hi, Frank

Thanks for your answer.

On Thu, Nov 7, 2013 at 8:41 AM, Frank Holton fhol...@gmail.com wrote:
 Hey Anatol,

 I just checked and on my filesystem inode number 362 corresponds to
 part of the free space cache. You can check this yourself by running
 (as root)

 btrfs-debug-tree /dev/sdb | grep (362  -A 3 -B 1

 where /dev/sdb is one of the devices from your filesystem.

 It printed the following for me, note the location key (362
 INODE_ITEM) under the FREE_SPACE key. Yours might be different but if
 you see FREE_SPACE that points to the free space cache.

 item 100 key (362 INODE_ITEM 0) itemoff 21857 itemsize 160
 inode generation 2004 transid 2004 size 262144 block
 group 0 mode 100600 links 1
 item 101 key (362 EXTENT_DATA 0) itemoff 21804 itemsize 53
 extent data disk byte 41903296512 nr 262144
 extent data offset 0 nr 262144 ram 262144
 extent compression 0
 --
 item 148 key (FREE_SPACE UNTYPED 113845993472) itemoff 23807 itemsize 
 41
 location key (362 INODE_ITEM 0)
 cache generation 2004 entries 2 bitmaps 0


Indeed my case similar to yours

# btrfs-debug-tree /dev/sda3 | grep (309  -A 3 -B 1

item 1 key (309 INODE_ITEM 0) itemoff 3675 itemsize 160
inode generation 190480 transid 190647 size 0 block group 0 mode
100600 links 1
item 51 key (FREE_SPACE UNTYPED 56937676800) itemoff 1863 itemsize 41
location key (309 INODE_ITEM 0)

So I mounted my filesystem with 'clear_cache' flag:

# mount -o clear_cache /dev/sda3 mydata/

mount says:
/dev/sdc1 on /root/mydata type btrfs (rw,relatime,space_cache,clear_cache)

dmesg also mentions the cache:

[  634.991845] device fsid 25e6a6fa-fe1f-4be5-a638-eeac948f8c21 devid
9 transid 190479 /dev/sda3
[  634.993431] btrfs: force clearing of disk cache
[  634.993435] btrfs: disk space caching is enabled
[  635.046803] btrfs: bdev /dev/sda3 errs: wr 0, rd 0, flush 0,
corrupt 58481, gen 0


The I started raid1 rebalance but the error still presents:

[ 1571.787664] BTRFS info (device sda3): csum failed ino 309 off
4993024 csum 1283121890 private 3720296651
[ 1571.791027] BTRFS info (device sda3): csum failed ino 309 off
5242880 csum 857237386 private 2562492866
[ 1571.793998] BTRFS info (device sda3): csum failed ino 309 off
5767168 csum 645194099 private 3149624654
[ 1571.794389] BTRFS info (device sda3): csum failed ino 309 off
4993024 csum 1283121890 private 3720296651


So my problem still exists. How to fix the block with wrong csum?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to fix/remove csum failed ino error

[Anatol Pomozov]

Hi

I ran btrfsck hoping that it fix the filesystem so 'balance' would not
crash anymore. But btrfsck itself crashed :(

# btrfsck --repair /dev/sda3

   :(
enabling repair mode
Checking filesystem on /dev/sda3
UUID: 25e6a6fa-fe1f-4be5-a638-eeac948f8c21
checking extents
checking fs roots
root 5 inode 522858 errors 1000
root 5 inode 1437358 errors 1000
root 5 inode 1437359 errors 1000
root 5 inode 1437360 errors 1000
root 5 inode 1437361 errors 1000
root 5 inode 1437362 errors 1000
root 5 inode 1437363 errors 1000
root 5 inode 1437368 errors 1000
root 5 inode 1437369 errors 1000
root 5 inode 1437370 errors 1000
root 5 inode 1437371 errors 1000
root 5 inode 1437372 errors 1000
root 5 inode 1437373 errors 1000
root 5 inode 1437374 errors 1000
root 5 inode 1437375 errors 1000
root 5 inode 1437376 errors 1000
root 5 inode 1437377 errors 1000
root 5 inode 1437378 errors 1000
root 5 inode 1437379 errors 1000
root 5 inode 1437380 errors 1000
root 5 inode 1437381 errors 1000
root 5 inode 1437382 errors 1000
root 5 inode 1437383 errors 1000
root 5 inode 1437384 errors 1000
root 5 inode 1437385 errors 1000
root 5 inode 1437386 errors 1000
root 5 inode 1437387 errors 1000
root 5 inode 1437388 errors 1000
root 5 inode 1437389 errors 1000
root 5 inode 1437390 errors 1000
root 5 inode 1437391 errors 1000
root 5 inode 1437392 errors 1000
root 5 inode 1437393 errors 1000
root 5 inode 1437394 errors 1000
root 5 inode 1437395 errors 1000
root 5 inode 1437396 errors 1000
root 5 inode 1437397 errors 1000
root 5 inode 1437398 errors 1000
root 5 inode 1437399 errors 1000
root 5 inode 1437400 errors 1000
root 5 inode 5073119 errors 400
Unable to find block group for 0
btrfsck: extent-tree.c:284: find_search_start: Assertion `!(1)' failed.
[1]583 abort (core dumped)  btrfsck --repair /dev/sda3

On Thu, Nov 7, 2013 at 8:07 PM, Anatol Pomozov anatol.pomo...@gmail.com wrote:
 Hi, Frank

 Thanks for your answer.

 On Thu, Nov 7, 2013 at 8:41 AM, Frank Holton fhol...@gmail.com wrote:
 Hey Anatol,

 I just checked and on my filesystem inode number 362 corresponds to
 part of the free space cache. You can check this yourself by running
 (as root)

 btrfs-debug-tree /dev/sdb | grep (362  -A 3 -B 1

 where /dev/sdb is one of the devices from your filesystem.

 It printed the following for me, note the location key (362
 INODE_ITEM) under the FREE_SPACE key. Yours might be different but if
 you see FREE_SPACE that points to the free space cache.

 item 100 key (362 INODE_ITEM 0) itemoff 21857 itemsize 160
 inode generation 2004 transid 2004 size 262144 block
 group 0 mode 100600 links 1
 item 101 key (362 EXTENT_DATA 0) itemoff 21804 itemsize 53
 extent data disk byte 41903296512 nr 262144
 extent data offset 0 nr 262144 ram 262144
 extent compression 0
 --
 item 148 key (FREE_SPACE UNTYPED 113845993472) itemoff 23807 
 itemsize 41
 location key (362 INODE_ITEM 0)
 cache generation 2004 entries 2 bitmaps 0


 Indeed my case similar to yours

 # btrfs-debug-tree /dev/sda3 | grep (309  -A 3 -B 1

 item 1 key (309 INODE_ITEM 0) itemoff 3675 itemsize 160
 inode generation 190480 transid 190647 size 0 block group 0 mode
 100600 links 1
 item 51 key (FREE_SPACE UNTYPED 56937676800) itemoff 1863 itemsize 41
 location key (309 INODE_ITEM 0)

 So I mounted my filesystem with 'clear_cache' flag:

 # mount -o clear_cache /dev/sda3 mydata/

 mount says:
 /dev/sdc1 on /root/mydata type btrfs (rw,relatime,space_cache,clear_cache)

 dmesg also mentions the cache:

 [  634.991845] device fsid 25e6a6fa-fe1f-4be5-a638-eeac948f8c21 devid
 9 transid 190479 /dev/sda3
 [  634.993431] btrfs: force clearing of disk cache
 [  634.993435] btrfs: disk space caching is enabled
 [  635.046803] btrfs: bdev /dev/sda3 errs: wr 0, rd 0, flush 0,
 corrupt 58481, gen 0


 The I started raid1 rebalance but the error still presents:

 [ 1571.787664] BTRFS info (device sda3): csum failed ino 309 off
 4993024 csum 1283121890 private 3720296651
 [ 1571.791027] BTRFS info (device sda3): csum failed ino 309 off
 5242880 csum 857237386 private 2562492866
 [ 1571.793998] BTRFS info (device sda3): csum failed ino 309 off
 5767168 csum 645194099 private 3149624654
 [ 1571.794389] BTRFS info (device sda3): csum failed ino 309 off
 4993024 csum 1283121890 private 3720296651


 So my problem still exists. How to fix the block with wrong csum?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to fix/remove csum failed ino error

[Frank Holton]

Hi Anatol,

That certainly does not look good, definitely more than just a bad
space cache. A this point I would strongly suggest that before you try
anything else on the file system that you make sure you have a backup
of everything up there. After you have backed up everything a scrub
may be able to fix some of the corruption or at least tell you which
files are corrupted (the names are printed to the kernel log.) Its
also possible that this will lock up the kernel again so be prepared
for that. Since you are not on raid1 yet it cannot fix those files, it
only reports the ones with checksum errors so you would need to delete
them manually.

as root
btrfs scrub start /mount_point

Another option you can try is to mount with the recovery option.

But before you go any further ensure you have good backups separate
from your BTRFS file system.

Hope some of that helps.



On Thu, Nov 7, 2013 at 11:55 PM, Anatol Pomozov
anatol.pomo...@gmail.com wrote:
 Hi

 I ran btrfsck hoping that it fix the filesystem so 'balance' would not
 crash anymore. But btrfsck itself crashed :(

 # btrfsck --repair /dev/sda3

:(
 enabling repair mode
 Checking filesystem on /dev/sda3
 UUID: 25e6a6fa-fe1f-4be5-a638-eeac948f8c21
 checking extents
 checking fs roots
 root 5 inode 522858 errors 1000
 root 5 inode 1437358 errors 1000
 root 5 inode 1437359 errors 1000
 root 5 inode 1437360 errors 1000
 root 5 inode 1437361 errors 1000
 root 5 inode 1437362 errors 1000
 root 5 inode 1437363 errors 1000
 root 5 inode 1437368 errors 1000
 root 5 inode 1437369 errors 1000
 root 5 inode 1437370 errors 1000
 root 5 inode 1437371 errors 1000
 root 5 inode 1437372 errors 1000
 root 5 inode 1437373 errors 1000
 root 5 inode 1437374 errors 1000
 root 5 inode 1437375 errors 1000
 root 5 inode 1437376 errors 1000
 root 5 inode 1437377 errors 1000
 root 5 inode 1437378 errors 1000
 root 5 inode 1437379 errors 1000
 root 5 inode 1437380 errors 1000
 root 5 inode 1437381 errors 1000
 root 5 inode 1437382 errors 1000
 root 5 inode 1437383 errors 1000
 root 5 inode 1437384 errors 1000
 root 5 inode 1437385 errors 1000
 root 5 inode 1437386 errors 1000
 root 5 inode 1437387 errors 1000
 root 5 inode 1437388 errors 1000
 root 5 inode 1437389 errors 1000
 root 5 inode 1437390 errors 1000
 root 5 inode 1437391 errors 1000
 root 5 inode 1437392 errors 1000
 root 5 inode 1437393 errors 1000
 root 5 inode 1437394 errors 1000
 root 5 inode 1437395 errors 1000
 root 5 inode 1437396 errors 1000
 root 5 inode 1437397 errors 1000
 root 5 inode 1437398 errors 1000
 root 5 inode 1437399 errors 1000
 root 5 inode 1437400 errors 1000
 root 5 inode 5073119 errors 400
 Unable to find block group for 0
 btrfsck: extent-tree.c:284: find_search_start: Assertion `!(1)' failed.
 [1]583 abort (core dumped)  btrfsck --repair /dev/sda3

 On Thu, Nov 7, 2013 at 8:07 PM, Anatol Pomozov anatol.pomo...@gmail.com 
 wrote:
 Hi, Frank

 Thanks for your answer.

 On Thu, Nov 7, 2013 at 8:41 AM, Frank Holton fhol...@gmail.com wrote:
 Hey Anatol,

 I just checked and on my filesystem inode number 362 corresponds to
 part of the free space cache. You can check this yourself by running
 (as root)

 btrfs-debug-tree /dev/sdb | grep (362  -A 3 -B 1

 where /dev/sdb is one of the devices from your filesystem.

 It printed the following for me, note the location key (362
 INODE_ITEM) under the FREE_SPACE key. Yours might be different but if
 you see FREE_SPACE that points to the free space cache.

 item 100 key (362 INODE_ITEM 0) itemoff 21857 itemsize 160
 inode generation 2004 transid 2004 size 262144 block
 group 0 mode 100600 links 1
 item 101 key (362 EXTENT_DATA 0) itemoff 21804 itemsize 53
 extent data disk byte 41903296512 nr 262144
 extent data offset 0 nr 262144 ram 262144
 extent compression 0
 --
 item 148 key (FREE_SPACE UNTYPED 113845993472) itemoff 23807 
 itemsize 41
 location key (362 INODE_ITEM 0)
 cache generation 2004 entries 2 bitmaps 0


 Indeed my case similar to yours

 # btrfs-debug-tree /dev/sda3 | grep (309  -A 3 -B 1

 item 1 key (309 INODE_ITEM 0) itemoff 3675 itemsize 160
 inode generation 190480 transid 190647 size 0 block group 0 mode
 100600 links 1
 item 51 key (FREE_SPACE UNTYPED 56937676800) itemoff 1863 itemsize 41
 location key (309 INODE_ITEM 0)

 So I mounted my filesystem with 'clear_cache' flag:

 # mount -o clear_cache /dev/sda3 mydata/

 mount says:
 /dev/sdc1 on /root/mydata type btrfs (rw,relatime,space_cache,clear_cache)

 dmesg also mentions the cache:

 [  634.991845] device fsid 25e6a6fa-fe1f-4be5-a638-eeac948f8c21 devid
 9 transid 190479 /dev/sda3
 [  634.993431] btrfs: force clearing of disk cache
 [  634.993435] btrfs: disk space caching is enabled
 [  635.046803] btrfs: bdev /dev/sda3 errs: wr 0, rd 0, flush 0,
 corrupt 58481, gen 0


 The I started raid1 rebalance but the error still presents:

 [ 

Re: How to fix/remove csum failed ino error

[Chris Murphy]

What's the kernel and btrfs progs version?

I wish the dmesg errors were more explicit about the nature of checksum errors: 
do the two metadata checksums mismatch each other (one of them matches with 
data), or the metadata checksums match each other but mismatch with data?

Hopefully I'm mistaken, but it looks in this case that the data is actually 
corrupt, not the metadata. In which case repair would only be possible for 
raid1 or raid10 data profiles.

Why the corruption occurred depends on kernel and btrfs-progs versions, and 
what you were doing prior to the corruption so dmesg prior to the corruption 
would be needed and also when trying to mount with the recovery option so it 
might be worth:

dmesg
[note the last time entry]
dmesg -n7
btrfs mount -o recovery dev mp
dmesg

report results since the previously noted last time entry


Chris Murphy--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html