Re: [PATCH v2 0/3] implement zone-aware probing/wiping for zoned btrfs

2021-04-14 Thread Karel Zak
On Wed, Apr 14, 2021 at 10:33:36AM +0900, Naohiro Aota wrote:
> This series implements probing and wiping of the superblock of zoned btrfs.

I have no disk with zones support, but it seems scsi_debug supports
it. Do you have any step by step example how to test with btrfs? If
yes, I will add a test to our test suite.

  Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com



Re: [PATCH v2 3/3] blkid: support zone reset for wipefs

2021-04-14 Thread Karel Zak
On Wed, Apr 14, 2021 at 10:33:39AM +0900, Naohiro Aota wrote:
> +static int is_conventional(blkid_probe pr, uint64_t offset)
> +{
> + struct blk_zone_report *rep = NULL;
> + size_t rep_size;
> + int ret;
> + uint64_t zone_mask;
> +
> + if (!pr->zone_size)
> + return 1;
> +
> + rep_size = sizeof(struct blk_zone_report) + sizeof(struct blk_zone);
> + rep = calloc(1, rep_size);
> + if (!rep)
> + return -1;
> +
> + zone_mask = ~(pr->zone_size - 1);
> + rep->sector = (offset & zone_mask) >> 9;
> + rep->nr_zones = 1;
> + ret = ioctl(blkid_probe_get_fd(pr), BLKREPORTZONE, rep);
> + if (ret) {
> + free(rep);
> + return -1;
> + }

  ret = blkdev_get_zonereport()

:-)

>  /**
>   * blkid_do_wipe:
>   * @pr: prober
> @@ -1267,6 +1310,7 @@ int blkid_do_wipe(blkid_probe pr, int dryrun)
>   const char *off = NULL;
>   size_t len = 0;
>   uint64_t offset, magoff;
> + bool conventional;

BTW, nowhere in libblkid we use "bool". It would be probably better to include
 to blkidP.h.

  Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com



Re: [PATCH v2 2/3] blkid: add magic and probing for zoned btrfs

2021-04-14 Thread Karel Zak
On Wed, Apr 14, 2021 at 10:33:38AM +0900, Naohiro Aota wrote:
> +#define ASSERT(x) assert(x)

Really? ;-)

> +typedef uint64_t u64;
> +typedef uint64_t sector_t;
> +typedef uint8_t u8;

I do not see a reason for u64 and u8 here.

> +
> +#ifdef HAVE_LINUX_BLKZONED_H
> +static int sb_write_pointer(int fd, struct blk_zone *zones, u64 *wp_ret)
> +{
> + bool empty[BTRFS_NR_SB_LOG_ZONES];
> + bool full[BTRFS_NR_SB_LOG_ZONES];
> + sector_t sector;
> +
> + ASSERT(zones[0].type != BLK_ZONE_TYPE_CONVENTIONAL &&
> +zones[1].type != BLK_ZONE_TYPE_CONVENTIONAL);

assert()

 ...
> + for (i = 0; i < BTRFS_NR_SB_LOG_ZONES; i++) {
> + u64 bytenr;
> +
> + bytenr = ((zones[i].start + zones[i].len)
> +<< SECTOR_SHIFT) - BTRFS_SUPER_INFO_SIZE;
> +
> + ret = pread64(fd, buf[i], BTRFS_SUPER_INFO_SIZE,
> +   bytenr);

 please, use  

 ptr = blkid_probe_get_buffer(pr, BTRFS_SUPER_INFO_SIZE, bytenr);

 the library will care about the buffer and reuse it. It's also
 important to keep blkid_do_wipe() usable.

> + if (ret != BTRFS_SUPER_INFO_SIZE)
> + return -EIO;
> + super[i] = (struct btrfs_super_block *)&buf[i];

  super[i] = (struct btrfs_super_block *) ptr;

> + }
> +
> + if (super[0]->generation > super[1]->generation)
> + sector = zones[1].start;
> + else
> + sector = zones[0].start;
> + } else if (!full[0] && (empty[1] || full[1])) {
> + sector = zones[0].wp;
> + } else if (full[0]) {
> + sector = zones[1].wp;
> + } else {
> + return -EUCLEAN;
> + }
> + *wp_ret = sector << SECTOR_SHIFT;
> + return 0;
> +}
> +
> +static int sb_log_offset(blkid_probe pr, uint64_t *bytenr_ret)
> +{
> + uint32_t zone_num = 0;
> + uint32_t zone_size_sector;
> + struct blk_zone_report *rep;
> + struct blk_zone *zones;
> + size_t rep_size;
> + int ret;
> + uint64_t wp;
> +
> + zone_size_sector = pr->zone_size >> SECTOR_SHIFT;
> +
> + rep_size = sizeof(struct blk_zone_report) + sizeof(struct blk_zone) * 2;
> + rep = malloc(rep_size);
> + if (!rep)
> + return -errno;
> +
> + memset(rep, 0, rep_size);
> + rep->sector = zone_num * zone_size_sector;
> + rep->nr_zones = 2;

what about to add to lib/blkdev.c a new function:

   struct blk_zone_report *blkdev_get_zonereport(int fd, uint64 sector, int 
nzones);

and call this function from your sb_log_offset() as well as from 
blkid_do_wipe()?

Anyway, calloc() is better than malloc()+memset().

> + if (zones[0].type == BLK_ZONE_TYPE_CONVENTIONAL) {
> + *bytenr_ret = zones[0].start << SECTOR_SHIFT;
> + ret = 0;
> + goto out;
> + } else if (zones[1].type == BLK_ZONE_TYPE_CONVENTIONAL) {
> + *bytenr_ret = zones[1].start << SECTOR_SHIFT;
> +         ret = 0;
> + goto out;
> + }

what about:

 for (i = 0; i < BTRFS_NR_SB_LOG_ZONES; i++) {
   if (zones[i].type == BLK_ZONE_TYPE_CONVENTIONAL) {
  *bytenr_ret = zones[i].start << SECTOR_SHIFT;
  ret = 0;
  goto out;
   }
 }




 Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com



Re: [PATCH v2 1/3] blkid: implement zone-aware probing

2021-04-14 Thread Karel Zak
On Wed, Apr 14, 2021 at 10:33:37AM +0900, Naohiro Aota wrote:
> --- a/configure.ac
> +++ b/configure.ac
> @@ -302,6 +302,7 @@ AC_CHECK_HEADERS([ \
>   lastlog.h \
>   libutil.h \
>   linux/btrfs.h \
> + linux/blkzoned.h \

unnecessary, there is already AC_CHECK_HEADERS([linux/blkzoned.h]) on
another place.

>   linux/capability.h \
>   linux/cdrom.h \
>   linux/falloc.h \
> diff --git a/libblkid/src/blkidP.h b/libblkid/src/blkidP.h
> index a3fe6748a969..e3a160aa97c0 100644
> --- a/libblkid/src/blkidP.h
> +++ b/libblkid/src/blkidP.h
> @@ -150,6 +150,10 @@ struct blkid_idmag
>   const char  *hoff;  /* hint which contains byte offset to 
> kboff */
>   longkboff;  /* kilobyte offset of superblock */
>   unsigned intsboff;  /* byte offset within superblock */
> +
> + int is_zoned;   /* indicate magic location is 
> calcluated based on zone position  */
> + longzonenum;/* zone number which has superblock */
> + longkboff_inzone;   /* kilobyte offset of superblock in a 
> zone */

It would be better to use 'flags' struct field and

  #define BLKID_FL_ZONED_DEV (1 << 6)

like we use for another stuff.

> diff --git a/libblkid/src/probe.c b/libblkid/src/probe.c
> index a47a8720d4ac..9d180aab5242 100644
> --- a/libblkid/src/probe.c
> +++ b/libblkid/src/probe.c
> @@ -94,6 +94,9 @@
>  #ifdef HAVE_LINUX_CDROM_H
>  #include 
>  #endif
> +#ifdef HAVE_LINUX_BLKZONED_H
> +#include 
> +#endif
>  #ifdef HAVE_SYS_STAT_H
>  #include 
>  #endif
> @@ -897,6 +900,7 @@ int blkid_probe_set_device(blkid_probe pr, int fd,
>   pr->wipe_off = 0;
>   pr->wipe_size = 0;
>   pr->wipe_chain = NULL;
> + pr->zone_size = 0;

you also need to update blkid_clone_probe() function

  Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com



Re: Ideas on unified real-ro mount option across all filesystems

2015-12-17 Thread Karel Zak
On Wed, Dec 16, 2015 at 09:15:59PM -0600, Eric Sandeen wrote:
> I have always interpreted it as simply "no user changes to the filesystem,"
> and that is clearly what the vfs does with the flag...

Yep,

> Given that nothing in the documentation implies that the block device itself
> must remain unchanged on a read-only mount, I don't see any problem which
> needs fixing.  MS_RDONLY rejects user IO; that's all.

I agree, it's FS specific business to interpret the 'ro'.

And it's already enough complicated, we have three levels of "read-only":

 - read-only device (blockdev --setro ioctl)
 - read-only filesystem (mount -o ro)
 - read-only VFS node (mount -o remount,ro,bind /src /dst)

and for example in /proc/self/mountinfo we distinguish between FS "ro"
and VFS "ro" flag:

# grep test /proc/self/mountinfo
185 59 8:5 / /mnt/test ro,relatime shared:32 - ext4 /dev/sda5 rw,data=ordered
   ^^ ^^


BTW, util-linux 2.27 mount(8) man page:

   -r, --read-only
   Mount the filesystem read-only.  A synonym is -o ro.

   Note  that,  depending  on the filesystem type, state and
   kernel behavior, the system may still write to the device.  For
   example, ext3 and ext4 will replay the journal if the
   filesystem is dirty.  To prevent this kind of write access, you
   may want to mount an ext3 or ext4 filesystem with the ro,noload
   mount options or set the  block  device itself to read-only
   mode, see the blockdev(8) command.


 (maybe we need to copy this note to "ro" description too and add hint
 about btrfs too :-)

 Karel


-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] btrfs-progs: Fix partitioned loop devices resolve.

2015-11-10 Thread Karel Zak
On Tue, Nov 10, 2015 at 09:35:22AM +0100, Florian Margaine wrote:
> 
> 
> On 11/09/2015 03:12 PM, Karel Zak wrote:
> > On Mon, Nov 09, 2015 at 02:06:26PM +0100, Florian Margaine wrote:
> >> Instead of using string functions to extract the device name and reading
> >> this file, this patch uses the loop device API through ioctl to get the
> >> correct backing file.
> > 
> > #define LO_NAME_SIZE64
> > 
> > struct loop_info64 {
> > ...
> > uint8_t lo_file_name[LO_NAME_SIZE];
> > };
> > 
> > 
> > The loopdev is based on file descriptor, the lo_file_name[] is hint
> > only and it does not have to match with the real path and the most
> > important problem is that it uses 64-bytes buffer.
> > 
> > For losetup we use LOOP_GET_STATUS64 ioctl as fallback solution only.
> 
> I was thinking that this kind of code could be used, can you confirm
> that this would be fine? Untested code:
> 
> static int resolve_loop_device()
> {
> int ret;
> ret = fopen('/sys/...', 'r');
> if (ret == NULL)
> if (errno == ENOENT)
> return resolve_loop_device_ioctl();
> }
> 
> static int __attribute__((noinline)) resolve_loop_device_ioctl()
> {
> /* use ioctl */
> }
> 
> This would use the normal path most of the time, and use the fallback
> only if necessary. The 64-bytes buffer issue would be mitigated.

Yep, first try /sys/... and when unsuccessful then try ioctl.

losetup example:
https://github.com/karelzak/util-linux/blob/master/lib/loopdev.c#L686

(it's probably too complex, but the basic idea is obvious)

Maybe we need libloop.so to share all these things between various
project :-)

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] btrfs-progs: Fix partitioned loop devices resolve.

2015-11-09 Thread Karel Zak
On Mon, Nov 09, 2015 at 02:06:26PM +0100, Florian Margaine wrote:
> Instead of using string functions to extract the device name and reading
> this file, this patch uses the loop device API through ioctl to get the
> correct backing file.

#define LO_NAME_SIZE64

struct loop_info64 {
...
uint8_t lo_file_name[LO_NAME_SIZE];
};


The loopdev is based on file descriptor, the lo_file_name[] is hint
only and it does not have to match with the real path and the most
important problem is that it uses 64-bytes buffer.

For losetup we use LOOP_GET_STATUS64 ioctl as fallback solution only.

Karel


-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: bad handling of unpartitioned device in sysfs_devno_to_wholedisk() (which breaks mkfs.btrfs)

2015-10-30 Thread Karel Zak
On Fri, Oct 30, 2015 at 09:43:28AM +0800, Tom Yan wrote:
> So I noticed that SSD detection does work on unpartitioned devices in
> mkfs.btrfs somehow:
> https://bugzilla.kernel.org/show_bug.cgi?id=102921
> 
> Later I found out that it breaks at blkid_devno_to_wholedisk() in is_ssd():
> http://git.kernel.org/cgit/linux/kernel/git/kdave/btrfs-progs.git/tree/mkfs.c?h=v4.2.3#n1103
> 
> which Elliot had shown an example with strace:
> https://lists.01.org/pipermail/linux-nvdimm/2015-September/002109.html
> 
> And I think the problem occurs in the sysfs_get_devname() here:
> https://git.kernel.org/cgit/utils/util-linux/util-linux.git/tree/lib/sysfs.c?h=v2.27#n785
> 
> Since sysfs_get_devname() has to call sysfs_readlink() later, which
> output a long full device path in /sys, I don't think we should call
> it directly with the buffer "diskname", which people won't expect that
> it has to be large enough to carry the path in the middle of the
> process. For example in is_sdd(), a char array of size 32 is used
> ("wholedisk").

You're right. The function sysfs_get_devname() is not too elegant as
it uses devname buffer for readlink. Fixed, the bugfix will be in
v2.27.1.

Thanks!

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: mount command now shows subvol and subvolid

2015-09-01 Thread Karel Zak
On Tue, Sep 01, 2015 at 07:17:53AM +, Duncan wrote:
> David Sterba posted on Mon, 31 Aug 2015 15:45:17 +0200 as excerpted:
> 
> > On Sun, Aug 30, 2015 at 11:17:44AM -0600, Chris Murphy wrote:
> >> Does anyone know when this changed? Maybe it's a 4.2 thing... anyway
> >> it's very much welcome!
> > 
> > And another "side effect" is that /proc/self/mountinfo will report the
> > mounted subvolume, even if it was an implicit mount of non-toplevel
> > subvolume.
> 
> Another very nice side effect, likely accidental, but still very nice... 
> is how subvol= ends up being reported for bind-mounts.
> 
> I don't use subvolumes here, but I do use multiple separate btrfs, so 
> subvolid=5, subvol=/ for all of them, would be technically correct, if 
> not particularly useful.  But while the subvolid=5 is indeed constant 
> (and actually useful given the following), subvol= turns out to be rather 
> more helpful than the / that would normally be expected.
> 
> I have a number of bind-mounts.  Formerly mount's output for these wasn't 
> particularly helpful, since I ended up with a number of...
> 
> /dev/sda5 on  ...

 Try findmnt(8) rather than mount(8).

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: i_version vs iversion (Was: Re: [RFC PATCH v2 1/2] Btrfs: add noi_version option to disable MS_I_VERSION)

2015-06-19 Thread Karel Zak
On Thu, Jun 18, 2015 at 04:38:56PM +0200, David Sterba wrote:
> AFAICS, ext4 had added it's own i_version before iversion was added to
> mount:

It is not so unusual that some mount option is introduced as
fs-specific and later re-implemented as generic (another example is
MS_LAZYTIME). The mount(8) cares about the generic options only.

> ext4:
> 
> Commit:  25ec56b518257a56d2ff41a941d288e4b5ff9488
> Commit date: Mon Jan 28 23:58:27 2008 -0500
> Subject: ext4: Add inode version support in ext4
> 
> util-linux:
> 
> Commti:  4fa5e73d16828c94234ba0aeafaec2470f79011c
> Commit date: Thu Nov 27 12:08:44 2008 +0100
> Subject: mount: add i_version support
> 
> I don't know the history, this looks like adding the options was not
> coordinated.

I don't remember this change, but MS_I_VERSION is part of the kernel
API (include/uapi/linux/fs.h) so I guess it's fine to support it in
mount(8).

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in


Re: btrfs-prog: improve build-system by autoconf

2015-02-04 Thread Karel Zak
On Wed, Jan 28, 2015 at 04:38:15PM +0100, David Sterba wrote:
> Rebased on 3.18.3, fixed some minor conflicts.
> 
> * I'm a bit surprised that automake is required for the
>   config.{guess,sub} and install-sh files

 well, it's not required, but autoconf does not provide the scripts.
 You have to maintain the scripts in git three (IMHO not elegant
 solution) or use the scripts from automake.
 
 This problem does not exist if you have fully "autotoolized" project,
 because automake provides automatically all the scripts.

> * my oldish testbox' automake does not support the --print-libdir option
>   (automake-1.11), enterprise distros ship automake of similar age
> 
> * library build test fails, but this may be because I've mismerged
>   something, I'll check again
> 
> Otherwise looks ok and I'll merge it, plus a few fixups to make it build
> for me.
> 
> > https://github.com/karelzak/btrfs-progs/commits/autoconf
> > 
> >  you can merge it (now or later) on command line by:
> > 
> > git pull --log g...@github.com:karelzak/btrfs-progs.git autoconf
> 
> git-pulls are not (yet) established workflow, mailinglist is preferred,
> but it does not hurt to publish branches along.

 OK.

> I've noticed the 'automake' branch that switches to automake. Looking at
> the amount of changes and the result, I'm not quite happy and don't see
> the benefit of automake. An extra layer that only obfuscates the build.

 I don't think the set of changes is so huge, all the changes are
 mostly Makefile.am.
 
 Anyway, why automake:

  - it's way how to standardize build-system, the automake rules are
really easy to read, the Makefile.am files are almost the same in
all projects. 

  - automake+autoconf provides AM_CONDITIONAL(), so you can keep your
makefiles pretty simple and maintain all if-then logic in ./configure
script where you have available shell and huge number of tests.

For example in util-linux we have many internal and external
dependencies, but in makefiles we have only "if BUILD_UTILNAME",
all logic is strictly in ./configure.ac

  - provides large set of targets

  - provides more portable environment
(for example: if you want to distribute man pages, then
dist_man_MANS += foo.8 is enough, you don't have to care about
target directories for the man page section, etc.)

  - forces maintainer to generate better tarballs, after successful
"make diskcheck" you can be sure that your tarballs are really
usable (for example -- do you know how many header files are
missing in the current btrfs-progs Makefile? Do you have a way how
to check it?)

  - manually write and maintain smart build-system is difficult and
result is almost always very complex stuff

 There is only one disadvantage, developers have to learn something
 new. Yes, people hate this thing :-)

 I guess we will see more and more stuff in btrfs-progs, so it's
 better to the change now than later. 
 
 See e2fsprogs (sorry Ted), it's result of "automake only obfuscates", 
 many Makefile.in files, '@' everywhere, nightmare.

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs-prog: improve build-system by autoconf

2015-01-16 Thread Karel Zak
On Wed, Dec 17, 2014 at 03:07:26PM +0100, David Sterba wrote:
> On Fri, Dec 12, 2014 at 01:35:14PM +0100, Karel Zak wrote:
> > This is first step to make btrfs-progs build system more conventional
> > for userspace users and developers. All is implemented by small incremental
> > patches to keep things review-able.
> 
> Thanks. I went through the patches and haven't found major problems. The
> changes are affecting build system and this will need a longer period
> before all distros have a chance to adapt to that, so I'm postponing it
> to 3.19.

 Note that I have rebased the autoconf patches and fixed some issues,
 the latest version is available at

https://github.com/karelzak/btrfs-progs/commits/autoconf

 you can merge it (now or later) on command line by:

git pull --log g...@github.com:karelzak/btrfs-progs.git autoconf


  Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs-prog: improve build-system by autoconf

2014-12-18 Thread Karel Zak
On Wed, Dec 17, 2014 at 03:07:26PM +0100, David Sterba wrote:
> On Fri, Dec 12, 2014 at 01:35:14PM +0100, Karel Zak wrote:
> > This is first step to make btrfs-progs build system more conventional
> > for userspace users and developers. All is implemented by small incremental
> > patches to keep things review-able.
> 
> Thanks. I went through the patches and haven't found major problems. The
> changes are affecting build system and this will need a longer period
> before all distros have a chance to adapt to that, so I'm postponing it
> to 3.19.

Cool, I'll try to prepare next set of patches with automake.

BTW, I have good experience with build-system changes -- downstream
distributions (maintainers) are usually pretty flexible :-)

> > Note that there is also strange unused btrfs_convert_libs, btrfs_image_libs 
> > and
> > btrfs_fragments_libs variables with things like "-lgd -lpng -ljpeg 
> > -lfreetype".
> > I guess it's some legacy, right? I didn't touch these variables as I have no
> > clue about sense of this stuff.
> 
> No, it's part of the macro magic. There are pattern rules that accept
> any source in the form btrfs-something.c and also pick the libraries for
> that from variable btrfs_something_libs:
> 
> btrfs-%: $(objects) $(libs) btrfs-%.o
> @echo "[LD] $@"
> $(Q)$(CC) $(CFLAGS) -o $@ $(objects) $@.o $(LDFLAGS) $(LIBS) 
> $($(subst -,_,$@-libs))
> 
> This is for convenience, if this turns out to be hard to do with in 
> combination
> with autotools, I don't insist on keeping it but it has simplified the 
> Makefile
> significantly.

OK, so -ljpeg in the Makefile is just example, right?


Anyway, for these things is better to introduce extra autoconf
AC_ARG_VAR() variables, keep is empty by default and use it in
Makefile. The advantage is that the variables are documented and
visible by ./configure --help.

For example in util-linux we have many {SUID,DAEMON,SOLIB,...}_CFLAGS
and LDFLAGS for distributions that require extensions like -fPIE,
-Wl,-z,relro etc. The same is possible to do with $LIBS.

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 01/10] btrfs-progs: add ./configure script

2014-12-12 Thread Karel Zak
Add ./autogen.sh script, you have to use it after "git clone/clean" to
generate ./configure from configure.ac.

Modify version.sh to be usable from the configure script.

The patch also renames Makefile to Makefile.in, but does NOT change
anything in the file.

Signed-off-by: Karel Zak 
---
 .gitignore   |  28 ++
 Makefile |  87 +
 Makefile.in  | 314 +++
 autogen.sh   |  58 +++
 configure.ac | 102 +++
 version.sh   |   7 ++
 6 files changed, 557 insertions(+), 39 deletions(-)
 create mode 100644 Makefile.in
 create mode 100755 autogen.sh
 create mode 100644 configure.ac
 mode change 100644 => 100755 version.sh

diff --git a/.gitignore b/.gitignore
index e637b17..beddedb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -39,3 +39,31 @@ libbtrfs.so.0
 libbtrfs.so.0.1
 library-test
 library-test-static
+
+
+aclocal.m4
+autom4te.cache
+compile
+config.cache
+config.guess
+config.h
+config.h.in
+config.log
+config.rpath
+config.status
+config.sub
+config/ltmain.sh
+config/py-compile
+config/test-driver
+configure
+cscope.out
+depcomp
+install-sh
+libtool
+m4/*.m4
+Makefile
+missing
+mkinstalldirs
+stamp-h
+stamp-h.in
+stamp-h1
diff --git a/Makefile b/Makefile
index 4cae30c..95700be 100644
--- a/Makefile
+++ b/Makefile
@@ -1,11 +1,30 @@
 # Export all variables to sub-makes by default
 export
 
-CC = gcc
-LN = ln
-AR = ar
-AM_CFLAGS = -Wall -D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES 
-fno-strict-aliasing -fPIC
-CFLAGS = -g -O1 -fno-strict-aliasing -rdynamic
+CC = gcc -std=gnu99
+LN_S = ln -s
+AR = /usr/bin/ar
+INSTALL = /usr/bin/install -c
+
+# Non-static compilation flags
+CFLAGS = -g -O1 \
+-include config.h -Wall \
+-D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES \
+-fno-strict-aliasing -fPIC \
+-rdynamic
+
+LDFLAGS = 
+
+LIBS = -luuid  -lblkid  -L/usr/lib64 -lz  -lzo2 -lm -L.
+LIBBTRFS_LIBS = $(LIBS)
+
+# Static compilation flags
+STATIC_CFLAGS = $(CFLAGS) -ffunction-sections -fdata-sections
+STATIC_LDFLAGS = -static -Wl,--gc-sections
+STATIC_LIBS = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ \
+ @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ -lpthread
+
+
 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
  root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
  extent-cache.o extent_io.o volumes.o utils.o repair.o \
@@ -23,13 +42,10 @@ libbtrfs_headers = send-stream.h send-utils.h send.h 
rbtree.h btrfs-list.h \
   extent_io.h ioctl.h ctree.h btrfsck.h version.h
 TESTS = fsck-tests.sh convert-tests.sh
 
-INSTALL = install
-prefix ?= /usr/local
-bindir = $(prefix)/bin
-lib_LIBS = -luuid -lblkid -lm -lz -llzo2 -L.
-libdir ?= $(prefix)/lib
-incdir = $(prefix)/include/btrfs
-LIBS = $(lib_LIBS) $(libs_static)
+prefix ?= /usr
+bindir = ${exec_prefix}/bin
+libdir ?= ${exec_prefix}/lib
+incdir = ${prefix}/include/btrfs
 
 ifeq ("$(origin V)", "command line")
   BUILD_VERBOSE = $(V)
@@ -67,7 +83,7 @@ INSTALLDIRS = $(patsubst %,install-%,$(SUBDIRS))
 CLEANDIRS = $(patsubst %,clean-%,$(SUBDIRS))
 
 ifeq ($(DISABLE_BACKTRACE),1)
-AM_CFLAGS += -DBTRFS_DISABLE_BACKTRACE
+CFLAGS += -DBTRFS_DISABLE_BACKTRACE
 endif
 
 ifneq ($(DISABLE_DOCUMENTATION),1)
@@ -87,10 +103,6 @@ static_objects = $(patsubst %.o, %.static.o, $(objects))
 static_cmds_objects = $(patsubst %.o, %.static.o, $(cmds_objects))
 static_libbtrfs_objects = $(patsubst %.o, %.static.o, $(libbtrfs_objects))
 
-# Define static compilation flags
-STATIC_CFLAGS = $(CFLAGS) -ffunction-sections -fdata-sections
-STATIC_LDFLAGS = -static -Wl,--gc-sections
-STATIC_LIBS = $(lib_LIBS) -lpthread
 
 libs_shared = libbtrfs.so.0.1
 libs_static = libbtrfs.a
@@ -118,7 +130,7 @@ ifdef C
 else
check = true
check_echo = true
-   AM_CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
+   CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
 endif
 
 %.o.d: %.c
@@ -126,13 +138,13 @@ endif
 
 .c.o:
@$(check_echo) "[SP] $<"
-   $(Q)$(check) $(AM_CFLAGS) $(CFLAGS) $<
+   $(Q)$(check) $(CFLAGS) $<
@echo "[CC] $@"
-   $(Q)$(CC) $(AM_CFLAGS) $(CFLAGS) -c $<
+   $(Q)$(CC) $(CFLAGS) -c $<
 
 %.static.o: %.c
@echo "[CC] $@"
-   $(Q)$(CC) $(AM_CFLAGS) $(STATIC_CFLAGS) -c $< -o $@
+   $(Q)$(CC) $(STATIC_CFLAGS) -c $< -o $@
 
 all: $(progs) $(BUILDDIRS)
 $(SUBDIRS): $(BUILDDIRS)
@@ -152,13 +164,9 @@ test:
 #
 static: $(progs_static)
 
-version.h:
-   @echo "[SH] $@"
-   $(Q)bash version.sh
-
 $(libs_shared): $(libbtrfs_objects) $(lib_links) send.h
@echo "[LD] $@"
-   $(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(LDFLAGS) $(lib_LIBS) \
+   $(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(LDFLAGS) $(LIBBTRFS_LIBS) \
-shared -Wl,-soname,libbtrfs.so.0 -o libbtrfs.so.0.1
 
 $(

[PATCH 03/10] btrfs-progs: use standard PACKAGE_* macros

2014-12-12 Thread Karel Zak
- use standard PACKAGE_{NAME,VERSION,STRING,URL,...} autoconf macros
  rather than homemade BTRFS_BUILD_VERSION

- don't #include version.h, now the file is necessary for library API only

Note that "btrfs version" returns "btrfs-progs " instead of
the original confusing "btrfs ".

Signed-off-by: Karel Zak 
---
 btrfs-calc-size.c | 1 -
 btrfs-corrupt-block.c | 1 -
 btrfs-debug-tree.c| 5 ++---
 btrfs-find-root.c | 1 -
 btrfs-image.c | 1 -
 btrfs-map-logical.c   | 1 -
 btrfs-select-super.c  | 3 +--
 btrfs-show-super.c| 3 +--
 btrfs-zero-log.c  | 3 +--
 btrfs.c   | 3 +--
 btrfstune.c   | 1 -
 chunk-recover.c   | 1 -
 cmds-check.c  | 3 +--
 cmds-filesystem.c | 5 ++---
 cmds-restore.c| 1 -
 library-test.c| 1 -
 mkfs.c| 9 -
 17 files changed, 13 insertions(+), 30 deletions(-)

diff --git a/btrfs-calc-size.c b/btrfs-calc-size.c
index 3ec8230..2be0d64 100644
--- a/btrfs-calc-size.c
+++ b/btrfs-calc-size.c
@@ -32,7 +32,6 @@
 #include "print-tree.h"
 #include "transaction.h"
 #include "list.h"
-#include "version.h"
 #include "volumes.h"
 #include "utils.h"
 
diff --git a/btrfs-corrupt-block.c b/btrfs-corrupt-block.c
index e993680..ba7358d 100644
--- a/btrfs-corrupt-block.c
+++ b/btrfs-corrupt-block.c
@@ -30,7 +30,6 @@
 #include "print-tree.h"
 #include "transaction.h"
 #include "list.h"
-#include "version.h"
 #include "utils.h"
 
 #define FIELD_BUF_LEN 80
diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index e46500d..4468297 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -26,7 +26,6 @@
 #include "disk-io.h"
 #include "print-tree.h"
 #include "transaction.h"
-#include "version.h"
 #include "utils.h"
 
 static int print_usage(void)
@@ -43,7 +42,7 @@ static int print_usage(void)
 " only\n");
fprintf(stderr,
"\t-t tree_id : print only the tree with the given id\n");
-   fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
+   fprintf(stderr, "%s\n", PACKAGE_STRING);
exit(1);
 }
 
@@ -406,7 +405,7 @@ no_node:
uuidbuf[BTRFS_UUID_UNPARSED_SIZE - 1] = '\0';
uuid_unparse(info->super_copy->fsid, uuidbuf);
printf("uuid %s\n", uuidbuf);
-   printf("%s\n", BTRFS_BUILD_VERSION);
+   printf("%s\n", PACKAGE_STRING);
 close_root:
return close_ctree(root);
 }
diff --git a/btrfs-find-root.c b/btrfs-find-root.c
index b24dddf..571c86a 100644
--- a/btrfs-find-root.c
+++ b/btrfs-find-root.c
@@ -30,7 +30,6 @@
 #include "print-tree.h"
 #include "transaction.h"
 #include "list.h"
-#include "version.h"
 #include "volumes.h"
 #include "utils.h"
 #include "crc32c.h"
diff --git a/btrfs-image.c b/btrfs-image.c
index 1257966..74681ba 100644
--- a/btrfs-image.c
+++ b/btrfs-image.c
@@ -33,7 +33,6 @@
 #include "disk-io.h"
 #include "transaction.h"
 #include "utils.h"
-#include "version.h"
 #include "volumes.h"
 #include "extent_io.h"
 
diff --git a/btrfs-map-logical.c b/btrfs-map-logical.c
index c34484f..fc4a29b 100644
--- a/btrfs-map-logical.c
+++ b/btrfs-map-logical.c
@@ -30,7 +30,6 @@
 #include "print-tree.h"
 #include "transaction.h"
 #include "list.h"
-#include "version.h"
 #include "utils.h"
 
 /* we write the mirror info to stdout unless they are dumping the data
diff --git a/btrfs-select-super.c b/btrfs-select-super.c
index 54ac436..492d38d 100644
--- a/btrfs-select-super.c
+++ b/btrfs-select-super.c
@@ -29,13 +29,12 @@
 #include "print-tree.h"
 #include "transaction.h"
 #include "list.h"
-#include "version.h"
 #include "utils.h"
 
 static void print_usage(void)
 {
fprintf(stderr, "usage: btrfs-select-super -s number dev\n");
-   fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
+   fprintf(stderr, "%s\n", PACKAGE_STRING);
exit(1);
 }
 
diff --git a/btrfs-show-super.c b/btrfs-show-super.c
index 9702eb0..15b9e36 100644
--- a/btrfs-show-super.c
+++ b/btrfs-show-super.c
@@ -33,7 +33,6 @@
 #include "print-tree.h"
 #include "transaction.h"
 #include "list.h"
-#include "version.h"
 #include "utils.h"
 #include "crc32c.h"
 
@@ -51,7 +50,7 @@ static void print_usage(void)
fprintf(stderr, "\t-a : print information of all superblocks\n");
fprintf(stderr, "\t-i  : specify which mirror to print 
out\n");
fprintf(stderr, "\t-F : attempt to dump superblocks with bad magic\n");

[PATCH 08/10] btrfs-progs: clean generated files, make version.h stuff more robust

2014-12-12 Thread Karel Zak
- add rule to generated version.h when any relevant stuff changed
- add rule to clean generated files on "make clean-all"

Signed-off-by: Karel Zak 
---
 Makefile.in | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index 58200ca..df752d3 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -164,6 +164,10 @@ test:
 #
 static: $(progs_static)
 
+version.h: version.sh version.h.in configure.ac
+   @echo "[SH] $@"
+   $(Q)bash ./config.status --silent $@
+
 $(libs_shared): $(libbtrfs_objects) $(lib_links) send.h
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(LDFLAGS) $(LIBBTRFS_LIBS) \
@@ -271,14 +275,15 @@ test-build:
 manpages:
$(Q)$(MAKE) $(MAKEOPTS) -C Documentation
 
-clean-all: clean-doc clean
+
+clean-all: clean clean-doc clean-gen
 
 clean: $(CLEANDIRS)
@echo "Cleaning"
$(Q)rm -f $(progs) cscope.out *.o *.o.d \
  dir-test ioctl-test quick-test send-test library-test 
library-test-static \
  btrfs.static mkfs.btrfs.static \
- version.h $(check_defs) \
+ $(check_defs) \
  $(libs) $(lib_links) \
  $(progs_static) $(progs_extra)
 
@@ -286,6 +291,11 @@ clean-doc:
@echo "Cleaning Documentation"
$(Q)$(MAKE) $(MAKEOPTS) -C Documentation clean
 
+clean-gen:
+   @echo "Cleaning Generated Files"
+   $(Q)rm -f version.h config.status config.cache connfig.log \
+   configure.lineno config.status.lineno Makefile
+
 $(CLEANDIRS):
@echo "Cleaning $(patsubst clean-%,%,$@)"
$(Q)$(MAKE) $(MAKEOPTS) -C $(patsubst clean-%,%,$@) clean
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 10/10] btrfs-progs: add --disable-documentation

2014-12-12 Thread Karel Zak
Signed-off-by: Karel Zak 
---
 Makefile.in  | 1 +
 configure.ac | 9 +
 2 files changed, 10 insertions(+)

diff --git a/Makefile.in b/Makefile.in
index bdd7683..5889224 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -5,6 +5,7 @@ CC = @CC@
 LN_S = @LN_S@
 AR = @AR@
 INSTALL = @INSTALL@
+DISABLE_DOCUMENTATION = @DISABLE_DOCUMENTATION@
 
 # Non-static compilation flags
 CFLAGS = @CFLAGS@ \
diff --git a/configure.ac b/configure.ac
index 290d022..79cb591 100644
--- a/configure.ac
+++ b/configure.ac
@@ -66,6 +66,15 @@ AS_IF([test "x$enable_backtrace" = xno], [
   AC_DEFINE([BTRFS_DISABLE_BACKTRACE], [1], [disable backtrace stuff in 
kerncompat.h ])
 ])
 
+
+AC_ARG_ENABLE([documentation],
+ AS_HELP_STRING([--disable-documentation], [do not build 
domumentation]),
+  [], [enable_documentation=yes]
+)
+AS_IF([test "x$enable_documentation" = xyes], [DISABLE_DOCUMENTATION=0], 
[DISABLE_DOCUMENTATION=1])
+AC_SUBST([DISABLE_DOCUMENTATION])
+
+
 dnl Define _LIBS= and _CFLAGS= by pkg-config
 dnl
 dnl The default PKG_CHECK_MODULES() action-if-not-found is end the
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 07/10] btrfs-progs: cleanup compilation flags usage

2014-12-12 Thread Karel Zak
- define basic default CFLAGS in configure.ac, because:

   * autoconf default is -g -O2, but btrfs uses -g -O1

   * it's better to follow autoconf; standard way to modify
 CFLAGS is to call:  CFLAGS="foo bar" ./configure

- move all flags to one place in Makefile.in

- don't use AM_CFLAGS, the CFLAGS and STATIC_CFLAGS are enough

- don't mix objects and flags in $LIBS, it's more readable to
  add $(libs) to make rules

Signed-off-by: Karel Zak 
---
 Makefile.in  | 67 +++-
 configure.ac |  2 ++
 2 files changed, 41 insertions(+), 28 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index df590ab..58200ca 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -4,10 +4,27 @@ export
 CC = @CC@
 LN_S = @LN_S@
 AR = @AR@
-AM_CFLAGS = -include config.h -Wall \
-   -D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES \
-   -fno-strict-aliasing -fPIC
-CFLAGS = -g -O1 -fno-strict-aliasing -rdynamic
+INSTALL = @INSTALL@
+
+# Non-static compilation flags
+CFLAGS = @CFLAGS@ \
+-include config.h -Wall \
+-D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES \
+-fno-strict-aliasing -fPIC \
+-rdynamic
+
+LDFLAGS = @LDFLAGS@
+
+LIBS = @UUID_LIBS@ @BLKID_LIBS@ @ZLIB_LIBS@ @LZO2_LIBS@ -lm -L.
+LIBBTRFS_LIBS = $(LIBS)
+
+# Static compilation flags
+STATIC_CFLAGS = $(CFLAGS) -ffunction-sections -fdata-sections
+STATIC_LDFLAGS = -static -Wl,--gc-sections
+STATIC_LIBS = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ \
+ @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ -lpthread
+
+
 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
  root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
  extent-cache.o extent_io.o volumes.o utils.o repair.o \
@@ -25,13 +42,10 @@ libbtrfs_headers = send-stream.h send-utils.h send.h 
rbtree.h btrfs-list.h \
   extent_io.h ioctl.h ctree.h btrfsck.h version.h
 TESTS = fsck-tests.sh convert-tests.sh
 
-INSTALL = @INSTALL@
 prefix ?= @prefix@
 bindir = @bindir@
-lib_LIBS = @UUID_LIBS@ @BLKID_LIBS@ @ZLIB_LIBS@ @LZO2_LIBS@ -m -L.
 libdir ?= @libdir@
 incdir = @includedir@/btrfs
-LIBS = $(lib_LIBS) $(libs_static)
 
 ifeq ("$(origin V)", "command line")
   BUILD_VERBOSE = $(V)
@@ -69,7 +83,7 @@ INSTALLDIRS = $(patsubst %,install-%,$(SUBDIRS))
 CLEANDIRS = $(patsubst %,clean-%,$(SUBDIRS))
 
 ifeq ($(DISABLE_BACKTRACE),1)
-AM_CFLAGS += -DBTRFS_DISABLE_BACKTRACE
+CFLAGS += -DBTRFS_DISABLE_BACKTRACE
 endif
 
 ifneq ($(DISABLE_DOCUMENTATION),1)
@@ -89,10 +103,6 @@ static_objects = $(patsubst %.o, %.static.o, $(objects))
 static_cmds_objects = $(patsubst %.o, %.static.o, $(cmds_objects))
 static_libbtrfs_objects = $(patsubst %.o, %.static.o, $(libbtrfs_objects))
 
-# Define static compilation flags
-STATIC_CFLAGS = $(CFLAGS) -ffunction-sections -fdata-sections
-STATIC_LDFLAGS = -static -Wl,--gc-sections
-STATIC_LIBS = $(lib_LIBS) -lpthread
 
 libs_shared = libbtrfs.so.0.1
 libs_static = libbtrfs.a
@@ -120,7 +130,7 @@ ifdef C
 else
check = true
check_echo = true
-   AM_CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
+   CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
 endif
 
 %.o.d: %.c
@@ -128,13 +138,13 @@ endif
 
 .c.o:
@$(check_echo) "[SP] $<"
-   $(Q)$(check) $(AM_CFLAGS) $(CFLAGS) $<
+   $(Q)$(check) $(CFLAGS) $<
@echo "[CC] $@"
-   $(Q)$(CC) $(AM_CFLAGS) $(CFLAGS) -c $<
+   $(Q)$(CC) $(CFLAGS) -c $<
 
 %.static.o: %.c
@echo "[CC] $@"
-   $(Q)$(CC) $(AM_CFLAGS) $(STATIC_CFLAGS) -c $< -o $@
+   $(Q)$(CC) $(STATIC_CFLAGS) -c $< -o $@
 
 all: $(progs) $(BUILDDIRS)
 $(SUBDIRS): $(BUILDDIRS)
@@ -156,7 +166,7 @@ static: $(progs_static)
 
 $(libs_shared): $(libbtrfs_objects) $(lib_links) send.h
@echo "[LD] $@"
-   $(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(LDFLAGS) $(lib_LIBS) \
+   $(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(LDFLAGS) $(LIBBTRFS_LIBS) \
-shared -Wl,-soname,libbtrfs.so.0 -o libbtrfs.so.0.1
 
 $(libs_static): $(libbtrfs_objects)
@@ -186,12 +196,13 @@ btrfs-%.static: $(static_objects) btrfs-%.static.o 
$(static_libbtrfs_objects)
 
 btrfs-%: $(objects) $(libs) btrfs-%.o
@echo "[LD] $@"
-   $(Q)$(CC) $(CFLAGS) -o $@ $(objects) $@.o $(LDFLAGS) $(LIBS) $($(subst 
-,_,$@-libs))
+   $(Q)$(CC) $(CFLAGS) -o $@ $(objects) $@.o $(libs) \
+   $(LDFLAGS) $(LIBS) $($(subst -,_,$@-libs))
 
 btrfs: $(objects) btrfs.o help.o $(cmds_objects) $(libs)
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) -o btrfs btrfs.o help.o $(cmds_objects) \
-   $(objects) $(LDFLAGS) $(LIBS) -lpthread
+   $(objects) $(libs) $(LDFLAGS) $(LIBS) -lpthread
 
 btrfs.static: $(static_objects) btrfs.static.o help.static.o 
$(static_cmds_objects) $(static_libbtrfs_objects)

[PATCH 05/10] btrfs-progs: check for build programs in ./configure

2014-12-12 Thread Karel Zak
Signed-off-by: Karel Zak 
---
 Makefile.in  | 12 ++--
 configure.ac |  2 ++
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index dad1685..17eea58 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -1,9 +1,9 @@
 # Export all variables to sub-makes by default
 export
 
-CC = gcc
-LN = ln
-AR = ar
+CC = @CC@
+LN_S = @LN_S@
+AR = @AR@
 AM_CFLAGS = -include config.h -Wall \
-D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES \
-fno-strict-aliasing -fPIC
@@ -25,7 +25,7 @@ libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h 
btrfs-list.h \
   extent_io.h ioctl.h ctree.h btrfsck.h version.h
 TESTS = fsck-tests.sh convert-tests.sh
 
-INSTALL = install
+INSTALL = @INSTALL@
 prefix ?= /usr/local
 bindir = $(prefix)/bin
 lib_LIBS = -luuid -lblkid -lm -lz -llzo2 -L.
@@ -165,8 +165,8 @@ $(libs_static): $(libbtrfs_objects)
 
 $(lib_links):
@echo "[LN] $@"
-   $(Q)$(LN) -sf libbtrfs.so.0.1 libbtrfs.so.0
-   $(Q)$(LN) -sf libbtrfs.so.0.1 libbtrfs.so
+   $(Q)$(LN_S) -f libbtrfs.so.0.1 libbtrfs.so.0
+   $(Q)$(LN_S) -f libbtrfs.so.0.1 libbtrfs.so
 
 # keep intermediate files from the below implicit rules around
 .PRECIOUS: $(addsuffix .o,$(progs))
diff --git a/configure.ac b/configure.ac
index 937d50f..662d9ff 100644
--- a/configure.ac
+++ b/configure.ac
@@ -27,6 +27,8 @@ AC_C_BIGENDIAN
 AC_SYS_LARGEFILE
 
 AC_PROG_INSTALL
+AC_PROG_LN_S
+AC_PATH_PROG([AR], [ar])
 
 AC_CHECK_FUNCS([openat], [],
[AC_MSG_ERROR([cannot find openat() function])])
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 09/10] btrfs-progs: add --disable-backtrace

2014-12-12 Thread Karel Zak
It's better to use ./configure than manually edit Makefile.

Signed-off-by: Karel Zak 
---
 Makefile.in  |  4 
 configure.ac | 10 ++
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index df752d3..bdd7683 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -82,10 +82,6 @@ BUILDDIRS = $(patsubst %,build-%,$(SUBDIRS))
 INSTALLDIRS = $(patsubst %,install-%,$(SUBDIRS))
 CLEANDIRS = $(patsubst %,clean-%,$(SUBDIRS))
 
-ifeq ($(DISABLE_BACKTRACE),1)
-CFLAGS += -DBTRFS_DISABLE_BACKTRACE
-endif
-
 ifneq ($(DISABLE_DOCUMENTATION),1)
 BUILDDIRS += build-Documentation
 INSTALLDIRS += install-Documentation
diff --git a/configure.ac b/configure.ac
index f6adefb..290d022 100644
--- a/configure.ac
+++ b/configure.ac
@@ -56,6 +56,16 @@ AC_DEFUN([PKG_STATIC], [
   fi
 ])
 
+
+AC_ARG_ENABLE([backtrace],
+  AS_HELP_STRING([--disable-backtrace], [disable btrfs backtrace]),
+  [], [enable_backtrace=yes]
+)
+
+AS_IF([test "x$enable_backtrace" = xno], [
+  AC_DEFINE([BTRFS_DISABLE_BACKTRACE], [1], [disable backtrace stuff in 
kerncompat.h ])
+])
+
 dnl Define _LIBS= and _CFLAGS= by pkg-config
 dnl
 dnl The default PKG_CHECK_MODULES() action-if-not-found is end the
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 04/10] btrfs-progs: use ./configure to generate version.h

2014-12-12 Thread Karel Zak
The original homemade solution is unnecessary, autotools provides better
infrastructure to generate files.

Signed-off-by: Karel Zak 
---
 Makefile.in  |  4 
 configure.ac |  9 +
 version.h.in | 11 +++
 version.sh   | 30 +++---
 4 files changed, 23 insertions(+), 31 deletions(-)
 create mode 100644 version.h.in

diff --git a/Makefile.in b/Makefile.in
index 0dd83ea..dad1685 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -154,10 +154,6 @@ test:
 #
 static: $(progs_static)
 
-version.h:
-   @echo "[SH] $@"
-   $(Q)bash version.sh
-
 $(libs_shared): $(libbtrfs_objects) $(lib_links) send.h
@echo "[LD] $@"
$(Q)$(CC) $(CFLAGS) $(libbtrfs_objects) $(LDFLAGS) $(lib_LIBS) \
diff --git a/configure.ac b/configure.ac
index 7a6c264..937d50f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3,6 +3,10 @@ AC_INIT([btrfs-progs],
[linux-btrfs@vger.kernel.org],,
[http://btrfs.wiki.kernel.org])
 
+dnl library version
+LIBBTRFS_MAJOR=0
+LIBBTRFS_MINOR=1
+LIBBTRFS_PATCHLEVEL=1
 
 AC_PREREQ([2.60])
 
@@ -74,11 +78,16 @@ AC_SUBST([LZO2_LIBS_STATIC])
 AC_SUBST([LZO2_CFLAGS])
 
 
+dnl library stuff
+AC_SUBST([LIBBTRFS_MAJOR])
+AC_SUBST([LIBBTRFS_MINOR])
+AC_SUBST([LIBBTRFS_PATCHLEVEL])
 
 AC_CONFIG_HEADERS([config.h])
 
 AC_CONFIG_FILES([
 Makefile
+version.h
 ])
 
 AC_OUTPUT
diff --git a/version.h.in b/version.h.in
new file mode 100644
index 000..012d265
--- /dev/null
+++ b/version.h.in
@@ -0,0 +1,11 @@
+#ifndef __LIBBTRFS_VERSION_H__
+#define __LIBBTRFS_VERSION_H__
+
+#define BTRFS_LIB_MAJOR@LIBBTRFS_MAJOR@
+#define BTRFS_LIB_MINOR@LIBBTRFS_MINOR@
+#define BTRFS_LIB_PATCHLEVEL   @LIBBTRFS_PATCHLEVEL@
+
+#define BTRFS_LIB_VERSION ( BTRFS_LIB_MAJOR * 1 + \
+BTRFS_LIB_MINOR * 100 + \
+BTRFS_LIB_PATCHLEVEL )
+#endif
diff --git a/version.sh b/version.sh
index 456853c..42b47c4 100755
--- a/version.sh
+++ b/version.sh
@@ -9,9 +9,6 @@
 v="v3.17.3"
 
 opt=$1
-lib_major=0
-lib_minor=1
-lib_patchlevel=1
 
 which git &> /dev/null
 if [ $? == 0 -a -d .git ]; then
@@ -32,30 +29,9 @@ fi
 if [ "$opt" = "--configure" ]; then
# Omit the trailing newline, so that m4_esyscmd can use the result 
directly.
echo "$v" | tr -d '\n'
-   exit 0
+else
+   echo "$v"
 fi
 
-echo "/* NOTE: this file is autogenerated by version.sh, do not edit */" > 
.build-version.h
-echo "#ifndef __BUILD_VERSION" >> .build-version.h
-echo >> .build-version.h
-echo "#define __BUILD_VERSION" >> .build-version.h
-echo >> .build-version.h
-echo "#define BTRFS_LIB_MAJOR $lib_major" >> .build-version.h
-echo "#define BTRFS_LIB_MINOR $lib_minor" >> .build-version.h
-echo "#define BTRFS_LIB_PATCHLEVEL $lib_patchlevel" >> .build-version.h
-echo >> .build-version.h
-echo "#define BTRFS_LIB_VERSION ( BTRFS_LIB_MAJOR * 1 + \\" >> 
.build-version.h
-echo "BTRFS_LIB_MINOR * 100 + \\" >> 
.build-version.h
-echo "BTRFS_LIB_PATCHLEVEL )" >> .build-version.h
-echo >> .build-version.h
-echo "#define BTRFS_BUILD_VERSION \"Btrfs $v\"" >> .build-version.h
-echo "#endif" >> .build-version.h
+exit 0
 
-diff -q version.h .build-version.h >& /dev/null
-
-if [ $? == 0 ]; then
-rm .build-version.h
-exit 0
-fi
-
-mv .build-version.h version.h
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 02/10] btrfs-progs: use config.h

2014-12-12 Thread Karel Zak
- the header file is generated by ./configure, the standard autotools
  way is to use -include config.h on compiler command line rather than
  include the file directly from code

- remove _GNU_SOURCE from code, the macros is already defined in config.h
  by AC_USE_SYSTEM_EXTENSIONS autoconf macro

Signed-off-by: Karel Zak 
---
 Makefile.in   | 4 +++-
 btrfs-calc-size.c | 1 -
 btrfs-convert.c   | 1 -
 btrfs-corrupt-block.c | 2 +-
 btrfs-find-root.c | 2 +-
 btrfs-fragments.c | 1 -
 btrfs-image.c | 2 +-
 btrfs-list.c  | 1 -
 btrfs-map-logical.c   | 2 +-
 btrfs-select-super.c  | 2 +-
 btrfs-show-super.c| 2 +-
 btrfs-zero-log.c  | 2 +-
 btrfs.c   | 1 -
 btrfstune.c   | 2 +-
 chunk-recover.c   | 1 -
 cmds-check.c  | 2 +-
 cmds-receive.c| 1 -
 cmds-restore.c| 1 -
 cmds-send.c   | 2 --
 disk-io.c | 2 +-
 mkfs.c| 1 -
 send-test.c   | 2 --
 super-recover.c   | 1 -
 utils-lib.c   | 2 --
 utils.c   | 2 +-
 25 files changed, 14 insertions(+), 28 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index 4cae30c..0dd83ea 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -4,7 +4,9 @@ export
 CC = gcc
 LN = ln
 AR = ar
-AM_CFLAGS = -Wall -D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES 
-fno-strict-aliasing -fPIC
+AM_CFLAGS = -include config.h -Wall \
+   -D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES \
+   -fno-strict-aliasing -fPIC
 CFLAGS = -g -O1 -fno-strict-aliasing -rdynamic
 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
  root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
diff --git a/btrfs-calc-size.c b/btrfs-calc-size.c
index 50c..3ec8230 100644
--- a/btrfs-calc-size.c
+++ b/btrfs-calc-size.c
@@ -17,7 +17,6 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
 #include 
 #include 
 #include 
diff --git a/btrfs-convert.c b/btrfs-convert.c
index 02c5e94..c88acc1 100644
--- a/btrfs-convert.c
+++ b/btrfs-convert.c
@@ -17,7 +17,6 @@
  */
 
 #define _XOPEN_SOURCE 600
-#define _GNU_SOURCE 1
 
 #include "kerncompat.h"
 
diff --git a/btrfs-corrupt-block.c b/btrfs-corrupt-block.c
index af9ae4d..e993680 100644
--- a/btrfs-corrupt-block.c
+++ b/btrfs-corrupt-block.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/btrfs-find-root.c b/btrfs-find-root.c
index 6fa61cc..b24dddf 100644
--- a/btrfs-find-root.c
+++ b/btrfs-find-root.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/btrfs-fragments.c b/btrfs-fragments.c
index d03c2c3..ca45686 100644
--- a/btrfs-fragments.c
+++ b/btrfs-fragments.c
@@ -14,7 +14,6 @@
  * Boston, MA 021110-1307, USA.
  */
 
-#define _GNU_SOURCE
 #include 
 #include 
 #include 
diff --git a/btrfs-image.c b/btrfs-image.c
index cb17f16..1257966 100644
--- a/btrfs-image.c
+++ b/btrfs-image.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/btrfs-list.c b/btrfs-list.c
index 50edcf4..3e29cf8 100644
--- a/btrfs-list.c
+++ b/btrfs-list.c
@@ -16,7 +16,6 @@
  * Boston, MA 021110-1307, USA.
  */
 
-#define _GNU_SOURCE
 #include 
 #include 
 #include "ioctl.h"
diff --git a/btrfs-map-logical.c b/btrfs-map-logical.c
index 47d1104..c34484f 100644
--- a/btrfs-map-logical.c
+++ b/btrfs-map-logical.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/btrfs-select-super.c b/btrfs-select-super.c
index 6231d42..54ac436 100644
--- a/btrfs-select-super.c
+++ b/btrfs-select-super.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/btrfs-show-super.c b/btrfs-show-super.c
index 2b48f44..9702eb0 100644
--- a/btrfs-show-super.c
+++ b/btrfs-show-super.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/btrfs-zero-log.c b/btrfs-zero-log.c
index 4154175..411fae3 100644
--- a/btrfs-zero-log.c
+++ b/btrfs-zero-log.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/btrfs.c b/btrfs.c
index e83349c..2451885 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -14,7 +14,6 @@
  * Boston, MA 021110-1307, USA.
  */
 
-#define _GNU_SOURCE
 #include 
 #include 
 #include 
diff --git a/btrfstune.c b/btrfstune.c
index 050418a..899a721 100644
--- a/btrfstune.c
+++ b/btrfstune.c
@@ -17,7 +17,7 @@
  */
 
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE 1
+
 #include 
 #include 
 #include 
diff --git a/chunk-recover.c b/chunk-recover.c
index 6f43066..688a7d7 100644
--- a/chunk-recover.c
+++ b/chunk-recover.c
@@ -16,7 +16,6 @@
  * Boston, MA 021110-1307, USA.
  */
 #define _XOPEN_SOURCE 500
-#define _GNU_SOURCE
 
 #incl

[PATCH 06/10] btrfs-progs: use paths and $*_LIBS from ./configure

2014-12-12 Thread Karel Zak
Signed-off-by: Karel Zak 
---
 Makefile.in | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index 17eea58..df590ab 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -26,11 +26,11 @@ libbtrfs_headers = send-stream.h send-utils.h send.h 
rbtree.h btrfs-list.h \
 TESTS = fsck-tests.sh convert-tests.sh
 
 INSTALL = @INSTALL@
-prefix ?= /usr/local
-bindir = $(prefix)/bin
-lib_LIBS = -luuid -lblkid -lm -lz -llzo2 -L.
-libdir ?= $(prefix)/lib
-incdir = $(prefix)/include/btrfs
+prefix ?= @prefix@
+bindir = @bindir@
+lib_LIBS = @UUID_LIBS@ @BLKID_LIBS@ @ZLIB_LIBS@ @LZO2_LIBS@ -m -L.
+libdir ?= @libdir@
+incdir = @includedir@/btrfs
 LIBS = $(lib_LIBS) $(libs_static)
 
 ifeq ("$(origin V)", "command line")
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


btrfs-prog: improve build-system by autoconf

2014-12-12 Thread Karel Zak
This is first step to make btrfs-progs build system more conventional
for userspace users and developers. All is implemented by small incremental
patches to keep things review-able.

The Makefile targets and rules are no changed, things like V=1 (verbose), C=1
(sparse) static builds, etc. still work as expected. All the changes are mostly
about $LIBS, $CFLAGS and proper libraries (uuid, blkid, lzo2, ..) detection.

Note that there is also strange unused btrfs_convert_libs, btrfs_image_libs and
btrfs_fragments_libs variables with things like "-lgd -lpng -ljpeg -lfreetype".
I guess it's some legacy, right? I didn't touch these variables as I have no
clue about sense of this stuff.


[PATCH 01/10] btrfs-progs: add ./configure script
[PATCH 02/10] btrfs-progs: use config.h
[PATCH 03/10] btrfs-progs: use standard PACKAGE_* macros
[PATCH 04/10] btrfs-progs: use ./configure to generate version.h
[PATCH 05/10] btrfs-progs: check for build programs in ./configure
[PATCH 06/10] btrfs-progs: use paths and $*_LIBS from ./configure
[PATCH 07/10] btrfs-progs: cleanup compilation flags usage
[PATCH 08/10] btrfs-progs: clean generated files, make version.h
[PATCH 09/10] btrfs-progs: add --disable-backtrace
[PATCH 10/10] btrfs-progs: add --disable-documentation

The next possible step is automake, but I'd like merge ./configure stuff first.

Karel


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] btrfs-progs: fix typedef

2014-12-12 Thread Karel Zak
Signed-off-by: Karel Zak 
---
 kerncompat.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kerncompat.h b/kerncompat.h
index 8afadc8..5c1cca9 100644
--- a/kerncompat.h
+++ b/kerncompat.h
@@ -123,7 +123,7 @@ typedef unsigned long long u64;
 typedef unsigned char u8;
 typedef unsigned short u16;
 typedef long long s64;
-typedef int s32
+typedef int s32;
 #endif
 
 
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/2 v4] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-11-28 Thread Karel Zak
On Fri, Nov 14, 2014 at 05:51:27PM +0100, David Sterba wrote:
> On Tue, Nov 11, 2014 at 12:47:35PM +0100, Karel Zak wrote:
> >  What I see critical is missing ./configure, because it's pretty ugly
> >  to add hardcoded dependencies (e.g. libudev), there is also no checks
> >  for another libs, Makefile does not care about place where libs are
> >  installed, header files,  etc. etc.
> 
> It does, prefix and libdir are set conditionally, DESTDIR works.
> 
> >  Is there any fundamental problem with autoconf? If no, then I'm ready
> >  to send patches with some autotools stuff. Comments?
> 
> Yeah the build dependencies checks would be nice, there's no problem
> with autoconf.

 OK, I'll try to prepare something next week.

> The Makefile has been manualy crafted and supports some macro magic to
> build several binaries from one rule, static targets, quiet/verbose
> build. I want to preserve all of this so transition to automake may take
> time (or may not happen in the end).

 Just note, it's fine to expect (require) some build-system features,
 but IMHO it's bad idea to think about build system as about stable
 and always backwardly compatible interface (./configure options,
 Makefile vars, etc).

 For example for util-linux we have changed many many things in last
 (~7) years without negative feedback from downstream maintainers or
 users. IMHO more important is to follow usual conventions than assume
 that my "make FOO=bar" will work forever.

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/2] btrfs-progs: use udev to scan for devices (v2)

2014-11-11 Thread Karel Zak
Currently btrfs uses libblkid to scan for block devices with BTRFS.
The disadvantage is that this method is expensive. The udevd maintains
information about all block devices in udev db and all devices are
probed by udevd (it's linked with libblkid). We don't have to duplicate
this thing in btrfs.

This patch links btrfs with libudev, the original blkid based method
is used as fallback (for systems without udev, etc.)

Note that btrfs-progs project does not use autoconf, so dependence on
libudev is hardcoded to Makefile for now. It would be probably better
to add ./configure script to the project and make dependence on
libudev optional.

udev:
# time ./btrfs device scan
Scanning for Btrfs filesystems
real0m0.012s
user0m0.004s
sys 0m0.003s

blkid:
# time ./btrfs device scan
Scanning for Btrfs filesystems
real0m0.076s
user0m0.008s
sys 0m0.024s

V2:
- use udev_enumerate_add_match_property()

Signed-off-by: Karel Zak 
---
 Makefile |   2 +-
 utils.c  | 104 ---
 2 files changed, 81 insertions(+), 25 deletions(-)

diff --git a/Makefile b/Makefile
index 203597c..768c5e0 100644
--- a/Makefile
+++ b/Makefile
@@ -26,7 +26,7 @@ TESTS = fsck-tests.sh convert-tests.sh
 INSTALL = install
 prefix ?= /usr/local
 bindir = $(prefix)/bin
-lib_LIBS = -luuid -lblkid -lm -lz -llzo2 -L.
+lib_LIBS = -ludev -luuid -lblkid -lm -lz -llzo2 -L.
 libdir ?= $(prefix)/lib
 incdir = $(prefix)/include/btrfs
 LIBS = $(lib_LIBS) $(libs_static)
diff --git a/utils.c b/utils.c
index cc33cfc..889 100644
--- a/utils.c
+++ b/utils.c
@@ -38,6 +38,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "kerncompat.h"
 #include "radix-tree.h"
 #include "ctree.h"
@@ -2181,20 +2182,34 @@ int test_dev_for_mkfs(char *file, int force_overwrite, 
char *estr)
return 0;
 }
 
-int btrfs_scan_devices()
+static int scan_device(const char *path)
+{
+   int rc;
+   int fd = open(path, O_RDONLY);
+   struct btrfs_fs_devices *devs = NULL;
+   u64 ndevs;
+
+   if (fd < 0) {
+   printf("ERROR: could not open %s\n", path);
+   return 1;
+   }
+
+   rc = btrfs_scan_one_device(fd, path, &devs, &ndevs, 
BTRFS_SUPER_INFO_OFFSET, 0);
+   if (rc)
+   printf("ERROR: could not scan %s\n", path);
+
+   close(fd);
+   return rc;
+}
+
+
+static int btrfs_scan_devices_by_blkid(void)
 {
-   int fd = -1;
-   int ret;
-   u64 num_devices;
-   struct btrfs_fs_devices *tmp_devices;
blkid_dev_iterate iter = NULL;
blkid_dev dev = NULL;
blkid_cache cache = NULL;
char path[PATH_MAX];
 
-   if (btrfs_scan_done)
-   return 0;
-
if (blkid_get_cache(&cache, 0) < 0) {
printf("ERROR: lblkid cache get failed\n");
return 1;
@@ -2209,29 +2224,70 @@ int btrfs_scan_devices()
/* if we are here its definitely a btrfs disk*/
strncpy(path, blkid_dev_devname(dev), PATH_MAX);
 
-   fd = open(path, O_RDONLY);
-   if (fd < 0) {
-   printf("ERROR: could not open %s\n", path);
-   continue;
-   }
-   ret = btrfs_scan_one_device(fd, path, &tmp_devices,
-   &num_devices, BTRFS_SUPER_INFO_OFFSET, 0);
-   if (ret) {
-   printf("ERROR: could not scan %s\n", path);
-   close (fd);
-   continue;
-   }
-
-   close(fd);
+   scan_device(path);
}
blkid_dev_iterate_end(iter);
blkid_put_cache(cache);
 
-   btrfs_scan_done = 1;
-
return 0;
 }
 
+static int btrfs_scan_devices_by_udev(void)
+{
+   int rc = 1;
+   struct udev *udev = NULL;
+   struct udev_enumerate *en = NULL;
+   struct udev_list_entry *ent, *devs;
+
+   udev = udev_new();
+   if (!udev)
+   goto done;
+   en = udev_enumerate_new(udev);
+   if (!en)
+   goto done;
+
+   udev_enumerate_add_match_subsystem(en, "block");
+   udev_enumerate_add_match_property(en, "ID_FS_TYPE", "btrfs");
+   if (udev_enumerate_scan_devices(en) != 0)
+   goto done;
+   devs = udev_enumerate_get_list_entry(en);
+   if (!devs)
+   goto done;
+
+   udev_list_entry_foreach(ent, devs) {
+   struct udev_device *dev = udev_device_new_from_syspath(udev,
+   udev_list_entry_get_name(ent));
+   if (dev) {
+   const char *path = udev_device_get_devnode(dev);
+   if (path)
+

Re: [PATCH 2/2 v4] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-11-11 Thread Karel Zak
On Fri, Oct 31, 2014 at 07:04:58PM +0800, Anand Jain wrote:
> 
> 
> 
> 
> On 31/10/2014 17:08, Karel Zak wrote:
> >On Fri, Oct 31, 2014 at 12:11:20PM +0800, Anand Jain wrote:
> >>btrfs_scan_lblikd() is called by most the device related command functions.
> >>And btrfs_scan_lblkid() is most expensive function and it becomes more 
> >>expensive
> >>as number of devices in the system increase. Further some threads call this
> >
> >wouldn't be possible to ask udev rather than scan all devices? I
> >understand than in some cases it's necessary to have robust and
> >independent solution, but for usual use-cases it would be less
> >expensive to read the info from udev where we already keep track about
> >all block devices and where we call libblkid.
> >
> >It would be possible to implement it as optional feature (#ifdev 
> >HAVE_LIBUDEV),
> >the library API is very easy to use.
> >
> >(For example lsblk uses libblkid as fallback, the default is udev).
> >
> > Karel
> >
> >
> 
> I might be missing something, correct me if wrong. Is there any udev API
> which gives me a list of devices which hold btrfs ? I just browsed
> through there isn't any.

 See  udev_enumerate_* API, nice example:
 http://www.signal11.us/oss/udev/

 Anyway, I have sent patches that implement this feature to btrfs-progs. 

 What I see critical is missing ./configure, because it's pretty ugly
 to add hardcoded dependencies (e.g. libudev), there is also no checks
 for another libs, Makefile does not care about place where libs are
 installed, header files,  etc. etc.

 Is there any fundamental problem with autoconf? If no, then I'm ready
 to send patches with some autotools stuff. Comments?

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/2] btrfs-progs: use udev to scan for devices

2014-11-11 Thread Karel Zak
Currently btrfs uses libblkid to scan for block devices with BTRFS.
The disadvantage is that this method is expensive. The udevd maintains
information about all block devices in udev db and all devices are
probed by udevd (it's linked with libblkid). We don't have to duplicate
this thing in btrfs.

This patch links btrfs with libudev, the original blkid based method
is used as fallback (for systems without udev, etc.)

Note that btrfs-progs project does not use autoconf, so dependence on
libudev is hardcoded to Makefile for now. It would be probably better
to add ./configure script to the project and make dependence on
libudev optional.

udev:
# time ./btrfs device scan
Scanning for Btrfs filesystems
real0m0.012s
user0m0.004s
sys 0m0.003s

blkid:
# time ./btrfs device scan
Scanning for Btrfs filesystems
real0m0.076s
user0m0.008s
sys 0m0.024s

Signed-off-by: Karel Zak 
---
 Makefile |   2 +-
 utils.c  | 107 +--
 2 files changed, 84 insertions(+), 25 deletions(-)

diff --git a/Makefile b/Makefile
index 203597c..768c5e0 100644
--- a/Makefile
+++ b/Makefile
@@ -26,7 +26,7 @@ TESTS = fsck-tests.sh convert-tests.sh
 INSTALL = install
 prefix ?= /usr/local
 bindir = $(prefix)/bin
-lib_LIBS = -luuid -lblkid -lm -lz -llzo2 -L.
+lib_LIBS = -ludev -luuid -lblkid -lm -lz -llzo2 -L.
 libdir ?= $(prefix)/lib
 incdir = $(prefix)/include/btrfs
 LIBS = $(lib_LIBS) $(libs_static)
diff --git a/utils.c b/utils.c
index cc33cfc..5c71336 100644
--- a/utils.c
+++ b/utils.c
@@ -38,6 +38,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "kerncompat.h"
 #include "radix-tree.h"
 #include "ctree.h"
@@ -2181,20 +2182,34 @@ int test_dev_for_mkfs(char *file, int force_overwrite, 
char *estr)
return 0;
 }
 
-int btrfs_scan_devices()
+static int scan_device(const char *path)
+{
+   int rc;
+   int fd = open(path, O_RDONLY);
+   struct btrfs_fs_devices *devs = NULL;
+   u64 ndevs;
+
+   if (fd < 0) {
+   printf("ERROR: could not open %s\n", path);
+   return 1;
+   }
+
+   rc = btrfs_scan_one_device(fd, path, &devs, &ndevs, 
BTRFS_SUPER_INFO_OFFSET, 0);
+   if (rc)
+   printf("ERROR: could not scan %s\n", path);
+
+   close(fd);
+   return rc;
+}
+
+
+static int btrfs_scan_devices_by_blkid(void)
 {
-   int fd = -1;
-   int ret;
-   u64 num_devices;
-   struct btrfs_fs_devices *tmp_devices;
blkid_dev_iterate iter = NULL;
blkid_dev dev = NULL;
blkid_cache cache = NULL;
char path[PATH_MAX];
 
-   if (btrfs_scan_done)
-   return 0;
-
if (blkid_get_cache(&cache, 0) < 0) {
printf("ERROR: lblkid cache get failed\n");
return 1;
@@ -2209,29 +2224,73 @@ int btrfs_scan_devices()
/* if we are here its definitely a btrfs disk*/
strncpy(path, blkid_dev_devname(dev), PATH_MAX);
 
-   fd = open(path, O_RDONLY);
-   if (fd < 0) {
-   printf("ERROR: could not open %s\n", path);
-   continue;
-   }
-   ret = btrfs_scan_one_device(fd, path, &tmp_devices,
-   &num_devices, BTRFS_SUPER_INFO_OFFSET, 0);
-   if (ret) {
-   printf("ERROR: could not scan %s\n", path);
-   close (fd);
-   continue;
-   }
-
-   close(fd);
+   scan_device(path);
}
blkid_dev_iterate_end(iter);
blkid_put_cache(cache);
 
-   btrfs_scan_done = 1;
-
return 0;
 }
 
+static int btrfs_scan_devices_by_udev(void)
+{
+   int rc = 1;
+   struct udev *udev = NULL;
+   struct udev_enumerate *en = NULL;
+   struct udev_list_entry *ent, *devs;
+
+   udev = udev_new();
+   if (!udev)
+   goto done;
+   en = udev_enumerate_new(udev);
+   if (!en)
+   goto done;
+
+   udev_enumerate_add_match_subsystem(en, "block");
+   if (udev_enumerate_scan_devices(en) != 0)
+   goto done;
+   devs = udev_enumerate_get_list_entry(en);
+   if (!devs)
+   goto done;
+
+   udev_list_entry_foreach(ent, devs) {
+   const char *val, *name = udev_list_entry_get_name(ent);
+   struct udev_device *dev = udev_device_new_from_syspath(udev, 
name);
+
+   if (!dev)
+   continue;
+   val = udev_device_get_property_value(dev, "ID_FS_TYPE");
+   if (val && strcmp(val, "btrfs") == 0) {
+   const char *path = udev_device_g

[PATCH 1/2] btrfs-progs: rename btrfs_scan_lblkid() to btrfs_scan_devices()

2014-11-11 Thread Karel Zak
It seems like bad idea to use a library name (lblkid) within generic
function name. The currently used scanning library is implementation
detail and this detail should be hidden for rest of the code.

Signed-off-by: Karel Zak 
---
 cmds-device.c | 2 +-
 cmds-filesystem.c | 2 +-
 disk-io.c | 2 +-
 utils.c   | 6 +++---
 utils.h   | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/cmds-device.c b/cmds-device.c
index 9323986..e18ef4b 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -235,7 +235,7 @@ static int cmd_scan_dev(int argc, char **argv)
 
if (all || argc == 1) {
printf("Scanning for Btrfs filesystems\n");
-   ret = btrfs_scan_lblkid();
+   ret = btrfs_scan_devices();
if (ret)
fprintf(stderr, "ERROR: error %d while scanning\n", 
ret);
ret = btrfs_register_all_devices();
diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index e4b2785..25ef382 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -948,7 +948,7 @@ static int cmd_show(int argc, char **argv)
goto out;
 
 devs_only:
-   ret = btrfs_scan_lblkid();
+   ret = btrfs_scan_devices();
 
if (ret) {
fprintf(stderr, "ERROR: %d while scanning\n", ret);
diff --git a/disk-io.c b/disk-io.c
index bade5f0..85a2a57 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -1019,7 +1019,7 @@ int btrfs_scan_fs_devices(int fd, const char *path,
}
 
if (total_devs != 1) {
-   ret = btrfs_scan_lblkid();
+   ret = btrfs_scan_devices();
if (ret)
return ret;
}
diff --git a/utils.c b/utils.c
index 4b3bace..cc33cfc 100644
--- a/utils.c
+++ b/utils.c
@@ -1188,7 +1188,7 @@ int check_mounted_where(int fd, const char *file, char 
*where, int size,
 
/* scan other devices */
if (is_btrfs && total_devs > 1) {
-   ret = btrfs_scan_lblkid();
+   ret = btrfs_scan_devices();
if (ret)
return ret;
}
@@ -1270,7 +1270,7 @@ int btrfs_register_one_device(const char *fname)
 
 /*
  * Register all devices in the fs_uuid list created in the user
- * space. Ensure btrfs_scan_lblkid() is called before this func.
+ * space. Ensure btrfs_scan_devices() is called before this func.
  */
 int btrfs_register_all_devices(void)
 {
@@ -2181,7 +2181,7 @@ int test_dev_for_mkfs(char *file, int force_overwrite, 
char *estr)
return 0;
 }
 
-int btrfs_scan_lblkid()
+int btrfs_scan_devices()
 {
int fd = -1;
int ret;
diff --git a/utils.h b/utils.h
index 8c94624..919dfcf 100644
--- a/utils.h
+++ b/utils.h
@@ -128,7 +128,7 @@ int csum_tree_block(struct btrfs_root *root, struct 
extent_buffer *buf,
   int verify);
 int ask_user(char *question);
 int lookup_ino_rootid(int fd, u64 *rootid);
-int btrfs_scan_lblkid(void);
+int btrfs_scan_devices(void);
 int get_btrfs_mount(const char *dev, char *mp, size_t mp_size);
 int find_mount_root(const char *path, char **mount_root);
 int get_device_info(int fd, u64 devid,
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/2 v4] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-10-31 Thread Karel Zak
On Fri, Oct 31, 2014 at 12:11:20PM +0800, Anand Jain wrote:
> btrfs_scan_lblikd() is called by most the device related command functions.
> And btrfs_scan_lblkid() is most expensive function and it becomes more 
> expensive
> as number of devices in the system increase. Further some threads call this

wouldn't be possible to ask udev rather than scan all devices? I
understand than in some cases it's necessary to have robust and
independent solution, but for usual use-cases it would be less
expensive to read the info from udev where we already keep track about
all block devices and where we call libblkid. 

It would be possible to implement it as optional feature (#ifdev HAVE_LIBUDEV),
the library API is very easy to use.

(For example lsblk uses libblkid as fallback, the default is udev).

    Karel


-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] mount: add btrfs to mount.8

2014-06-09 Thread Karel Zak
On Sat, Jun 07, 2014 at 06:41:50AM -0700, Christoph Hellwig wrote:
> On Fri, Jun 06, 2014 at 10:52:48AM -0500, Eric Sandeen wrote:
> > On 6/6/14, 5:03 AM, Karel Zak wrote:
> > > On Fri, Jun 06, 2014 at 11:44:28AM +0200, Karel Zak wrote:
> > >>  I personally have no problem to maintain information about arbitrary
> > >>  FS in mount.8, the problem are updates. Unfortunately, kernel FS 
> > >> developers
> > >>  don't care about the man page at all and it's very often not up to date.
> > > 
> > >  Hmm.. another possible way would be to create a script for util-linux
> > >  that will analyze kernel Documentation/filesystems/.txt and
> > >  report changes that is necessary to make to mount.8. It should be
> > >  relative simple with git. I'll try it..
> > 
> > I like that idea.  Maybe  will need a defined format, though,
> > right?  Maybe asciidoc?
> > 
> > I've still been meaning (in theory) to produce a mount manpage just for xfs.
> > I'm still willing to do that if the above doesn't pan out.  I just need
> > to get to it.  I'd be happy to do it for extN as well.
> 
> Autogenerating man pages from an adhoc format sounds like the wrong
> approach.  I'd much rather have proper man paged for every filesystem.
> With those we could also drop all that information from the kernel
> Documentation directory, where users won't looks for them anyway.
> 
> Eric, if you take care of xfs an extN I'll get started on man pages
> for the various "minor" filesystems that don't really have active
> maintainers.
> 
> Not sure if we should go for mount..8 man pages or just improve
> the .5 pages, but I think the second one is more obvious.

I think .5 provides opportunity to distribute more information
about the filesystem than just mount options only. See for example
nfs.5 where is complete overview about the filesystem.

Karel


-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] mount: add btrfs to mount.8

2014-06-06 Thread Karel Zak
On Thu, Jun 05, 2014 at 10:05:19AM +0800, Gui Hecheng wrote:
> Based on Documentation/filesystems/btrfs.txt
> 
> Signed-off-by: Gui Hecheng 
> ---
>  sys-utils/mount.8 | 186 
> ++
>  1 file changed, 186 insertions(+)

 Applied, thanks.

    Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] mount: add btrfs to mount.8

2014-06-06 Thread Karel Zak
On Fri, Jun 06, 2014 at 11:44:28AM +0200, Karel Zak wrote:
>  I personally have no problem to maintain information about arbitrary
>  FS in mount.8, the problem are updates. Unfortunately, kernel FS developers
>  don't care about the man page at all and it's very often not up to date.

 Hmm.. another possible way would be to create a script for util-linux
 that will analyze kernel Documentation/filesystems/.txt and
 report changes that is necessary to make to mount.8. It should be
 relative simple with git. I'll try it..

    Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] mount: add btrfs to mount.8

2014-06-06 Thread Karel Zak
On Fri, Jun 06, 2014 at 02:32:39PM +0800, Gui Hecheng wrote:
> But for now, there are btrfs *users* complaining that they cannot find
> any help by refering to the mount manpage when they want to mount their
> btrfs. Actually, not every btrfs user have the mood to check the
> Documentation/filesystems/btrfs.txt or anywhere else when they just want
> to use a simple mount cmd. 

 The kernel is the "ideal place", but it's more dream than anything
 else... what we use for example for NFS is to maintain FS specific
 man page in userspace FS package (e.g. nfs-utils).

 Wouldn't be possible to add the man page to btrfs-progs package? The
 package is maintained by btrfs guys and I guess all btrfs users have
 very probably installed the package.

> So, If it does not bother you much... 

 I personally have no problem to maintain information about arbitrary
 FS in mount.8, the problem are updates. Unfortunately, kernel FS developers
 don't care about the man page at all and it's very often not up to date.

 [My experience is very bad. Not sure why, but for many kernel devels
  commit into Linus' tree is end of their work, they don't care if there
  is usable userpsace util or docs for users, people like Eric Biederman
  or Lukas Czerner are rare exception.]
 
> And I think there are many btrfs developers who will be
> glad to continue helping correct the btrfs part.

 OK, we can try it, if you still prefer mount.8.

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] mount: add btrfs to mount.8

2014-06-05 Thread Karel Zak
On Thu, Jun 05, 2014 at 10:05:19AM +0800, Gui Hecheng wrote:
> Based on Documentation/filesystems/btrfs.txt
> 
> Signed-off-by: Gui Hecheng 
> ---
>  sys-utils/mount.8 | 186 
> ++
>  1 file changed, 186 insertions(+)

 Thanks, but I'm not sure this is the right way. It would be better to
 create BTRFS specific btrfs.5 man page and maintain it together with
 another BTRFS stuff rather then within util-linux. In the mount.8
 should be only a note about btrfs.5.

 My long time goal is to remove all FS specific sections (at least for
 mainstream filesystems) from mount.8, because we duplicate effort
 here. The primary and well maintained is usually kernel
 Documentation/filesystems/ and unfortunately FS developers don't care
 about mount.8 at all.

 The ideal solution would be to generate the FS man pages from kernel
 docs (for example from asciidoc etc.)

 See 6 years old discussions: http://marc.info/?t=122767310200002&r=1&w=2


Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: missing /sbin/fsck.btrfs

2014-01-06 Thread Karel Zak
On Mon, Dec 30, 2013 at 06:18:53PM +0100, Tom Gundersen wrote:
>  * fsck is skipped for filesystems where the relevant helper does not
> exist, so fs_passno=1 has the same effect for xfs and btrfs
> filesystems (either way, nothing happens).
> 
> That still leaves non-systemd systems and calling "fsck -A" manually.
> Maybe a good solution would be to patch fsck to adopt systemd's
> behavior, which would avoid every filesystem having to ship these
> "fake" fsck helpers? What do you think Karel?

 It's already implemented for years, "fsck -A" ignores filesystems 
 without fsck. helpers. It only complains if you explicitly
 specify the device on command line (e.g. fsck /dev/sdb1).

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [systemd-devel] is mounting subvolumes with a read-only root subvolume allowed?

2013-11-14 Thread Karel Zak
On Fri, Nov 15, 2013 at 12:32:10AM +0100, Zbigniew Jędrzejewski-Szmek wrote:
> Hi,
> I have a box with / and /home being subvolumes from the same btrfs filesystem.
> 
> /etc/fstab:
> UUID=c0686...  /  btrfs subvol=root,x-systemd.device-timeout=0 1 1
> UUID=c0686...  /home  btrfs subvol=home,x-systemd.device-timeout=0 1 1
> ...
> 
> / is initially mounted readonly by the initramfs, and then after switching
> to the real system, /home is attempted to be mounted in parallel with /
> being remounted rw. If remounting rw happens first, boot proceeds. If
> mounting /home is attempted to realy, it fails.
> 
> $ /bin/mount /home
> mount: /dev/mapper/luks-765... is already mounted or /home busy
>/dev/mapper/luks-765... is already mounted on /
> $ /bin/mount -o remount,rw /
> $ /bin/mount /home
> $
> 
> So, is this expected that the other subvolume must be mounted rw?

 This is known and pretty stupid issue:
 http://www.spinics.net/lists/linux-btrfs/msg25502.html

 ... but it seems that btrfs guys are fine with this "feature".

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] btrfs: document mount options in Documentation/fs/btrfs.txt

2013-03-26 Thread Karel Zak
On Sat, Mar 23, 2013 at 01:45:36PM -0500, Eric Sandeen wrote:
> The mount manpage is maintained in the util-linux pkg,
> but it's not kernel-version specific, and the util-linux
> maintainer does not have specific knowledge of all filesytem
> options.

Absolutely true. The util-linux maintainer is not happy that we 
don't have filesystem specific man pages for many filesystems.

The goal is to have in mount.8 only links to another man pages, for
example see smart NFS guys:

  Mount options for nfs and nfs4:
   See the options section of the nfs(5) man page (nfs-utils
   package must be installed).

  
> I think kernel docs are the right place for the developers
> to first document these things

 Definitely.

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] xfs_mkfs: wipe old signatures from the device

2013-02-13 Thread Karel Zak
On Wed, Feb 13, 2013 at 11:41:08AM +0100, Lukáš Czerner wrote:
> However
> 
> mkfs.btrfs /dev/sda
> mkfs.ext4 -F /dev/sda
> 
> works well, however I am not sure why that is. Is that some kind of
> mount(8) magic ?

 This is bug in libmount. Fixed in upstream tree. The libmount in some
 cases ignores the ambivalent probing result from libblkid and tries 
 stuff from /etc/filesystems (where is for example ext4).

> So I think that liblkid _and_ btrfs tools has to be fixed not to treat
> backup superblocks as primary!

 The problem with additional btrfs superblocks has been already reported
 to btrfs guys:

   http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg20957.html

 and I don't see a reply "yep, this is btrfs-progs bug" :-)

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH][v2] Btrfs: wipe all the superblock [redhat bugzilla 889888]

2013-01-10 Thread Karel Zak
On Wed, Jan 09, 2013 at 10:14:43PM +0100, Goffredo Baroncelli wrote:
>  libblkid/src/superblocks/btrfs.c |8 
>  1 file changed, 8 insertions(+)

 Applied, thanks.

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

2013-01-09 Thread Karel Zak
On Wed, Jan 09, 2013 at 06:48:28PM +0100, Goffredo Baroncelli wrote:
> Hi Karel,
> 
> > 
> >  You can specify more than one magic strings for the same filesystem,
> >  the .magics = { } is array.
> 
> thanks for you suggestion. However this seems to me not applicable. I
> tried to change the code, and what I got to me seems inconsistently:
> 
> Whit this change
> 
> 1) if I do "wipefs ", I got the offset of the first superblock
> (good enough)
> 2) if I do "wipefs -a ", I clean-up *all three* superblocks
> (very good)
> 3) if I do "wipefs -o  ", I clean-up only the superblock
> located at  (very bad)

 this is expected behavior described in wipefs man page:

Note  that  some  filesystems  or  some partition tables store more
magic strings on the devices. The wipefs lists the first offset where
a magic string has been detected. The device is not  scanned  for
additional magic strings for the same filesystem. It's possible that
after wipefs -o  will be the same filesystem or partition
table visible by another magic string on another offset.

> If the user doesn't know enough btrfs, trying 1) and 3) could think that
> the disk is cleaned-up. Instead the 2nd and the 3rd super-blocks still
> exist.

 well, users (and installers) usually use wipefs -a or wipefs -t 

> >  see for example libblkid/src/superblocks/reiserfs.c 
> 
> I think that this is a different case: the reiser superblocks are

 it was example how to specify the magic strings in the code

> *alternative*; instead in the btrfs case, *all the three superblocks*
> exist at the same time.

 this is pretty common to have backup superblock (e.g. GPT) or more
 ways how to detect the filesystem (e.g. FAT).

 Please, send me the patch with the magic strings :-)
 
 I really don't want to add dummy filesystems to the library (like you
 did in the first version of the patch) -- it's very bad idea with
 many side effects.

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

2013-01-08 Thread Karel Zak
On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
> If the first superblock is valid except that the "magic field" is zeroed,
> btrfs skips the check of the other superblocks.
> If the first superblock is fully invalid, btrfs checks for the other
> superblock.

 Hmm... why inconsistent (or missing) superblock is not reported as a
 problem? If I good understand the filesystem is still mountable,
 right?

> So zeroing the first superblock "magic field" at the beginning seems
> that the filesystem is wiped.

 Well, this is exactly the idea behind wipefs(8), just wipe minimal
 number of bytes from the device to make the filesystem invisible for
 libblkid (udev, ...). This concept is relatively safe, if you make a
 mistake than you can restore the magic string, your data should not
 be affected by wipefs(8).

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

2013-01-08 Thread Karel Zak
On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
> +const struct blkid_idinfo btrfs_idinfo1 =
> +{
> + .name   = "btrfs [bak #1]",
> + .usage  = BLKID_USAGE_FILESYSTEM,
> + .probefunc  = probe_btrfs,
> + .minsz  = 64 * 1024 * 1024 + 4 * 1024,
> + .magics =
> + {
> + { .magic = "_BHRfS_M", 
> +   .len = 8, 
> +   .kboff = 64 * 1024, 
> +   .sboff = 0x40 },
> + { NULL }
> + }
> +};
> +
> +const struct blkid_idinfo btrfs_idinfo2 =
> +{
> + .name   = "btrfs [bak #2]",
> + .usage  = BLKID_USAGE_FILESYSTEM,
> + .probefunc  = probe_btrfs,
> + .minsz  = 256 * 1024 * 1024 * 1024 + 4 *1024,
> + .magics =
> + {
> + { .magic = "_BHRfS_M", 
> +   .len = 8, 
> +   .kboff = 256 * 1024 * 1024, 
> +   .sboff = 0x40 },
> + { NULL }
> + }
> +};

 You can specify more than one magic strings for the same filesystem,
 the .magics = { } is array.

 .magics = {

/* backup #1 */
{ .magic = "_BHRfS_M", 
  .len = 8, 
  .kboff = 64 * 1024, 
  .sboff = 0x40 },
},

/* backup #2 */
{   .magic = "_BHRfS_M",
.len = 8,
    .kboff = 256 * 1024 * 1024,
.sboff = 0x40
}, 
...
 }

 see for example libblkid/src/superblocks/reiserfs.c 

 Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Btrfs: do not mount when we have a sectorsize unequal to PAGE_SIZE

2012-04-03 Thread Karel Zak
On Tue, Apr 03, 2012 at 09:56:53AM +0800, Liu Bo wrote:
> Our code is not ready to cope with a sectorsize that's not equal to PAGE_SIZE.
> It will lead to hanging-on while writing something.
> 
> Signed-off-by: Liu Bo 
> ---
>  fs/btrfs/disk-io.c |6 +++---
>  1 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
> index 20196f4..b9866f2 100644
> --- a/fs/btrfs/disk-io.c
> +++ b/fs/btrfs/disk-io.c
> @@ -2254,9 +2254,9 @@ int open_ctree(struct super_block *sb,
>   goto fail_sb_buffer;
>   }
>  
> - if (sectorsize < PAGE_SIZE) {
> - printk(KERN_WARNING "btrfs: Incompatible sector size "
> -"found on %s\n", sb->s_id);
> + if (sectorsize != PAGE_SIZE) {
> + printk(KERN_WARNING "btrfs: Incompatible sector size(%lu) "
> +"found on %s\n", (unsigned long)sectorsize, sb->s_id);

 That's strange. Does it mean that if I create the filesystem and then
 reboot to another kernel with different PAGE_SIZE then the filesystem
 is unaccessible for me?

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: "Invalid argument" when mounting a btrfs raid1 filesystem

2012-03-26 Thread Karel Zak
On Sat, Mar 24, 2012 at 06:21:05PM +, Hugo Mills wrote:
>As Sadner says, you have to run "btrfs dev scan" before you try to
> mount the FS. If you have root on btrfs, this will have to go in an
> initrd; otherwise, it can go in your initscripts anywhere before the
> non-root filesystem mounts.
> 
>Basically, the kernel needs to know which devices hold which btrfs
> filesystems (organised by UUID) before it tries to mount them. So,
> there's an ioctl that is used for sending that data to the kernel, and
> a userspace tool (btrfs dev scan) that enumerates all of the block
> devices it can see, looks for a btrfs superblock on them, and tells
> the kernel.

 Please, move all this logic to udev rules where we already scans all
 devices. It's really bad to scan all device more than once. We spent
 years to fix this problem for LVM, I don't think that btrfs has to
 repeat the same mistakes.

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: LABEL only 1 device

2012-02-28 Thread Karel Zak
On Sun, Feb 26, 2012 at 06:07:31PM +, Duncan wrote:
> Unfortunately, since gpt is reasonably new in terms of filesystem and 
> partitioning tools, there isn't really anything (mount, etc) that makes 
> /use/ of that label yet,

udev exports GPT labels and uuids by symlinks, see

  ls /dev/disk/by-partlabel/
  ls /dev/disk/by-partuuid/

you can use these links in your fstab. And if I good remember kernel
supports PARTUUID for root= command line option.

    Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: subvolume info in /proc/mounts

2012-02-06 Thread Karel Zak
On Sat, Feb 04, 2012 at 08:40:51PM +0200, Nikos Voutsinas wrote:
> There was a patch to include the subvolume mount option into /proc/mounts.
> Did that make it into the kernel;
> 
> If not, what is the formal way to find out which subvolume is mounted;

 see /proc/self/mountinfo there is 'fs root' column, for bind mounts
 and btrfs subvolumes is there '/' (and '/' for normal mounts).

 findmnt(8) uses the path in SOURCE column, for example /dev/sda1[/subvolume].

Karel


-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: UUID of subvolumes

2010-04-08 Thread Karel Zak
On Thu, Apr 08, 2010 at 08:51:42AM -0700, David Brown wrote:
> I am developing backup software (<http://github.com/d3zd3z/jpool> for
> the curious), and have been doing some testing with btrfs.
>
> Jpool currently uses the blkid database to map between device numbers
> (st_rdev) and the uuid of a particular filesystem.  I originally
> created this because LVM device numbers sometimes changed.  Jpool
> uses the uuid to track files within a tree.
>
> The subvolumes on btrfs seem to be getting ephemeral device numbers,
> which aren't listed in the blkid output.  The program falls back to
> using the mountpoint, but that misses mountpoints changing.
>
>   - Do subvolumes in btrfs even have separate uuids, and should they?

I think yes, in libblkid code is:

   blkid_probe_set_uuid(pr, bfs->fsid);
   blkid_probe_set_uuid_as(pr, bfs->dev_item.uuid, "UUID_SUB");

so there should be two UUIDs for the device.

However, I'd like to see more information about btrfs UUIDs.

BTW, see my old discussion with Andreas about UUIDs:
http://marc.info/?l=linux-fsdevel&m=126754081900406&w=2

Karel

-- 
 Karel Zak  
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/2] btrfs-progs: multidevice support for check_mounted

2009-11-23 Thread Karel Zak
On Sun, Nov 22, 2009 at 02:33:13PM +0100, Andi Drebes wrote:
> In the new patch below, is_pseudo_fs() is replaced by
> is_existing_blk_or_reg_file(). We ignore entries associated with an
> invalid path or paths that don't point to a regular or block file.
> However, if a path used in a pseudo-filesystem entry points to the
> file that is being checked, check_mounted() returns 1. In my eyes,
> this is extremely unlikely.
...
> +/* Checks if a file exists and is a block or regular file*/
> +int is_existing_blk_or_reg_file(const char* filename)
> +{
> + struct stat st_buf;
> +
> + if(stat(filename, &st_buf) < 0) {
> + if(errno == ENOENT)
> + return 0;
> + else
> + return -errno;
> + }
> +
> + return (S_ISBLK(st_buf.st_mode) || S_ISREG(st_buf.st_mode));
> +}

 Yes, this looks better.

Karel

-- 
 Karel Zak  
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/2] btrfs-progs: multidevice support for check_mounted

2009-11-21 Thread Karel Zak
On Sat, Nov 21, 2009 at 03:38:38PM +0100, Andi Drebes wrote:
> +
> +/* Checks if an mntentry represents a pseudo FS */
> +int is_pseudo_fs(const struct mntent* mnt)
> +{
> + struct stat st_buf;
> +
> + if(stat(mnt->mnt_fsname, &st_buf) < 0) {
> + if(errno == ENOENT)
> + return 1;
> + else
> + return -errno;
> + }
> +
> + return 0;
> +}

 This is bad idea. The mnt_fsname field could be an arbitrary string
 include valid paths.

# grep sysfs /proc/mounts 
    /sys /sys sysfs rw,relatime 0 0

 Karel

-- 
 Karel Zak  
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html