Re: [PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON
On 3/7/18 1:34 AM, Qu Wenruo wrote: > > > On 2018年03月03日 02:47, je...@suse.com wrote: >> diff --git a/configure.ac b/configure.ac >> index 56d17c3a..6aec672a 100644 >> --- a/configure.ac >> +++ b/configure.ac >> @@ -197,6 +197,12 @@ PKG_STATIC(UUID_LIBS_STATIC, [uuid]) >> PKG_CHECK_MODULES(ZLIB, [zlib]) >> PKG_STATIC(ZLIB_LIBS_STATIC, [zlib]) >> >> +PKG_CHECK_MODULES(JSON, [json-c], [ > > Json-c is quite common and also used by cryptsetup, so pretty good > library choice. Yep, so that puts it in the base system packages of most distros. >> diff --git a/qgroup.c b/qgroup.c >> index 2d0a6947..f632a45c 100644 >> --- a/qgroup.c >> +++ b/qgroup.c >> return ret; >> } >> >> +#ifdef HAVE_JSON >> +static void format_qgroupid(char *buf, size_t size, u64 qgroupid) >> +{ >> +int ret; >> + >> +ret = snprintf(buf, size, "%llu/%llu", >> + btrfs_qgroup_level(qgroupid), >> + btrfs_qgroup_subvid(qgroupid)); >> +ASSERT(ret < sizeof(buf)); > > This is designed to catch truncated snprintf(), right? > This can be addressed by setting up the @buf properly. > (See below) > > And in fact, due to that super magic number, we won't hit this ASSERT() > anyway. Yep, but ASSERTs are there to detect/prevent developer mistakes. This should only trigger due to a simple bug, so we ASSERT rather than handle the error gracefully. [...] >> +static bool export_one_qgroup(json_object *container, >> + const struct btrfs_qgroup *qgroup, bool compat) >> +{ >> +json_object *obj = json_object_new_object(); >> +json_object *tmp; >> +char buf[42]; > > Answer to the ultimate question of life, the universe, and everything. :) > > Although according to the format level/subvolid, it should be > count_digits(MAX_U16) + 1 + count_digits(MAX_U48) + 1. (1 for '/' and 1 > for '\n') > > Could be defined as a macro as: > #define QGROUP_FORMAT_BUF_LEN (count_digits(1ULL<<16) + 1 + \ >count_digits(1ULL<<48) + 1) Which would mean we'd execute count_digits twice for every qgroup to resolve a constant. I'll replace the magic number with a define though. > BTW, the result is just 22. It's a worst-case. We're using %llu, so 42 is the length of two 64-bit numbers in base ten, plus the slash and nul terminator. In practice we won't hit the limit, but it doesn't hurt. Thanks for the review. -Jeff -- Jeff Mahoney SUSE Labs signature.asc Description: OpenPGP digital signature
Re: [PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON
On 2018年03月03日 02:47, je...@suse.com wrote: > From: Jeff Mahoney> > One of the common requests I receive is for 'df' like facilities > for subvolume usage. Really, the request is for monitoring tools to be > able to understand when subvolumes may be approaching quota in the same > manner traditional file systems approach ENOSPC. > > This patch allows us to export the qgroups data in a machine-readable > format so that monitoring tools can parse it easily. > > There are two modes since JSON can technically handle 64-bit numbers > but JavaScript proper cannot. show -j enables JSON mode using 64-bit > integers directly. --json-compat presents 64-bit numbers as an array > of two 32-bit numbers (high followed by low). > > Signed-off-by: Jeff Mahoney > --- > Documentation/btrfs-qgroup.asciidoc | 4 + > Makefile.inc.in | 4 +- > cmds-qgroup.c | 36 +- > configure.ac| 6 + > qgroup.c| 211 > > qgroup.h| 3 + > 6 files changed, 258 insertions(+), 6 deletions(-) > > diff --git a/Documentation/btrfs-qgroup.asciidoc > b/Documentation/btrfs-qgroup.asciidoc > index 360b3269..22a9c2a7 100644 > --- a/Documentation/btrfs-qgroup.asciidoc > +++ b/Documentation/btrfs-qgroup.asciidoc > @@ -105,6 +105,10 @@ list all qgroups which impact the given path(include > ancestral qgroups) > list all qgroups which impact the given path(exclude ancestral qgroups) > -v > Be more verbose. Print pathnames of member qgroups when nested. > +-j > +If enabled, export qgroup usage information in JSON format. This implies > --raw. > +--json-compat > +By default, JSON output contains full 64-bit integers, which may be > incompatible with some JSON parsers. This option exports those values as an > array of 32-bit numbers in [high, low] format. > --raw > raw numbers in bytes, without the 'B' suffix. > --human-readable > diff --git a/Makefile.inc.in b/Makefile.inc.in > index 56271903..68bddbed 100644 > --- a/Makefile.inc.in > +++ b/Makefile.inc.in > @@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@ > SUBST_CFLAGS = @CFLAGS@ > SUBST_LDFLAGS = @LDFLAGS@ > > -LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread > +LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread > LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@ > -STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread > +STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ > -L. -pthread > STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@ > > prefix ?= @prefix@ > diff --git a/cmds-qgroup.c b/cmds-qgroup.c > index 94cd0fd3..eee15ef1 100644 > --- a/cmds-qgroup.c > +++ b/cmds-qgroup.c > @@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = { > " (excluding ancestral qgroups)", > "-P print first-level qgroups using pathname", > "-v verbose, prints all nested subvolumes", > +#ifdef HAVE_JSON > + "-j export in JSON format", > + "--json-compat export in JSON compatibility mode", > +#endif > HELPINFO_UNITS_LONG, > "--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname", > " list qgroups sorted by specified items", > @@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv) > unsigned unit_mode; > int sync = 0; > bool verbose = false; > + bool export_json = false; > + bool compat_json = false; > > struct btrfs_qgroup_comparer_set *comparer_set; > struct btrfs_qgroup_filter_set *filter_set; > @@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv) > int c; > enum { > GETOPT_VAL_SORT = 256, > - GETOPT_VAL_SYNC > + GETOPT_VAL_SYNC, > + GETOPT_VAL_JSCOMPAT, > }; > static const struct option long_options[] = { > {"sort", required_argument, NULL, GETOPT_VAL_SORT}, > {"sync", no_argument, NULL, GETOPT_VAL_SYNC}, > {"verbose", no_argument, NULL, 'v'}, > +#ifdef HAVE_JSON > + {"json-compat", no_argument, NULL, GETOPT_VAL_JSCOMPAT}, > +#endif > { NULL, 0, NULL, 0 } > }; > - > - c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL); > + const char getopt_chars[] = { > + 'p', 'P', 'c', 'r', 'e', 'F', 'f', 'v', > +#ifdef HAVE_JSON > + 'j', > +#endif > + '\0' }; > + > + c = getopt_long(argc, argv, getopt_chars, long_options, NULL); > if (c < 0) > break; > switch (c) { > @@ -353,6 +369,14 @@
[PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON
From: Jeff MahoneyOne of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC. This patch allows us to export the qgroups data in a machine-readable format so that monitoring tools can parse it easily. There are two modes since JSON can technically handle 64-bit numbers but JavaScript proper cannot. show -j enables JSON mode using 64-bit integers directly. --json-compat presents 64-bit numbers as an array of two 32-bit numbers (high followed by low). Signed-off-by: Jeff Mahoney --- Documentation/btrfs-qgroup.asciidoc | 4 + Makefile.inc.in | 4 +- cmds-qgroup.c | 36 +- configure.ac| 6 + qgroup.c| 211 qgroup.h| 3 + 6 files changed, 258 insertions(+), 6 deletions(-) diff --git a/Documentation/btrfs-qgroup.asciidoc b/Documentation/btrfs-qgroup.asciidoc index 360b3269..22a9c2a7 100644 --- a/Documentation/btrfs-qgroup.asciidoc +++ b/Documentation/btrfs-qgroup.asciidoc @@ -105,6 +105,10 @@ list all qgroups which impact the given path(include ancestral qgroups) list all qgroups which impact the given path(exclude ancestral qgroups) -v Be more verbose. Print pathnames of member qgroups when nested. +-j +If enabled, export qgroup usage information in JSON format. This implies --raw. +--json-compat +By default, JSON output contains full 64-bit integers, which may be incompatible with some JSON parsers. This option exports those values as an array of 32-bit numbers in [high, low] format. --raw raw numbers in bytes, without the 'B' suffix. --human-readable diff --git a/Makefile.inc.in b/Makefile.inc.in index 56271903..68bddbed 100644 --- a/Makefile.inc.in +++ b/Makefile.inc.in @@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@ SUBST_CFLAGS = @CFLAGS@ SUBST_LDFLAGS = @LDFLAGS@ -LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread +LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@ -STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread +STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ -L. -pthread STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@ prefix ?= @prefix@ diff --git a/cmds-qgroup.c b/cmds-qgroup.c index 94cd0fd3..eee15ef1 100644 --- a/cmds-qgroup.c +++ b/cmds-qgroup.c @@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = { " (excluding ancestral qgroups)", "-P print first-level qgroups using pathname", "-v verbose, prints all nested subvolumes", +#ifdef HAVE_JSON + "-j export in JSON format", + "--json-compat export in JSON compatibility mode", +#endif HELPINFO_UNITS_LONG, "--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname", " list qgroups sorted by specified items", @@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv) unsigned unit_mode; int sync = 0; bool verbose = false; + bool export_json = false; + bool compat_json = false; struct btrfs_qgroup_comparer_set *comparer_set; struct btrfs_qgroup_filter_set *filter_set; @@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv) int c; enum { GETOPT_VAL_SORT = 256, - GETOPT_VAL_SYNC + GETOPT_VAL_SYNC, + GETOPT_VAL_JSCOMPAT, }; static const struct option long_options[] = { {"sort", required_argument, NULL, GETOPT_VAL_SORT}, {"sync", no_argument, NULL, GETOPT_VAL_SYNC}, {"verbose", no_argument, NULL, 'v'}, +#ifdef HAVE_JSON + {"json-compat", no_argument, NULL, GETOPT_VAL_JSCOMPAT}, +#endif { NULL, 0, NULL, 0 } }; - - c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL); + const char getopt_chars[] = { + 'p', 'P', 'c', 'r', 'e', 'F', 'f', 'v', +#ifdef HAVE_JSON + 'j', +#endif + '\0' }; + + c = getopt_long(argc, argv, getopt_chars, long_options, NULL); if (c < 0) break; switch (c) { @@ -353,6 +369,14 @@ static int cmd_qgroup_show(int argc, char **argv) case 'f': filter_flag |= 0x2; break; +#ifdef HAVE_JSON + case
[PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON
From: Jeff MahoneyOne of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC. This patch allows us to export the qgroups data in a machine-readable format so that monitoring tools can parse it easily. There are two modes since JSON can technically handle 64-bit numbers but JavaScript proper cannot. show -j enables JSON mode using 64-bit integers directly. --json-compat presents 64-bit numbers as an array of two 32-bit numbers (high followed by low). Signed-off-by: Jeff Mahoney --- Documentation/btrfs-qgroup.asciidoc | 4 + Makefile.inc.in | 4 +- cmds-qgroup.c | 36 +- configure.ac| 6 + qgroup.c| 211 qgroup.h| 3 + 6 files changed, 258 insertions(+), 6 deletions(-) diff --git a/Documentation/btrfs-qgroup.asciidoc b/Documentation/btrfs-qgroup.asciidoc index 360b3269..22a9c2a7 100644 --- a/Documentation/btrfs-qgroup.asciidoc +++ b/Documentation/btrfs-qgroup.asciidoc @@ -105,6 +105,10 @@ list all qgroups which impact the given path(include ancestral qgroups) list all qgroups which impact the given path(exclude ancestral qgroups) -v Be more verbose. Print pathnames of member qgroups when nested. +-j +If enabled, export qgroup usage information in JSON format. This implies --raw. +--json-compat +By default, JSON output contains full 64-bit integers, which may be incompatible with some JSON parsers. This option exports those values as an array of 32-bit numbers in [high, low] format. --raw raw numbers in bytes, without the 'B' suffix. --human-readable diff --git a/Makefile.inc.in b/Makefile.inc.in index 56271903..68bddbed 100644 --- a/Makefile.inc.in +++ b/Makefile.inc.in @@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@ SUBST_CFLAGS = @CFLAGS@ SUBST_LDFLAGS = @LDFLAGS@ -LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread +LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@ -STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread +STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ -L. -pthread STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@ prefix ?= @prefix@ diff --git a/cmds-qgroup.c b/cmds-qgroup.c index 94cd0fd3..eee15ef1 100644 --- a/cmds-qgroup.c +++ b/cmds-qgroup.c @@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = { " (excluding ancestral qgroups)", "-P print first-level qgroups using pathname", "-v verbose, prints all nested subvolumes", +#ifdef HAVE_JSON + "-j export in JSON format", + "--json-compat export in JSON compatibility mode", +#endif HELPINFO_UNITS_LONG, "--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname", " list qgroups sorted by specified items", @@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv) unsigned unit_mode; int sync = 0; bool verbose = false; + bool export_json = false; + bool compat_json = false; struct btrfs_qgroup_comparer_set *comparer_set; struct btrfs_qgroup_filter_set *filter_set; @@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv) int c; enum { GETOPT_VAL_SORT = 256, - GETOPT_VAL_SYNC + GETOPT_VAL_SYNC, + GETOPT_VAL_JSCOMPAT, }; static const struct option long_options[] = { {"sort", required_argument, NULL, GETOPT_VAL_SORT}, {"sync", no_argument, NULL, GETOPT_VAL_SYNC}, {"verbose", no_argument, NULL, 'v'}, +#ifdef HAVE_JSON + {"json-compat", no_argument, NULL, GETOPT_VAL_JSCOMPAT}, +#endif { NULL, 0, NULL, 0 } }; - - c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL); + const char getopt_chars[] = { + 'p', 'P', 'c', 'r', 'e', 'F', 'f', 'v', +#ifdef HAVE_JSON + 'j', +#endif + '\0' }; + + c = getopt_long(argc, argv, getopt_chars, long_options, NULL); if (c < 0) break; switch (c) { @@ -353,6 +369,14 @@ static int cmd_qgroup_show(int argc, char **argv) case 'f': filter_flag |= 0x2; break; +#ifdef HAVE_JSON + case