date:20160518

[dm-devel] [PATCH v4 1/1] add display of map information in JSON format

2016-05-18 Thread Todd Gill

The patch adds these commands:

multipathd show maps json
multipathd show map $map json

Each command will output the requested map(s) in JSON.

For the "show maps json" command, the patch pre-allocates
INITIAL_REPLY_LEN * PRINT_JSON_MULTIPLIER(5).  The JSON text
is about 5x the size of the "show maps topology" text.

v3:

Added format specifiers at the map level to split out
vend/prod/rev.

A user can now specify the following with:

multipathd show map(s) format

%v - vend
%p - prod
%e - rev

v4:

removed space in major_version and minor_version keys.

Signed-off-by: Todd Gill 
---
 libmultipath/print.c  | 220 ++
 libmultipath/print.h  |  65 ++
 multipathd/cli.c  |   3 +
 multipathd/cli.h  |   2 +
 multipathd/cli_handlers.c |  93 
 multipathd/cli_handlers.h |   2 +
 multipathd/main.c |   2 +
 7 files changed, 387 insertions(+)

diff --git a/libmultipath/print.c b/libmultipath/print.c
index 7fec6e9..5d668bb 100644
--- a/libmultipath/print.c
+++ b/libmultipath/print.c
@@ -274,6 +274,61 @@ snprint_multipath_vpr (char * buff, size_t len, struct 
multipath * mpp)
return snprintf(buff, len, "##,##");
 }
 
+
+static int
+snprint_multipath_vend (char * buff, size_t len, struct multipath * mpp)
+{
+   struct pathgroup * pgp;
+   struct path * pp;
+   int i, j;
+
+   vector_foreach_slot(mpp->pg, pgp, i) {
+   if (!pgp)
+   continue;
+   vector_foreach_slot(pgp->paths, pp, j) {
+   if (strlen(pp->vendor_id))
+   return snprintf(buff, len, "%s", pp->vendor_id);
+   }
+   }
+   return snprintf(buff, len, "##");
+}
+
+static int
+snprint_multipath_prod (char * buff, size_t len, struct multipath * mpp)
+{
+   struct pathgroup * pgp;
+   struct path * pp;
+   int i, j;
+
+   vector_foreach_slot(mpp->pg, pgp, i) {
+   if (!pgp)
+   continue;
+   vector_foreach_slot(pgp->paths, pp, j) {
+   if (strlen(pp->product_id))
+   return snprintf(buff, len, "%s", 
pp->product_id);
+   }
+   }
+   return snprintf(buff, len, "##");
+}
+
+static int
+snprint_multipath_rev (char * buff, size_t len, struct multipath * mpp)
+{
+   struct pathgroup * pgp;
+   struct path * pp;
+   int i, j;
+
+   vector_foreach_slot(mpp->pg, pgp, i) {
+   if (!pgp)
+   continue;
+   vector_foreach_slot(pgp->paths, pp, j) {
+   if (strlen(pp->rev))
+   return snprintf(buff, len, "%s", pp->rev);
+   }
+   }
+   return snprintf(buff, len, "##");
+}
+
 static int
 snprint_action (char * buff, size_t len, struct multipath * mpp)
 {
@@ -577,6 +632,9 @@ struct multipath_data mpd[] = {
{'3', "total_q_time",  0, snprint_total_q_time},
{'4', "q_timeouts",0, snprint_q_timeouts},
{'s', "vend/prod/rev", 0, snprint_multipath_vpr},
+   {'v', "vend",  0, snprint_multipath_vend},
+   {'p', "prod",  0, snprint_multipath_prod},
+   {'e', "rev",   0, snprint_multipath_rev},
{0, NULL, 0 , NULL}
 };
 
@@ -1000,6 +1058,168 @@ snprint_multipath_topology (char * buff, int len, 
struct multipath * mpp,
 }
 
 static int
+snprint_json (char * buff, int len, int indent, char *json_str)
+{
+   int fwd = 0, i;
+
+   for (i = 0; i < indent; i++) {
+   fwd += snprintf(buff + fwd, len - fwd, PRINT_JSON_INDENT);
+   if (fwd > len)
+   return fwd;
+   }
+
+   fwd += snprintf(buff + fwd, len - fwd, "%s", json_str);
+   return fwd;
+}
+
+static int
+snprint_json_header (char * buff, int len)
+{
+   int fwd = 0;
+
+   fwd +=  snprint_json(buff, len, 0, PRINT_JSON_START_ELEM);
+   if (fwd > len)
+   return fwd;
+
+   fwd +=  snprintf(buff + fwd, len  - fwd, PRINT_JSON_START_VERSION,
+   PRINT_JSON_MAJOR_VERSION, PRINT_JSON_MINOR_VERSION);
+   return fwd;
+}
+
+static int
+snprint_json_elem_footer (char * buff, int len, int indent, int last)
+{
+   int fwd = 0, i;
+
+   for (i = 0; i < indent; i++) {
+   fwd += snprintf(buff + fwd, len - fwd, PRINT_JSON_INDENT);
+   if (fwd > len)
+   return fwd;
+   }
+
+   if (last == 1)
+   fwd += snprintf(buff + fwd, len - fwd, "%s", 
PRINT_JSON_END_LAST_ELEM);
+   else
+   fwd += snprintf(buff + fwd, len - fwd, "%s", 
PRINT_JSON_END_ELEM);
+   return fwd;
+}
+
+static int
+snprint_multipath_fields_json (char * buff, int len,
+   struct multipath * mpp, int last)
+{
+   int i, j, fwd = 0;
+   struct path *pp;
+   struct pathgroup *pgp;
+
+   fwd

[dm-devel] [PATCH v4 0/1] add option to output JSON for multipathd command

2016-05-18 Thread Todd Gill

Hi,

I had earlier sent an email to dm-devel proposing we add a feature
in multipathd to output multipath map topology in JSON format. This
patch contains to the code for that feature.

Having an option for the  CLI to output in JSON would allow higher
level applications to more easily monitor/manage multipath.

I thought it was best to take advantage of some of the existing
snprintf_xxx functions rather than adding a library dependency
to multipathd.

Any feedback welcome.

v2:
- added major/minor version fields
- updates from feedback
- added path groups
- changed indent levels to make it easier to read
- removed quotes from integer fields
- fixed bug when the JSON string length exceeded the allocated memory
- removed multipath field from path object - it isn't needed since
  the path is inside the map

v3:
- split out vend/prod/rev into seperate fields
- added new format specifiers for map:
  %v - vend
  %p - prod
  %e - rev
- removed vend/prod/rev from the path level since it is in the map

v4:
- removed space from major/minor version fields

Example of v4 JSON:

# multipathd show map mpathi json
{
   "major_version": 0,
   "minor_version": 1,
   "map":{
  "name" : "mpathi",
  "uuid" : "35000c5008868201b",
  "sysfs" : "dm-11",
  "failback" : "-",
  "queueing" : "-",
  "paths" : 2,
  "write_prot" : "rw",
  "dm-st" : "active",
  "size" : "279G",
  "features" : "0",
  "hwhandler" : "0",
  "action" : "",
  "path_faults" : 0,
  "vend" : "SEAGATE",
  "prod" : "ST300MM0026",
  "rev" : "0003",
  "switch_grp" : 0,
  "map_loads" : 1,
  "total_q_time" : 0,
  "q_timeouts" : 0,
  "path_groups": [{
 "selector" : "service-time 0",
 "pri" : 1,
 "dm_st" : "active",
 "paths": [{
"uuid" : "35000c5008868201b",
"hcil" : "5:0:16:0",
"dev" : "sdv",
"dev_t" : "65:80",
"dm_st" : "active",
"dev_st" : "running",
"chk_st" : "ready",
"checker" : "directio",
"next_check" : "X. 10/20",
"pri" : 1,
"size" : "279G",
"serial" : "S0K41A7WM535KGGG",
"host WWNN" : "[undef]",
"target WWNN" : "0x5000c50088682019",
"host WWPN" : "[undef]",
"target WWPN" : "[undef]",
"host adapter" : ":00:07.0"
 }]
  },{
 "selector" : "service-time 0",
 "pri" : 1,
 "dm_st" : "enabled",
 "paths": [{
"uuid" : "35000c5008868201b",
"hcil" : "5:0:36:0",
"dev" : "sdao",
"dev_t" : "66:128",
"dm_st" : "active",
"dev_st" : "running",
"chk_st" : "ready",
"checker" : "directio",
"next_check" : "X. 10/20",
"pri" : 1,
"size" : "279G",
"serial" : "S0K41A7WM535KGGG",
"host WWNN" : "[undef]",
"target WWNN" : "0x5000c5008868201a",
"host WWPN" : "[undef]",
"target WWPN" : "[undef]",
"host adapter" : ":00:07.0"
 }]
  }]
   }
}

Todd Gill (1):
  add display of map information in JSON format

 libmultipath/print.c  | 220 ++
 libmultipath/print.h  |  65 ++
 multipathd/cli.c  |   3 +
 multipathd/cli.h  |   2 +
 multipathd/cli_handlers.c |  93 
 multipathd/cli_handlers.h |   2 +
 multipathd/main.c |   2 +
 7 files changed, 387 insertions(+)

-- 
2.5.5

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] bcache gets stuck flushing writeback cache when used in combination with LUKS/dm-crypt and non-default bucket size

2016-05-18 Thread James Johnston

> On Sun, 8 May 2016, James Johnston wrote:
> 
> > Hi,
> >
> > [1.] One line summary of the problem:
> >
> > bcache gets stuck flushing writeback cache when used in combination with
> > LUKS/dm-crypt and non-default bucket size
> >
> > [2.] Full description of the problem/report:
> >
> > I've run into a problem where the bcache writeback cache can't be flushed to
> > disk when the backing device is a LUKS / dm-crypt device and the cache set 
> > has
> > a non-default bucket size.
> 
> You might try LUKS atop of bcache instead of under it.  This might be
> better for privacy too, otherwise your cached data is unencrypted.

Only in this test case; on my real setup, the cache device is also layered on 
top
Of LUKS.  (On both backing & cache, it's LUKS --> LVM2 --> bcache.  This gives 
me
flexibility to adjust volumes without messing with the encryption, or having 
more
encryption devices than really needed.  At any rate, I expect this setup to at
least work...)

> 
> > # Make cache set on second drive
> > # IMPORTANT:  Problem does not occur if I omit --bucket parameter.
> > make-bcache --bucket 2M -C /dev/sdb
> 
> 2MB is quite large, maybe it exceeds the 256-bvec limit.  I'm not sure if
> Ming Lei's patch got in to 4.6 yet, but try this:
>   https://lkml.org/lkml/2016/4/5/1046
> 
> and maybe Shaohua Li's patch too:
>   http://www.spinics.net/lists/raid/msg51830.html

Trying these is still on my TODO list (thus the belated reply here) but based
on the responses from Tim Small I'm doubtful this will fix anything, as it
sounds like he has the same problem (symptoms sound exactly the same) and he
says the patches didn't help.

Like Tim, I also chose a large bucket size because the manual page told me to.
Based on the high-level description of bcache and my knowledge of how flash
works, it certainly sounds necessary.

Perhaps the union of people who read manpages and people who use LUKS like
this is very small. :)

James

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH v4 03/21] fs: Allow sysfs and cgroupfs to share super blocks between user namespaces

2016-05-18 Thread Eric W. Biederman

Seth Forshee  writes:

> On Wed, May 18, 2016 at 10:45:31AM -0500, Eric W. Biederman wrote:
>> > But if we do that it violates some of the assumptions of the patch to
>> > rework MNT_NODEV on your testing branch (and also those behind patch 2
>> > in this series). Something will need to be changed there to prevent a
>> > regression in mount behavior when a user ns tries to mount without
>> > MNT_NODEV when the mount inherited from its parent has it set.
>> 
>> Thank you for pointing that out.  I will look into that.
>> 
>> I believe I know exactly what you are talking about.  Of the choices I
>> think it is better to a minor localized change in the fs_fully_visible
>> logic than it is to cause problems elsewhere.
>
> Agreed.
>
>> >> Apologies for not catching this earlier.
>> >
>> > Actually this is a more recent patch, so you possibly hadn't seen it
>> > before.
>> >
>> >> I am looking at folding all of this into the patch that introduces
>> >> sget_userns so that even bisects won't have regresssions.
>> >
>> > That's fine with me.
>> 
>> And thank you for keeping everything as separate patches.  That is at
>> least helping me catch up.  Even if I don't agree that these things
>> should be separate come merge time.
>
> Honestly I probably would have squashed some of them into that first
> patch myself if you hadn't already applied it to your testing branch, so
> that's all just luck.
>
> Keep in mind that I also have that patch for mqueue that isn't in this
> series, and I haven't yet checked to see if the 4.7 merges introduce
> anything which is going to require updating these patches. I was
> planning to wait and send out updates after -rc1, but if you want that
> stuff sooner just let me know.

As unfortunately I don't have anything going into -rc1 I am working on
this right now.

Let me finish sorting out the sget_userns and mnt nodev mess and I will
push something out and then we can compare notes.  I think I have mqueue
covered by other changes.  As it is in the set of filesystems that
should just use sget_userns.

I am sorting through the nodev corner of this now.  It should just be a
day or two.

Eric

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH v3 1/1] add display of map information in JSON format

2016-05-18 Thread Gris Ge

On Fri, May 13, 2016 at 10:39:00AM -0400, Todd Gill wrote:
>
Just found trivial issue:
> +#define PRINT_JSON_MAJOR_VERSION  0
> +#define PRINT_JSON_MINOR_VERSION  1
> +#define PRINT_JSON_START_VERSION  "   \"major_version \": %d,\n" \
> +  "   \"minor_version \": %d,\n"
Got extra space in json key name: 'major_version ' and
'minor_version ', should be 'major_version' and 'minor_version'

-- 
Gris Ge


signature.asc
Description: PGP signature
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

[dm-devel] [PATCH v4 1/1] add display of map information in JSON format

[dm-devel] [PATCH v4 0/1] add option to output JSON for multipathd command

Re: [dm-devel] bcache gets stuck flushing writeback cache when used in combination with LUKS/dm-crypt and non-default bucket size

Re: [dm-devel] [PATCH v4 03/21] fs: Allow sysfs and cgroupfs to share super blocks between user namespaces

Re: [dm-devel] [PATCH v3 1/1] add display of map information in JSON format

5 matches

Site Navigation

Mail list logo

Footer information