Re: [Qemu-devel] [PATCH v3 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Prasanna Kalever]

> On Tue, Sep 22, 2015 at 04:06:54 -0400, Prasanna Kalever wrote:
> > 
> > > On 09/21/2015 05:24 AM, Prasanna Kumar Kalever wrote:
> > > > This patch adds a way to specify multiple backup volfile servers to the
> > > > gluster
> > > > block backend of QEMU with tcp|rdma transport types and their port
> > > > numbers.
> > > > 
> > > 
> ter
> ...
> 
> > 
> > > > +##
> > > > +{ 'struct': 'BlockdevOptionsGluster',
> > > > +  'data': { 'volname': 'str',
> > > > +'image-path': 'str',
> > > > +'backup-volfile-servers': [ 'GlusterTuplePattern' ] } }
> > > 
> > > Shouldn't this be simply 'volfile-servers', as you are including the
> > > primary server in addition to the backup servers?
> > >
> > 
> > Again I want to maintain naming as mount.glusterfs do for fuse.
> 
> Well, I have to agree with Eric here. I think the option name doesn't
> need to be kept in sync with the gluster implementation since they don't
> share anything with qemu and since the array contains also the primary
> server to be queried the word backup doesn't make snese there.

Yes Peter, I have to agree with you and Eric.
I will rectify this changes in next patch-set, just waiting for 'gfapi' patch 
to get merged.

Thank you,

Best Regards, 
Prasanna Kumar Kalever.

> 
> Peter
> 

Re: [Qemu-devel] [PATCH v3 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Peter Krempa]

On Tue, Sep 22, 2015 at 04:06:54 -0400, Prasanna Kalever wrote:
> 
> > On 09/21/2015 05:24 AM, Prasanna Kumar Kalever wrote:
> > > This patch adds a way to specify multiple backup volfile servers to the
> > > gluster
> > > block backend of QEMU with tcp|rdma transport types and their port 
> > > numbers.
> > > 
> > 

...

> 
> > > +##
> > > +{ 'struct': 'BlockdevOptionsGluster',
> > > +  'data': { 'volname': 'str',
> > > +'image-path': 'str',
> > > +'backup-volfile-servers': [ 'GlusterTuplePattern' ] } }
> > 
> > Shouldn't this be simply 'volfile-servers', as you are including the
> > primary server in addition to the backup servers?
> >
> 
> Again I want to maintain naming as mount.glusterfs do for fuse.

Well, I have to agree with Eric here. I think the option name doesn't
need to be kept in sync with the gluster implementation since they don't
share anything with qemu and since the array contains also the primary
server to be queried the word backup doesn't make snese there.

Peter


signature.asc
Description: Digital signature

Re: [Qemu-devel] [PATCH v3 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Prasanna Kalever]

> On 09/21/2015 05:24 AM, Prasanna Kumar Kalever wrote:
> > This patch adds a way to specify multiple backup volfile servers to the
> > gluster
> > block backend of QEMU with tcp|rdma transport types and their port numbers.
> > 
> 
> > 
> > This patch gives a mechanism to provide all the server addresses which are
> > in
> > replica set, so in case server1 is down VM can still boot from any of the
> > active servers.
> > 
> > This is equivalent to the backup-volfile-servers option supported by
> > mount.glusterfs (FUSE way of mounting gluster volume)
> > 
> > This patch depends on a recent fix in libgfapi raised as part of this work:
> > http://review.gluster.org/#/c/12114/ (not merged yet)
> > 
> 
> It would be nice to get that merged before we take this in qemu.
> 
> > Credits: Sincere thanks to Kevin Wolf  and
> > "Deepak C Shetty"  for inputs and all their support
> > 
> 
> Up to here is good.

thank you

> 
> > v1:
> > multiple server addresses but common port number and transport type
> > pattern: URI syntax with query (?) delimitor
> > syntax:
> > file=gluster[+transport-type]://server1:24007/testvol/a.img\
> >  ?backup-volfile-servers=server2=server3
> 
> But the changelog information here should instead occur...

Agree, I will correct myself

> 
> 
> > 
> > Signed-off-by: Prasanna Kumar Kalever 
> > ---
> 
> ...here, so that it is helpful to reviewers but doesn't clog qemu.git
> history (a year from now, we won't care how many iterations a patch went
> through, only what got committed).
> 
> 
> > +++ b/qapi/block-core.json
> > @@ -1382,7 +1382,7 @@
> >'data': [ 'archipelago', 'blkdebug', 'blkverify', 'bochs', 'cloop',
> >  'dmg', 'file', 'ftp', 'ftps', 'host_cdrom', 'host_device',
> >  'host_floppy', 'http', 'https', 'null-aio', 'null-co',
> >  'parallels',
> > -'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'tftp', 'vdi',
> > 'vhdx',
> > +'qcow', 'qcow2', 'qed', 'quorum', 'gluster', 'raw', 'tftp',
> > 'vdi', 'vhdx',
> >  'vmdk', 'vpc', 'vvfat' ] }
> 
> Please keep this list sorted.  Also, missing documentation that
> 'gluster' was added in 2.5.
> 

Agree, I will change it

> >  
> >  ##
> > @@ -1794,6 +1794,43 @@
> >  '*read-pattern': 'QuorumReadPattern' } }
> >  
> >  ##
> > +# @GlusterTuplePattern
> 
> Name is long. Something like 'GlusterHost' would be nicer.
>

Hmm, I think rather than length, readability is important, Even 
'QuorumReadPattern' is also long, but gives  completeness.
Okay, If you still thing its better to have short name I can change it to 
"GlusterTuple"
 
> > +#
> > +# Gluster tuple pattern
> > +#
> > +# @transport:#transport type used to connect to gluster management
> > daemon
> > +# it can be tcp|rdma (default 'tcp')
> > +#
> > +# @port: port number on which glusterd is listening. (default
> > 24007)
> 
> To have a default, the parameter needs to be optional ('*port').
>

Thanks for correcting here, I really didn't knew this.
 
> > +#
> > +# @server:   server address (hostname/ipv4/ipv6 addresses)
> > +#
> > +# Since: 2.4+
> 
> s/2.4+/2.5/ (there is no 2.4+ release; the release that this will appear
> in is 2.5)
>

Okay.
 
> > +##
> > +{ 'struct': 'GlusterTuplePattern',
> > +  'data': { 'server': 'str',
> > +'transport': 'str',
> 
> Yuck. Please don't open-code this as a 'str', but instead define it as
> an enum:
> 
> { 'enum': 'GlusterTransport', 'data': [ 'tcp', 'rdma' ] }
> 
> > +'port': 'str' } }
> 
> Should port be an integer instead of a string? I guess we already allow
> named ports in other network-related interfaces, but it would still be
> nice to have an 'alternate' type that allows both integer and string.
> 
> It would be nice if your documentation of the parameters occurred in the
> same order as the parameters appear in the 'struct'.
> 

I expected, If I take it as an integer, while passing in json format
'json:{"driver":"qcow2","file":{"driver":"gluster","volname":"testvol",
 "image-path":"/path/a.qcow2","backup-volfile-servers":
 [{"server":"1.2.3.4","port":"24007","transport":"tcp"},
  {"server":"4.5.6.7","port":"24008","transport":"rdma"},] } }'
I should specify only "port":"24007" => "port":24007 (as integer)
Thanks for correcting me again.

I think the below should be enough

+##
+{ 'enum': 'GTransport', 'data': [ 'tcp', 'rdma' ] }

 ##
 # @GlusterTuplePattern
 #
@@ -1809,8 +1813,8 @@
 ##
 { 'struct': 'GlusterTuplePattern',
   'data': { 'server': 'str',
-'transport': 'str',
-'port': 'str' } }
+'*transport': 'GTransport',
+'*port': 'int' } }

Will do this.

> > +
> > +##
> > +# @BlockdevOptionsGluster
> > +#
> > +# Driver specific block device options for Gluster
> > +#
> > +# @volname:  name of gluster volume where our VM image resides
> > +#
> > +# 

[Qemu-devel] [PATCH v3 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Prasanna Kumar Kalever]

This patch adds a way to specify multiple backup volfile servers to the gluster
block backend of QEMU with tcp|rdma transport types and their port numbers.

Problem:

Currenly VM Image on gluster volume is specified like this:

file=gluster[+tcp]://server1[:port]/testvol/a.img

Assuming we have have three servers in trustred pool with replica 3 volume
in action and unfortunately server1 (mentioned in the command above) went down
for some reason, since the volume is replica 3 we now have other 2 servers
active from which we can boot the VM.

But currently there is no mechanism to pass the other 2 gluster server
addresses to qemu.

Solution:

New way of specifying VM Image on gluster volume with backup volfile servers:
(We still support old syntax to maintain backward compatibility)

Basic command line syntax looks like:

Pattern I:
 -drive driver=gluster,
volname=testvol,file.image-path=/path/a.raw,
backup-volfile-servers.0.server=1.2.3.4,
   [backup-volfile-servers.0.port=24007,]
   [backup-volfile-servers.0.transport=tcp,]
backup-volfile-servers.1.server=5.6.7.8,
   [backup-volfile-servers.1.port=24008,]
   [backup-volfile-servers.1.transport=rdma,] ...

Pattern II:
 'json:{"driver":"qcow2","file":{"driver":"gluster",
   "volname":"testvol","image-path":"/path/a.qcow2",
   "backup-volfile-servers":[{tuple0},{tuple1}, ...{tupleN}]}}'

   driver   => 'gluster' (protocol name)
   volname  => name of gluster volume where our VM image resides
   image-path   => is the absolute path of image in gluster volume

  {tuple}   => {"server":"1.2.3.4"[,"port":"24007","transport":"tcp"]}

   server   => server address (hostname/ipv4/ipv6 addresses)
   port => port number on which glusterd is listening. (default 24007)
   tranport => transport type used to connect to gluster management daemon,
it can be tcp|rdma (default 'tcp')

Examples:
1.
 -drive driver=qcow2,file.driver=gluster,
file.volname=testvol,file.image-path=/path/a.qcow2,
file.backup-volfile-servers.0.server=1.2.3.4,
file.backup-volfile-servers.0.port=24007,
file.backup-volfile-servers.0.transport=tcp,
file.backup-volfile-servers.1.server=5.6.7.8,
file.backup-volfile-servers.1.port=24008,
file.backup-volfile-servers.1.transport=rdma
2.
 'json:{"driver":"qcow2","file":{"driver":"gluster","volname":"testvol",
 "image-path":"/path/a.qcow2","backup-volfile-servers":
 [{"server":"1.2.3.4","port":"24007","transport":"tcp"},
  {"server":"4.5.6.7","port":"24008","transport":"rdma"},] } }'

This patch gives a mechanism to provide all the server addresses which are in
replica set, so in case server1 is down VM can still boot from any of the
active servers.

This is equivalent to the backup-volfile-servers option supported by
mount.glusterfs (FUSE way of mounting gluster volume)

This patch depends on a recent fix in libgfapi raised as part of this work:
http://review.gluster.org/#/c/12114/ (not merged yet)

Credits: Sincere thanks to Kevin Wolf  and
"Deepak C Shetty"  for inputs and all their support

v1:
multiple server addresses but common port number and transport type
pattern: URI syntax with query (?) delimitor
syntax:
file=gluster[+transport-type]://server1:24007/testvol/a.img\
 ?backup-volfile-servers=server2=server3

v2:
multiple server addresses each have their own port number, but all use
common transport type
pattern: URI syntax  with query (?) delimiter
syntax:
file=gluster[+transport-type]://[server[:port]]/testvol/a.img\
 [?backup-volfile-servers=server1[:port]\
  =server2[:port]]

v3:
multiple server addresses each have their own port number and transport type
pattern: changed to json
syntax:
'json:{"driver":"qcow2","file":{"driver":"gluster","volname":"testvol",
   "image-path":"/path/a.qcow2","backup-volfile-servers":
 [{"server":"1.2.3.4","port":"24007","transport":"tcp"},
  {"server":"4.5.6.7","port":"24008","transport":"rdma"},] } }'

Signed-off-by: Prasanna Kumar Kalever 
---
 block/gluster.c  | 412 +--
 qapi/block-core.json |  40 -
 2 files changed, 405 insertions(+), 47 deletions(-)

diff --git a/block/gluster.c b/block/gluster.c
index 1eb3a8c..48ae62e 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -11,6 +11,17 @@
 #include "block/block_int.h"
 #include "qemu/uri.h"
 
+#define MAX_PORTNUM_SIZE  6   /* max port num 65535 */
+
+#define GLUSTER_OPT_FILENAME  "filename"
+#define GLUSTER_OPT_VOLNAME   "volname"
+#define GLUSTER_OPT_IMAGE_PATH"image-path"
+#define GLUSTER_OPT_SERVER"server"
+#define GLUSTER_OPT_PORT  "port"
+#define GLUSTER_OPT_TRANSPORT "transport"
+#define GLUSTER_OPT_READ_PATTERN  "backup-volfile-servers."
+
+
 typedef struct 

Re: [Qemu-devel] [PATCH v3 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Eric Blake]

On 09/21/2015 05:24 AM, Prasanna Kumar Kalever wrote:
> This patch adds a way to specify multiple backup volfile servers to the 
> gluster
> block backend of QEMU with tcp|rdma transport types and their port numbers.
> 

> 
> This patch gives a mechanism to provide all the server addresses which are in
> replica set, so in case server1 is down VM can still boot from any of the
> active servers.
> 
> This is equivalent to the backup-volfile-servers option supported by
> mount.glusterfs (FUSE way of mounting gluster volume)
> 
> This patch depends on a recent fix in libgfapi raised as part of this work:
> http://review.gluster.org/#/c/12114/ (not merged yet)
> 

It would be nice to get that merged before we take this in qemu.

> Credits: Sincere thanks to Kevin Wolf  and
> "Deepak C Shetty"  for inputs and all their support
> 

Up to here is good.

> v1:
> multiple server addresses but common port number and transport type
> pattern: URI syntax with query (?) delimitor
> syntax:
> file=gluster[+transport-type]://server1:24007/testvol/a.img\
>  ?backup-volfile-servers=server2=server3

But the changelog information here should instead occur...


> 
> Signed-off-by: Prasanna Kumar Kalever 
> ---

...here, so that it is helpful to reviewers but doesn't clog qemu.git
history (a year from now, we won't care how many iterations a patch went
through, only what got committed).


> +++ b/qapi/block-core.json
> @@ -1382,7 +1382,7 @@
>'data': [ 'archipelago', 'blkdebug', 'blkverify', 'bochs', 'cloop',
>  'dmg', 'file', 'ftp', 'ftps', 'host_cdrom', 'host_device',
>  'host_floppy', 'http', 'https', 'null-aio', 'null-co', 
> 'parallels',
> -'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'tftp', 'vdi', 'vhdx',
> +'qcow', 'qcow2', 'qed', 'quorum', 'gluster', 'raw', 'tftp', 
> 'vdi', 'vhdx',
>  'vmdk', 'vpc', 'vvfat' ] }

Please keep this list sorted.  Also, missing documentation that
'gluster' was added in 2.5.

>  
>  ##
> @@ -1794,6 +1794,43 @@
>  '*read-pattern': 'QuorumReadPattern' } }
>  
>  ##
> +# @GlusterTuplePattern

Name is long. Something like 'GlusterHost' would be nicer.

> +#
> +# Gluster tuple pattern
> +#
> +# @transport:#transport type used to connect to gluster management daemon
> +# it can be tcp|rdma (default 'tcp')
> +#
> +# @port: port number on which glusterd is listening. (default 24007)

To have a default, the parameter needs to be optional ('*port').

> +#
> +# @server:   server address (hostname/ipv4/ipv6 addresses)
> +#
> +# Since: 2.4+

s/2.4+/2.5/ (there is no 2.4+ release; the release that this will appear
in is 2.5)

> +##
> +{ 'struct': 'GlusterTuplePattern',
> +  'data': { 'server': 'str',
> +'transport': 'str',

Yuck. Please don't open-code this as a 'str', but instead define it as
an enum:

{ 'enum': 'GlusterTransport', 'data': [ 'tcp', 'rdma' ] }

> +'port': 'str' } }

Should port be an integer instead of a string? I guess we already allow
named ports in other network-related interfaces, but it would still be
nice to have an 'alternate' type that allows both integer and string.

It would be nice if your documentation of the parameters occurred in the
same order as the parameters appear in the 'struct'.

> +
> +##
> +# @BlockdevOptionsGluster
> +#
> +# Driver specific block device options for Gluster
> +#
> +# @volname:  name of gluster volume where our VM image resides
> +#
> +# @image-path:   absolute path to image file in gluster volume
> +#
> +# @backup-volfile-servers: holds multiple tuples of {server, transport, port}
> +#
> +# Since: 2.4+

s/2.4+/2.5/

> +##
> +{ 'struct': 'BlockdevOptionsGluster',
> +  'data': { 'volname': 'str',
> +'image-path': 'str',
> +'backup-volfile-servers': [ 'GlusterTuplePattern' ] } }

Shouldn't this be simply 'volfile-servers', as you are including the
primary server in addition to the backup servers?

> +
> +##
>  # @BlockdevOptions
>  #
>  # Options for creating a block device.
> @@ -1814,6 +1851,7 @@
>'ftp':'BlockdevOptionsFile',
>'ftps':   'BlockdevOptionsFile',
>  # TODO gluster: Wait for structured options
> +  'gluster':'BlockdevOptionsGluster',
>'host_cdrom': 'BlockdevOptionsFile',
>'host_device':'BlockdevOptionsFile',
>'host_floppy':'BlockdevOptionsFile',
> 

There's also work on the list to expose NBD in a structured fashion; I'm
a bit worried that the two proposals might need to be sharing some
common functionality, rather than independently inventing slightly
different syntax.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature