On Wed, Sep 26, 2012 at 05:38:17PM +0300, Constantinos Venetsanopoulos wrote: > Update the shared storage design document to reflect the current > changes, after the implementation of the ExtStorage interface. > > Signed-off-by: Constantinos Venetsanopoulos <[email protected]> > --- > doc/design-shared-storage.rst | 204 ++++++++++++++++++++++------------------ > 1 files changed, 112 insertions(+), 92 deletions(-) > > diff --git a/doc/design-shared-storage.rst b/doc/design-shared-storage.rst > index c175476..7080182 100644 > --- a/doc/design-shared-storage.rst > +++ b/doc/design-shared-storage.rst > @@ -64,15 +64,11 @@ The design addresses the following procedures: > filesystems. > - Introduction of shared block device disk template with device > adoption. > +- Introduction of an External Storage Interface. > > Additionally, mid- to long-term goals include: > > - Support for external “storage pools”. > -- Introduction of an interface for communicating with external scripts, > - providing methods for the various stages of a block device's and > - instance's life-cycle. In order to provide storage provisioning > - capabilities for various SAN appliances, external helpers in the form > - of a “storage driver” will be possibly introduced as well. > > Refactoring of all code referring to constants.DTS_NET_MIRROR > ============================================================= > @@ -159,6 +155,104 @@ The shared block device template will make the > following assumptions: > - The device will be available with the same path under all nodes in the > node group. > > +Introduction of an External Storage Interface > +============================================== > +Overview > +-------- > + > +To extend the shared block storage template and give Ganeti the ability > +to control and manipulate external storage (provisioning, removal, > +growing, etc.) we need a more generic approach. The generic method for > +supporting external shared storage in Ganeti will be to have an > +ExtStorage provider for each external shared storage hardware type. The > +ExtStorage provider will be a set of files (executable scripts and text > +files), contained inside a directory which will be named after the > +provider. This directory must be present across all nodes of a nodegroup > +(Ganeti doesn't replicate it), in order for the provider to be usable by > +Ganeti for this nodegroup (valid).
How will Ganeti behave if they are not consistent? Report errors? (in cluster verify?) Ignore the provider? Etc. > The external shared storage hardware > +should also be accessible by all nodes of this nodegroup too. > + > +An “ExtStorage provider” will have to provide the following methods: > + > +- Create a disk > +- Remove a disk > +- Grow a disk > +- Attach a disk to a given node > +- Detach a disk from a given node > +- Verify its supported parameters > + > +The proposed ExtStorage interface borrows heavily from the OS > +interface and follows a one-script-per-function approach. An ExtStorage > +provider is expected to provide the following scripts: > + > +- `create` > +- `remove` > +- `grow` > +- `attach` > +- `detach` > +- `verify` > + > +All scripts will be called with no arguments and get their input via > +environment variables. A common set of variables will be exported for > +all commands, and some of them might have extra ones. > + > +- `VOL_NAME`: The name of the volume. This is unique for Ganeti and it > + uses it to refer to a specific volume inside the external storage. > +- `VOL_SIZE`: The volume's size in mebibytes. > +- `VOL_NEW_SIZE`: Available only to the `grow` script. It declares the > + new size of the volume after grow (in mebibytes). > +- `EXTP_name`: ExtStorage parameter, where `name` is the parameter in > + upper-case (same as OS interface's `OSP_*` parameters). > + > +All scripts except `attach` should return 0 on success and non-zero on > +error, accompanied by an appropriate error message on stderr. The > +`attach` script should return a string on stdout on success, which is > +the block device's full path, after it has been successfully attached to > +the host node. On error it should return non-zero. > + > +Implementation > +-------------- > + > +To support the ExtStorage interface, we will introduce a new disk > +template called `ext`. This template will implement the existing Ganeti > +disk interface in `lib/bdev.py` (create, remove, attach, assemble, > +shutdown, grow), and will simultaneously pass control to the external > +scripts to actually handle the above actions. The `ext` disk template > +will act as a translation layer between the current Ganeti disk > +interface and the ExtStorage providers. > + > +We will also introduce a new IDISK_PARAM called `IDISK_PROVIDER = > +provider`, which will be used at the command line to select the desired > +ExtStorage provider. This parameter will be valid only for template > +`ext` e.g.:: > + > + gnt-instance add -t ext --disk=0:size=2G,provider=sample_provider1 > + > +The Extstorage interface will support different disks to be created by > +different providers. e.g.:: > + > + gnt-instance add -t ext --disk=0:size=2G,provider=sample_provider1 > + --disk=1:size=1G,provider=sample_provider2 > + --disk=2:size=3G,provider=sample_provider1 This (also in the context of your other design changes) makes me a bit uneasy, with regards to coordinating changes across multiple providers in live migration and similar changes (even startup). Have you thought about this? > +Finally, the ExtStorage interface will support passing of parameters to > +the ExtStorage provider. This will also be done per disk, from the > +command line:: > + > + gnt-instance add -t ext --disk=0:size=1G,provider=sample_provider1, > + param1=value1,param2=value2 > + > +The above parameters will be exported to the ExtStorage provider's > +scripts as the enviromental variables: > + > +- `EXTP_PARAM1 = str(value1)` > +- `EXTP_PARAM2 = str(value2)` > + > +We will also introduce a new Ganeti client called `gnt-storage` which > +will be used to diagnose ExtStorage providers and show information about > +them, similarly to the way `gnt-os diagose` and `gnt-os info` handle OS > +definitions. Hmm… you got me here (I was hoping to avoid new python-based CLI front-ends, but it's too early for the switch ;-). Except for the above two small comments, LGTM, thanks. iustin
