Hi All,
Before heading into holiday, I'd like to update the current status of the
new storage framework since last collab12.
1. Class diagram of primary storage is evolved:
https://cwiki.apache.org/confluence/download/attachments/30741569/storage.jpg?version=1&modificationDate=1356640617613
Highlight the current design:
a. One storage provider can cover multiple storage protocols for
multiple hypervisors. The default storage provider can almost cover all the
current primary storage protocols. In most of cases, you don't need to write a
new storage provider, what you need to do is to write a new storage
configurator. Write a new storage provider needs to write a lot of code, which
we should avoid it as much as possible.
b. A new type hierarchy, primaryDataStoreConfigurator, is added. The
configurator is a factory for primaryDataStore, which assemble
StorageProtocolTransformer, PrimaryDataStoreLifeCycle and
PrimaryDataStoreDriver for PrimaryDataStore object, based on the hypervisor
type and the storage protocol. For example, for nfs primary storage on
xenserver, there is a class called XenNfsConfigurator, which put
DefaultXenPrimaryDataStoreLifeCycle, NfsProtocolTransformer and
DefaultPrimaryDataStoreDriverImpl into DefaultPrimaryDataStore. One provider
can only have one configurator for a pair of hypervisor type and storage
protocol. For example, if you want to add a new nfs protocol configurator for
xenserver hypervisor, you need to write a new storage provider.
c. A new interface, StorageProtocolTransformer, is added. The main
purpose of this interface is to handle the difference between different storage
protocols. It has four methods:
getInputParamNames: return a list of name of parameters for a
particular protocol. E.g. NFS protocol has ["server", "path"], ISCSI has
["iqn", "lun"] etc. UI shouldn't hardcode these parameters any more.
normalizeUserInput: given a user input from UI/API, need to
validate the input, and break it apart, then store them into database
getDataStoreTO/ getVolumeTO: each protocol can have its own
volumeTO and primaryStorageTO. TO is the object will be passed down to
resource, if your storage has extra information you want to pass to resource,
these two methods are the place you can override.
d. All the time-consuming API calls related to storage is async.
2. Minimal functionalities are implemented:
a. Can register a http template, without SSVM
b. Can register a NFS primary storage for xenserver
c. Can download a template into primary storage directly
d. Can create a volume from a template
3. All about test:
a. TestNG test framework is used, as it can provide parameter for
each test case. For integration test, we need to know ip address of hypervisor
host, the host uuid(if it's xenserver), the primary storage url, the template
url etc. These configurations are better to be parameterized, so for each test
run, we don't need to modify test case itself, instead, we provide a test
configuration file for each test run. TestNG framework already has this
functionality, I just reuse it.
b. Every pieces of code can be unit tested, which means:
b.1 the xcp plugin can be unit tested. I wrote a small python
code, called mockxcpplugin.py, which can directly call xcp plugin.
b.2 direct agent hypervisor resource can be tested. I wrote a
mock agent manger, which can load and initialize hypervisor resource, and also
can send command to resource.
b.3 a storage integration test maven project is created, which
can test the whole storage subsystem, such as create volume from template,
which including both image and volume components.
A new section, called "how to test", is added into
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+subsystem+2.0,
please check it out.
The code is on the javelin branch, the maven projects whose name starting
from cloud-engine-storage-* are the code related to storage subsystem. Most of
the primary storage code is in cloud-engine-storage-volume project.
Any feedback/comment is appreciated.
