On 09/17/2013 07:33 PM, Zhi Yan Liu wrote:
On Wed, Sep 18, 2013 at 6:16 AM, Mike Perez <thin...@gmail.com> wrote:
Folks,
Currently in Havana development, RBD as ephemeral storage has serious
stability
and performance issues that makes the Ceph cluster a bottleneck for using an
image as a source.
Nova has to currently communicate with the external service Glance, which
has
to talk to the separate Ceph storage backend to fetch path information. The
entire image is then downloaded to local disk, and then imported from local
disk to RBD. This leaves a stability concern, especially with large images
for
the instance to be successfully created, due to unnecessary data pulling and
pushing for solutions like RBD.
Due to the fact we have to do a import from local disk to RBD, this can make
performance even slower than a normal backend filesystem since the import is
single threaded.
This can be eliminated by instead having Nova's RBD image backend utility
communicate directly with the Ceph backend to do a copy-on-write of the
image.
Not only does this greatly improve stability, but performance is drastically
improved by not having to do a full copy of the image. A lot of the code to
make this happen came from the RBD Cinder driver which has been stable and
merged for quite a while.
Bug: https://code.launchpad.net/bugs/1226351
Patch: https://review.openstack.org/#/c/46879/1
Thanks,
Mike Perez
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Hi Mike Perez, folks,
I absolutely agree use zero-copy approach to prepare template image is
a good idea, such as CoW. But after check your patch I have some
concerns on the currently implementation.
Actually I had prepared some dedicated BPs [1][2] and a patch [3] to
cover such requirements and problems around zero-copy (aka your
'direct_fetch') image preparing, it been implemented as a framework
and allow other people realize such plug-ins for a particular image
storage backend/location. So I'd very like to invite you (and Josh
Durgin) to take a look on them, I believe (and welcome) your stuff
within #46879 around RBD image handling can be implemented as a
RBDImangeHandler plug-ins under my framework.
I like the plugin framework you've created there, but it was not
merged for Havana, which is why I did not use it in this patch. I
didn't want to include more new code than necessary. I simply used the
same approach that cinder uses to minimize the risk of regression.
I consider above implementation is better, since framework code within
#33409 can handle most common logic such as plug-ins loading, image
handler selecting base on image location, image multiple location
supporting and etc.. And each particular image handler can just
implement such special methods easily and don't need to rebuild the
existed (and tested) part.
I'm fine with that approach, and the direct fetch method in my patch
can be easily refactored into an RBDImageHandler like you suggest with
a little bit more information passed to the image handlers. I'm happy
to help with this for Icehouse.
Thanks!
Josh
Of cause, with the production of new handlers we probably need add
more interfaces and pass more context data to the structure /
ImageHandler base class as needed, we can talk this in irc.
[1] https://blueprints.launchpad.net/nova/+spec/image-multiple-location
[2]
https://blueprints.launchpad.net/nova/+spec/effective-template-base-image-preparing
[3] https://review.openstack.org/#/c/33409/
thanks,
zhiyan
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
