Re: Poor performance with open nebula driver

[Daniel Molina]

Hi Felix

On 25 June 2012 05:29, Felix Lee <fe...@twgrid.org> wrote:
> Dear All,
> I am new with deltacloud, for integration requirement, we are evaluating
> deltacloud solution.
> Here, I don't know if anybody ever reported this issue, however, recently, I
> noticed deltacloud is in poor performance with our open nebula instance.
> Usually, our opennebula always runs over 500 VM instances, and while we use
> deltacloud to query instances, it will take pretty long time to get all
> results.
>
>
> Here are some examples:
>
> With "deltacloudc instances" command:
>
> real    20m19.293s
> user    0m7.842s
> sys     0m0.681s
>
>
> With curl to deltacloud rest api "/api/instances":
> real    4m35.891s
> user    0m0.002s
> sys     0m0.005s
>
> With curl to opennebula OCCI interface "/compute" directly via the same
> host:
> real    0m1.249s
> user    0m0.001s
> sys     0m0.006s
>

In OpenNebula 3.2 when you retrieve the pool of instances through OCCI
no extended information is included, hence you have to do one more
request per instance to retrieve this information. In OpenNebula 3.4 a
verbose option was included for OCCI pools. You can use GET
/compute?verbose=true instead of GET /compute and the pool will
contain extended information for each element.

I have included a diff with the changes that would be required to
support this param in the deltacloud driver, but I didn't test it.
This should fix your performance issues.

Also using Nokogiri instead of REXML should improve the driver performance.

Hope this helps


diff --git a/server/lib/deltacloud/drivers/opennebula/occi_client.rb
b/server/lib/deltacloud/drivers/op
index 8688995..7d71962 100644
--- a/server/lib/deltacloud/drivers/opennebula/occi_client.rb
+++ b/server/lib/deltacloud/drivers/opennebula/occi_client.rb
@@ -83,8 +83,8 @@ module OCCIClient
     ######################################################################
     # Retieves the pool of Virtual Machines
     ######################################################################
-    def get_vms
-      get('/compute')
+    def get_vms(verbose=false)
+      get('/compute', verbose)
     end

     ######################################################################
@@ -196,8 +196,8 @@ module OCCIClient
     ######################################################################
     # Retieves the pool of Images owned by the user
     ######################################################################
-    def get_images
-      get('/storage')
+    def get_images(verbose=false)
+      get('/storage', verbose)
     end


@@ -275,10 +275,16 @@ module OCCIClient

     private

-    def get(path)
+    def get(path, verbose=false)
       url = URI.parse(@endpoint+path)
+
+      params = []
+      params << "verbose=true" if verbose
+      params << "#{url.query}" if url.query
+
       path = url.path
-      path << "?#{url.query}" if url.query
+      path << "?#{params.join('&')}"
+
       req = Net::HTTP::Get.new(path)

       do_request(url, req)
diff --git a/server/lib/deltacloud/drivers/opennebula/opennebula_driver.rb
b/server/lib/deltacloud/driv
index cc1f13e..7d2385f 100644
--- a/server/lib/deltacloud/drivers/opennebula/opennebula_driver.rb
+++ b/server/lib/deltacloud/drivers/opennebula/opennebula_driver.rb
@@ -84,13 +84,11 @@ class OpennebulaDriver < Deltacloud::BaseDriver
   def images(credentials, opts=nil)
     occi_client = new_client(credentials)

-    xml = treat_response(occi_client.get_images)
+    xml = treat_response(occi_client.get_images(true))

     # TBD Add extended info in the pool
     images = REXML::Document.new(xml).root.elements.map do |d|
-      im_id = d.attributes['href'].split("/").last
-      storage = treat_response(occi_client.get_image(im_id))
-      convert_image(storage, credentials)
+      convert_image(d, credentials)
     end
   end

@@ -156,13 +154,11 @@ class OpennebulaDriver < Deltacloud::BaseDriver
   def instances(credentials, opts=nil)
     occi_client = new_client(credentials)

-    xml = treat_response(occi_client.get_vms)
+    xml = treat_response(occi_client.get_vms(true))

     # TBD Add extended info in the pool
     instances = REXML::Document.new(xml).root.elements.map do |d|
-      vm_id = d.attributes['href'].split("/").last
-      computexml = treat_response(occi_client.get_vm(vm_id))
-      convert_instance(computexml, credentials)
+      convert_instance(d, credentials)
     end

     instances = filter_on( instances, :state, opts )

-- 
Daniel Molina
Project Engineer
OpenNebula - The Open Source Solution for Data Center Virtualization
www.OpenNebula.org | dmol...@opennebula.org | @OpenNebula