Re: [ovirt-devel] UI Plugin to Upload ISO Files

2015-01-05 Thread Lucas Vandroux
Dear all,


Thanks a lot for all your feedback, it is helping us a lot. We will let you 
know about our progress.


Best regards,


- Lucas Vandroux 冯凯  

 
 
-- Original --
From: Itamar Heim; 
Date: 2014年12月16日(星期二) 上午7:08
To: Keith Robertson; Vandroux Lucas; devel; 
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files

 
On 12/15/2014 07:59 PM, Keith Robertson wrote:

 On 12/11/14 10:15 PM, Lucas Vandroux wrote:
 Dear all,

 I'm actually working to create a custom user interface plugin for oVirt
 web administration application to let user upload iso files.

 I'm in the very first stage of the project. I'm planning to use
 angularjs with the ng-flow module https://github.com/flowjs/ng-flow; on
 the client-side and a java servlet using the ovirt-iso-uploader
 http://www.ovirt.org/OVirt_engine_tools#ovirt-iso-uploadergt; engine tool
 on the server-side.

 All my code is going to be on Github in the following repository
 : iso-uploader-plugin
 https://github.com/ovirt-china/iso-uploader-plugin. You can also check
 a more detailed version of the specifications on my wiki
 https://github.com/ovirt-china/iso-uploader-plugin/wiki/Specificationsgt;.

 I'm writing to you guys to know if there is a way for us to collaborate
 as you may also want to develop something like this to be integrated in
 the oVirt Engine.

 Best regards,

 - Lucas Vandroux (冯凯)


 Lucas,

 I am glad to see someone finally picking this up.

 I don't maintain the ISO uploader anymore but I can offer you some
 suggestions that may help you with your effort.

 1) The mount command which is used by the 'ovirt-iso-uploader' requires
 root.  Hence, the calling process must be root.  If you shell out to the
 'ovirt-iso-uploader' from a servlet running in Ovirt Engine the calling
 process will not be root.  Instead, it will be the user 'ovirt'.  As
 such, uploading a file will fail because the mount command will fail.

 To circumvent this issue, I suggest you investigate creating a
 consolehelper script to call the program as the user 'ovirt' and elevate
 your privileges to root.  I would avoid sudo for this purpose.

 2) The plug-in you are creating will only work if Ovirt Engine can
 actually reach the NFS server.  AIUI, there are valid configurations
 where Hypervisors can reach the NFS server but the Engine cannot.  I am
 not sure if this is still the case but you will want to gracefully
 handle this.

 3) SSH is easier.  If you can SSH to the NFS server and upload the files
 that way then you will not have to shell out to the
 'ovirt-iso-uploader'.  You can use the Java SSH library that ships with
 oVirt.  I would suggest you review lines in [1] so that you understand
 the necessary file permissions.  It is important to note; however, that
 not all NFS server support SSH so this should not be the default
 uploading mechanism.


though as mentioned on the thread, we'd really love to see this done via 
upload to storage via vdsm communication.

 HTH,
 Keith

 [1] http://goo.gl/0ihn8W




 ___
 Devel mailing list
 Devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/devel

 ___
 Devel mailing list
 Devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2015-01-04 Thread Itamar Heim

On 01/02/2015 05:16 PM, Tony James wrote:

On Fri, Jan 2, 2015 at 2:44 AM, Itamar Heim ih...@redhat.com wrote:

On 12/29/2014 06:13 PM, Tony James wrote:


On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim ih...@redhat.com wrote:


On 12/29/2014 09:25 AM, Nir Soffer wrote:



- Original Message -



From: Tony James t...@anthonyjames.org
To: devel@ovirt.org
Sent: Monday, December 29, 2014 3:30:49 AM
Subject: [ovirt-devel] UI Plugin to Upload ISO Files

This message is in response to an earlier thread regarding a UI plugin
to upload ISO files.  Like the original poster, Lucas, I began work on
a UI plugin to allow uploading ISO files through a UI plugin.  After
reading the previous thread I'm re-thinking the architecture.

It was suggested that the recommended approach to upload files to a
storage domain is through the VDSM API [1].  I'm pretty familiar with
the oVirt REST API but have been unable to find documentation
regarding accessing the VDSM API.  Should the VDSM API be accessible
by a UI plugin? If so, is there documentation available to do so?

[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html




Basically you have to:
1. Use the vdsm xmlrpc/jsonrpc to create an image
2. Use the vdsm http api to upload the data to the image. This will
create
  a task and return a task id.
3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear
  the task when done

The xmlrpc/jsonrpc api is documented here:


http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.json;h=1edcda86c8468b68c620eff4844b57ca30e44ea7;hb=HEAD

You can check the code for upload here:


http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py;h=759ed7845e63658a13c139684095bd56c03a29ac;hb=HEAD#l158




I assume the upload will be done via a servlet on the engine, not
directly
by the ui plugin accessing vdsm.
worth discussing your plans here, to make sure architecture/security are
correct.



I was planning on using a python CGI script which would accept the
upload via POST from the UI plugin.  The file would be stored in /tmp
on the engine host.

After the file was successfully uploaded, the CGI script would send a
POST to a python HTTP server (BaseHTTPServer, also running on engine
host) with the filename and storage domain information.  This python
script would then take care of mounting the storage domain and copying
the file to the appropriate location.

This was my initial approach, I plan on checking out the VDSM API as well.



my preference would be to stream via a servlet to the vdsm api, rather than
store and forward to avoid potentially exhausting space on engine or
having to deal with two phased task tracking.

the tricky part which requires a review is validating authentication and
authorization by the servlet - to make sure one has the permission to write
to a certain disk (for data domains) / iso domain.
this should be similar to the websocket novnc approach of validating user
has access to relevant VM (but Alon may correct me if its different)

notice there is one caveat for iso domains to having vdsm do the upload vs.
the iso-uploader utility - it would require vdsm to have write permissions
to the iso nfs path. but it allows uploading disks/vm's as well to data
stores, which i think is worth having the same pattern for both.


Would it be sufficient to verify that the user has been given the
StorageAdmin role before allowing an upload?



1. lets see if we can design this as a full blown feature wrt how it 
should look like. I still believe it should have a servlet for the 
actual communication streaming to vdsm. of course, if you want to do 
something until this is provided properly, you can do that.


2. allon - thoughts wrt permissions? would be relevant to the general 
use case. question is if this should be limited to admins, then probably 
storage domain level permission is enough, or we'd like to check disk 
level permissions (for data disks) to allow users to upload vm's as 
well. my personal take is we can probably start with admins first 
which means storage domain level permission check should be enough to 
start with.

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [ovirt-devel] UI Plugin to Upload ISO Files

2015-01-02 Thread Alon Bar-Lev


- Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Tony James t...@anthonyjames.org
 Cc: Nir Soffer nsof...@redhat.com, Liron Aravot lara...@redhat.com, 
 devel@ovirt.org, Alon Bar-Lev
 alo...@redhat.com
 Sent: Friday, January 2, 2015 9:44:45 AM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 On 12/29/2014 06:13 PM, Tony James wrote:
  On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim ih...@redhat.com wrote:
  On 12/29/2014 09:25 AM, Nir Soffer wrote:
 
  - Original Message -
 
  From: Tony James t...@anthonyjames.org
  To: devel@ovirt.org
  Sent: Monday, December 29, 2014 3:30:49 AM
  Subject: [ovirt-devel] UI Plugin to Upload ISO Files
 
  This message is in response to an earlier thread regarding a UI plugin
  to upload ISO files.  Like the original poster, Lucas, I began work on
  a UI plugin to allow uploading ISO files through a UI plugin.  After
  reading the previous thread I'm re-thinking the architecture.
 
  It was suggested that the recommended approach to upload files to a
  storage domain is through the VDSM API [1].  I'm pretty familiar with
  the oVirt REST API but have been unable to find documentation
  regarding accessing the VDSM API.  Should the VDSM API be accessible
  by a UI plugin? If so, is there documentation available to do so?
 
  [1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html
 
 
  Basically you have to:
  1. Use the vdsm xmlrpc/jsonrpc to create an image
  2. Use the vdsm http api to upload the data to the image. This will
  create
   a task and return a task id.
  3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear
   the task when done
 
  The xmlrpc/jsonrpc api is documented here:
 
  http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.json;h=1edcda86c8468b68c620eff4844b57ca30e44ea7;hb=HEAD
 
  You can check the code for upload here:
 
  http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py;h=759ed7845e63658a13c139684095bd56c03a29ac;hb=HEAD#l158
 
 
  I assume the upload will be done via a servlet on the engine, not directly
  by the ui plugin accessing vdsm.
  worth discussing your plans here, to make sure architecture/security are
  correct.
 
 
  I was planning on using a python CGI script which would accept the
  upload via POST from the UI plugin.  The file would be stored in /tmp
  on the engine host.
 
  After the file was successfully uploaded, the CGI script would send a
  POST to a python HTTP server (BaseHTTPServer, also running on engine
  host) with the filename and storage domain information.  This python
  script would then take care of mounting the storage domain and copying
  the file to the appropriate location.
 
  This was my initial approach, I plan on checking out the VDSM API as well.
 
 
 my preference would be to stream via a servlet to the vdsm api, rather
 than store and forward to avoid potentially exhausting space on engine
 or having to deal with two phased task tracking.
 
 the tricky part which requires a review is validating authentication and
 authorization by the servlet - to make sure one has the permission to
 write to a certain disk (for data domains) / iso domain.
 this should be similar to the websocket novnc approach of validating
 user has access to relevant VM (but Alon may correct me if its different)

these requirements are for core product feature, while I guess Tony is after a 
solution to bridge the gap of the missing basic product feature within his 
environment. his solution is the most simple to solve his local problem, please 
assign core developer(s) to resolve the generic problem. Expecting a 
contributer to apply something the product was not designed for or to fix the 
entire product is incorrect.

we have long way to enable pluggable server side logic; we have no stable 
interface for authentication check (although we plan to have one for 3.6), we 
have no stable xmlrpc wrapper, not to mention that the xmlrpc should be 
obsoleted once the json rpc is out, we do not have access to dao to obtain 
information about entities, we do not have stable interface to engine key and 
track its usage, we do not have restapi extensions to extend server side 
functionality, I guess we do not have many more, as engine was never designed 
as modular extensible component.

we should also consider dropping the image upload to vdsm in favor of glance, 
so that core product will support glance directly either by embedding it or by 
delegating into it. this will enable us to retire the xmlrpc entirely as far as 
I understand, it will also enable to implement similar functionality of what 
initially requested within this thread.

Regards,
Alon
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [ovirt-devel] UI Plugin to Upload ISO Files

2015-01-02 Thread Tony James
On Fri, Jan 2, 2015 at 2:44 AM, Itamar Heim ih...@redhat.com wrote:
 On 12/29/2014 06:13 PM, Tony James wrote:

 On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim ih...@redhat.com wrote:

 On 12/29/2014 09:25 AM, Nir Soffer wrote:


 - Original Message -


 From: Tony James t...@anthonyjames.org
 To: devel@ovirt.org
 Sent: Monday, December 29, 2014 3:30:49 AM
 Subject: [ovirt-devel] UI Plugin to Upload ISO Files

 This message is in response to an earlier thread regarding a UI plugin
 to upload ISO files.  Like the original poster, Lucas, I began work on
 a UI plugin to allow uploading ISO files through a UI plugin.  After
 reading the previous thread I'm re-thinking the architecture.

 It was suggested that the recommended approach to upload files to a
 storage domain is through the VDSM API [1].  I'm pretty familiar with
 the oVirt REST API but have been unable to find documentation
 regarding accessing the VDSM API.  Should the VDSM API be accessible
 by a UI plugin? If so, is there documentation available to do so?

 [1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html



 Basically you have to:
 1. Use the vdsm xmlrpc/jsonrpc to create an image
 2. Use the vdsm http api to upload the data to the image. This will
 create
  a task and return a task id.
 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear
  the task when done

 The xmlrpc/jsonrpc api is documented here:


 http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.json;h=1edcda86c8468b68c620eff4844b57ca30e44ea7;hb=HEAD

 You can check the code for upload here:


 http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py;h=759ed7845e63658a13c139684095bd56c03a29ac;hb=HEAD#l158



 I assume the upload will be done via a servlet on the engine, not
 directly
 by the ui plugin accessing vdsm.
 worth discussing your plans here, to make sure architecture/security are
 correct.


 I was planning on using a python CGI script which would accept the
 upload via POST from the UI plugin.  The file would be stored in /tmp
 on the engine host.

 After the file was successfully uploaded, the CGI script would send a
 POST to a python HTTP server (BaseHTTPServer, also running on engine
 host) with the filename and storage domain information.  This python
 script would then take care of mounting the storage domain and copying
 the file to the appropriate location.

 This was my initial approach, I plan on checking out the VDSM API as well.


 my preference would be to stream via a servlet to the vdsm api, rather than
 store and forward to avoid potentially exhausting space on engine or
 having to deal with two phased task tracking.

 the tricky part which requires a review is validating authentication and
 authorization by the servlet - to make sure one has the permission to write
 to a certain disk (for data domains) / iso domain.
 this should be similar to the websocket novnc approach of validating user
 has access to relevant VM (but Alon may correct me if its different)

 notice there is one caveat for iso domains to having vdsm do the upload vs.
 the iso-uploader utility - it would require vdsm to have write permissions
 to the iso nfs path. but it allows uploading disks/vm's as well to data
 stores, which i think is worth having the same pattern for both.

Would it be sufficient to verify that the user has been given the
StorageAdmin role before allowing an upload?
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [ovirt-devel] UI Plugin to Upload ISO Files

2015-01-01 Thread Itamar Heim

On 12/29/2014 06:13 PM, Tony James wrote:

On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim ih...@redhat.com wrote:

On 12/29/2014 09:25 AM, Nir Soffer wrote:


- Original Message -


From: Tony James t...@anthonyjames.org
To: devel@ovirt.org
Sent: Monday, December 29, 2014 3:30:49 AM
Subject: [ovirt-devel] UI Plugin to Upload ISO Files

This message is in response to an earlier thread regarding a UI plugin
to upload ISO files.  Like the original poster, Lucas, I began work on
a UI plugin to allow uploading ISO files through a UI plugin.  After
reading the previous thread I'm re-thinking the architecture.

It was suggested that the recommended approach to upload files to a
storage domain is through the VDSM API [1].  I'm pretty familiar with
the oVirt REST API but have been unable to find documentation
regarding accessing the VDSM API.  Should the VDSM API be accessible
by a UI plugin? If so, is there documentation available to do so?

[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html



Basically you have to:
1. Use the vdsm xmlrpc/jsonrpc to create an image
2. Use the vdsm http api to upload the data to the image. This will create
 a task and return a task id.
3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear
 the task when done

The xmlrpc/jsonrpc api is documented here:

http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.json;h=1edcda86c8468b68c620eff4844b57ca30e44ea7;hb=HEAD

You can check the code for upload here:

http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py;h=759ed7845e63658a13c139684095bd56c03a29ac;hb=HEAD#l158



I assume the upload will be done via a servlet on the engine, not directly
by the ui plugin accessing vdsm.
worth discussing your plans here, to make sure architecture/security are
correct.



I was planning on using a python CGI script which would accept the
upload via POST from the UI plugin.  The file would be stored in /tmp
on the engine host.

After the file was successfully uploaded, the CGI script would send a
POST to a python HTTP server (BaseHTTPServer, also running on engine
host) with the filename and storage domain information.  This python
script would then take care of mounting the storage domain and copying
the file to the appropriate location.

This was my initial approach, I plan on checking out the VDSM API as well.



my preference would be to stream via a servlet to the vdsm api, rather 
than store and forward to avoid potentially exhausting space on engine 
or having to deal with two phased task tracking.


the tricky part which requires a review is validating authentication and 
authorization by the servlet - to make sure one has the permission to 
write to a certain disk (for data domains) / iso domain.
this should be similar to the websocket novnc approach of validating 
user has access to relevant VM (but Alon may correct me if its different)


notice there is one caveat for iso domains to having vdsm do the upload 
vs. the iso-uploader utility - it would require vdsm to have write 
permissions to the iso nfs path. but it allows uploading disks/vm's as 
well to data stores, which i think is worth having the same pattern for 
both.

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-29 Thread Itamar Heim

On 12/29/2014 09:25 AM, Nir Soffer wrote:

- Original Message -

From: Tony James t...@anthonyjames.org
To: devel@ovirt.org
Sent: Monday, December 29, 2014 3:30:49 AM
Subject: [ovirt-devel] UI Plugin to Upload ISO Files

This message is in response to an earlier thread regarding a UI plugin
to upload ISO files.  Like the original poster, Lucas, I began work on
a UI plugin to allow uploading ISO files through a UI plugin.  After
reading the previous thread I'm re-thinking the architecture.

It was suggested that the recommended approach to upload files to a
storage domain is through the VDSM API [1].  I'm pretty familiar with
the oVirt REST API but have been unable to find documentation
regarding accessing the VDSM API.  Should the VDSM API be accessible
by a UI plugin? If so, is there documentation available to do so?

[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html


Basically you have to:
1. Use the vdsm xmlrpc/jsonrpc to create an image
2. Use the vdsm http api to upload the data to the image. This will create
a task and return a task id.
3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear
the task when done

The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.json;h=1edcda86c8468b68c620eff4844b57ca30e44ea7;hb=HEAD

You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py;h=759ed7845e63658a13c139684095bd56c03a29ac;hb=HEAD#l158


I assume the upload will be done via a servlet on the engine, not 
directly by the ui plugin accessing vdsm.
worth discussing your plans here, to make sure architecture/security are 
correct.


___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-29 Thread Tony James
On Mon, Dec 29, 2014 at 5:26 AM, Itamar Heim ih...@redhat.com wrote:
 On 12/29/2014 09:25 AM, Nir Soffer wrote:

 - Original Message -

 From: Tony James t...@anthonyjames.org
 To: devel@ovirt.org
 Sent: Monday, December 29, 2014 3:30:49 AM
 Subject: [ovirt-devel] UI Plugin to Upload ISO Files

 This message is in response to an earlier thread regarding a UI plugin
 to upload ISO files.  Like the original poster, Lucas, I began work on
 a UI plugin to allow uploading ISO files through a UI plugin.  After
 reading the previous thread I'm re-thinking the architecture.

 It was suggested that the recommended approach to upload files to a
 storage domain is through the VDSM API [1].  I'm pretty familiar with
 the oVirt REST API but have been unable to find documentation
 regarding accessing the VDSM API.  Should the VDSM API be accessible
 by a UI plugin? If so, is there documentation available to do so?

 [1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html


 Basically you have to:
 1. Use the vdsm xmlrpc/jsonrpc to create an image
 2. Use the vdsm http api to upload the data to the image. This will create
 a task and return a task id.
 3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear
 the task when done

 The xmlrpc/jsonrpc api is documented here:

 http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.json;h=1edcda86c8468b68c620eff4844b57ca30e44ea7;hb=HEAD

 You can check the code for upload here:

 http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py;h=759ed7845e63658a13c139684095bd56c03a29ac;hb=HEAD#l158


 I assume the upload will be done via a servlet on the engine, not directly
 by the ui plugin accessing vdsm.
 worth discussing your plans here, to make sure architecture/security are
 correct.


I was planning on using a python CGI script which would accept the
upload via POST from the UI plugin.  The file would be stored in /tmp
on the engine host.

After the file was successfully uploaded, the CGI script would send a
POST to a python HTTP server (BaseHTTPServer, also running on engine
host) with the filename and storage domain information.  This python
script would then take care of mounting the storage domain and copying
the file to the appropriate location.

This was my initial approach, I plan on checking out the VDSM API as well.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


[ovirt-devel] UI Plugin to Upload ISO Files

2014-12-28 Thread Tony James
This message is in response to an earlier thread regarding a UI plugin
to upload ISO files.  Like the original poster, Lucas, I began work on
a UI plugin to allow uploading ISO files through a UI plugin.  After
reading the previous thread I'm re-thinking the architecture.

It was suggested that the recommended approach to upload files to a
storage domain is through the VDSM API [1].  I'm pretty familiar with
the oVirt REST API but have been unable to find documentation
regarding accessing the VDSM API.  Should the VDSM API be accessible
by a UI plugin? If so, is there documentation available to do so?

[1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html

-Tony
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-28 Thread Nir Soffer
- Original Message -
 From: Tony James t...@anthonyjames.org
 To: devel@ovirt.org
 Sent: Monday, December 29, 2014 3:30:49 AM
 Subject: [ovirt-devel] UI Plugin to Upload ISO Files
 
 This message is in response to an earlier thread regarding a UI plugin
 to upload ISO files.  Like the original poster, Lucas, I began work on
 a UI plugin to allow uploading ISO files through a UI plugin.  After
 reading the previous thread I'm re-thinking the architecture.
 
 It was suggested that the recommended approach to upload files to a
 storage domain is through the VDSM API [1].  I'm pretty familiar with
 the oVirt REST API but have been unable to find documentation
 regarding accessing the VDSM API.  Should the VDSM API be accessible
 by a UI plugin? If so, is there documentation available to do so?
 
 [1] http://lists.ovirt.org/pipermail/devel/2014-December/009497.html

Basically you have to:
1. Use the vdsm xmlrpc/jsonrpc to create an image
2. Use the vdsm http api to upload the data to the image. This will create
   a task and return a task id.
3. Use the vdsm xmlrpc/jsonrpc api to check the task status, and clear
   the task when done

The xmlrpc/jsonrpc api is documented here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/vdsmapi-schema.json;h=1edcda86c8468b68c620eff4844b57ca30e44ea7;hb=HEAD

You can check the code for upload here:
http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm/rpc/BindingXMLRPC.py;h=759ed7845e63658a13c139684095bd56c03a29ac;hb=HEAD#l158

Nir
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Nir Soffer
- Original Message -
 From: Alon Bar-Lev alo...@redhat.com
 To: Itamar Heim ih...@redhat.com
 Cc: devel devel@ovirt.org, Lucas Vandroux lucas.vandr...@eayun.com, 
 李建盛 jiansheng...@eayun.com, 潘礼洋
 liyang@eayun.com, Michal Skrivanek mskri...@redhat.com
 Sent: Sunday, December 14, 2014 11:47:26 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 
 
 - Original Message -
  From: Itamar Heim ih...@redhat.com
  To: Lucas Vandroux lucas.vandr...@eayun.com, devel devel@ovirt.org,
  潘礼洋 liyang@eayun.com, 李建盛
  jiansheng...@eayun.com
  Cc: Michal Skrivanek mskri...@redhat.com, Scott Herold
  sher...@redhat.com, Allon Mureinik
  amure...@redhat.com, Federico Simoncelli fsimo...@redhat.com, Barak
  Azulay bazu...@redhat.com, Brian
  Proffitt bprof...@redhat.com, Alon Bar-Lev alo...@redhat.com
  Sent: Sunday, December 14, 2014 11:35:40 PM
  Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
  
  On 12/12/2014 05:15 AM, Lucas Vandroux wrote:
   Dear all,
  
   I'm actually working to create a custom user interface plugin for oVirt
   web administration application to let user upload iso files.
  
   I'm in the very first stage of the project. I'm planning to use
   angularjs with the ng-flow module https://github.com/flowjs/ng-flow on
   the client-side and a java servlet using the ovirt-iso-uploader
   http://www.ovirt.org/OVirt_engine_tools#ovirt-iso-uploader engine tool
   on the server-side.
  
   All my code is going to be on Github in the following repository :
   iso-uploader-plugin
   https://github.com/ovirt-china/iso-uploader-plugin. You can also check
   a more detailed version of the specifications on my wiki
   https://github.com/ovirt-china/iso-uploader-plugin/wiki/Specifications.
  
   I'm writing to you guys to know if there is a way for us to collaborate
   as you may also want to develop something like this to be integrated in
   the oVirt Engine.
  
   Best regards,
  
   - Lucas Vandroux (冯凯)
  
  
  
  looks very nice.
  
  This is actually very interesting (and requested by multiple folks) but
  i'd like to see if we should focus on the more simple upload iso's or
  it doing more than just upload ISO's, but also VMs.
  for the latter, architecture would probably be a servlet[1] on the
  engine and stream to vdsm to write to storage, so both vm disks and/or
  iso's could be uploaded/downloaded.
 
 using ssh and/or nfs to send artifacts to hosts is something we should avoid
 so using iso/image uploader tools are not a solution.
 vdsm should support uploading images using its own protocol based on the
 authentication between engine and vdsm, is it already?

Vdsm does support upload over http/https directly to storage.

This feature is used to store ovf backups on storage domains, and probably
not very efficient, but may be good enough for now.

See vdsm/rpc/BindingXMLRPC.py (do_PUT)

 this should be a core feature not an add-on within the current monolithic
 implementation, as there is no access to vdsm interaction from any of the
 interfaces.

Nir
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Alon Bar-Lev


- Original Message -
 From: Nir Soffer nsof...@redhat.com
 To: Alon Bar-Lev alo...@redhat.com
 Cc: Itamar Heim ih...@redhat.com, devel devel@ovirt.org, Lucas 
 Vandroux lucas.vandr...@eayun.com, 李建盛
 jiansheng...@eayun.com, 潘礼洋 liyang@eayun.com, Michal Skrivanek 
 mskri...@redhat.com, Liron Aravot
 lara...@redhat.com
 Sent: Monday, December 15, 2014 6:47:44 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 - Original Message -
  From: Alon Bar-Lev alo...@redhat.com
  To: Itamar Heim ih...@redhat.com
  Cc: devel devel@ovirt.org, Lucas Vandroux lucas.vandr...@eayun.com,
  李建盛 jiansheng...@eayun.com, 潘礼洋
  liyang@eayun.com, Michal Skrivanek mskri...@redhat.com
  Sent: Sunday, December 14, 2014 11:47:26 PM
  Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
  
  
  
  - Original Message -
   From: Itamar Heim ih...@redhat.com
   To: Lucas Vandroux lucas.vandr...@eayun.com, devel
   devel@ovirt.org,
   潘礼洋 liyang@eayun.com, 李建盛
   jiansheng...@eayun.com
   Cc: Michal Skrivanek mskri...@redhat.com, Scott Herold
   sher...@redhat.com, Allon Mureinik
   amure...@redhat.com, Federico Simoncelli fsimo...@redhat.com,
   Barak
   Azulay bazu...@redhat.com, Brian
   Proffitt bprof...@redhat.com, Alon Bar-Lev alo...@redhat.com
   Sent: Sunday, December 14, 2014 11:35:40 PM
   Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
   
   On 12/12/2014 05:15 AM, Lucas Vandroux wrote:
Dear all,
   
I'm actually working to create a custom user interface plugin for oVirt
web administration application to let user upload iso files.
   
I'm in the very first stage of the project. I'm planning to use
angularjs with the ng-flow module https://github.com/flowjs/ng-flow
on
the client-side and a java servlet using the ovirt-iso-uploader
http://www.ovirt.org/OVirt_engine_tools#ovirt-iso-uploader engine
tool
on the server-side.
   
All my code is going to be on Github in the following repository :
iso-uploader-plugin
https://github.com/ovirt-china/iso-uploader-plugin. You can also
check
a more detailed version of the specifications on my wiki
https://github.com/ovirt-china/iso-uploader-plugin/wiki/Specifications.
   
I'm writing to you guys to know if there is a way for us to collaborate
as you may also want to develop something like this to be integrated in
the oVirt Engine.
   
Best regards,
   
- Lucas Vandroux (冯凯)
   
   
   
   looks very nice.
   
   This is actually very interesting (and requested by multiple folks) but
   i'd like to see if we should focus on the more simple upload iso's or
   it doing more than just upload ISO's, but also VMs.
   for the latter, architecture would probably be a servlet[1] on the
   engine and stream to vdsm to write to storage, so both vm disks and/or
   iso's could be uploaded/downloaded.
  
  using ssh and/or nfs to send artifacts to hosts is something we should
  avoid
  so using iso/image uploader tools are not a solution.
  vdsm should support uploading images using its own protocol based on the
  authentication between engine and vdsm, is it already?
 
 Vdsm does support upload over http/https directly to storage.
 
 This feature is used to store ovf backups on storage domains, and probably
 not very efficient, but may be good enough for now.
 
 See vdsm/rpc/BindingXMLRPC.py (do_PUT)

thanks.
I hear this function is not supported by the new jsonrpc, nor will it be 
supported when messaging will be used... so not sure if it is a good idea to 
add functionality on top of this one.
this should be core feature, it cannot be implemented as plugin within the 
current monolithic implementation.

 
  this should be a core feature not an add-on within the current monolithic
  implementation, as there is no access to vdsm interaction from any of the
  interfaces.
 
 Nir
 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Federico Simoncelli
- Original Message -
 From: Alon Bar-Lev alo...@redhat.com
 To: Nir Soffer nsof...@redhat.com
 Cc: devel devel@ovirt.org, 李建盛 jiansheng...@eayun.com, Lucas 
 Vandroux lucas.vandr...@eayun.com, Liron
 Aravot lara...@redhat.com, 潘礼洋 liyang@eayun.com, Michal 
 Skrivanek mskri...@redhat.com
 Sent: Monday, December 15, 2014 5:52:36 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 - Original Message -
  From: Nir Soffer nsof...@redhat.com
  To: Alon Bar-Lev alo...@redhat.com
  Cc: Itamar Heim ih...@redhat.com, devel devel@ovirt.org, Lucas
  Vandroux lucas.vandr...@eayun.com, 李建盛
  jiansheng...@eayun.com, 潘礼洋 liyang@eayun.com, Michal Skrivanek
  mskri...@redhat.com, Liron Aravot
  lara...@redhat.com
  Sent: Monday, December 15, 2014 6:47:44 PM
  Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
  
  - Original Message -
   From: Alon Bar-Lev alo...@redhat.com
   To: Itamar Heim ih...@redhat.com
   Cc: devel devel@ovirt.org, Lucas Vandroux
   lucas.vandr...@eayun.com,
   李建盛 jiansheng...@eayun.com, 潘礼洋
   liyang@eayun.com, Michal Skrivanek mskri...@redhat.com
   Sent: Sunday, December 14, 2014 11:47:26 PM
   Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
   
   
   using ssh and/or nfs to send artifacts to hosts is something we should
   avoid
   so using iso/image uploader tools are not a solution.
   vdsm should support uploading images using its own protocol based on the
   authentication between engine and vdsm, is it already?
  
  Vdsm does support upload over http/https directly to storage.
  
  This feature is used to store ovf backups on storage domains, and probably
  not very efficient, but may be good enough for now.
  
  See vdsm/rpc/BindingXMLRPC.py (do_PUT)
 
 thanks.
 I hear this function is not supported by the new jsonrpc, nor will it be
 supported when messaging will be used... so not sure if it is a good idea to
 add functionality on top of this one.

The format (xmlrpc/jsonrpc) here is not much interesting, the interesting
part is the transport. In fact current download/uploadImage uses REST
GET/PUT for downloading/uploading the images (only the errors/messages are
reported with xmlrpc or eventually they could be reported with jsonrpc).

By design they were not mixed with the other control APIs (because it's
not control, it's data). And indeed it cannot be transported with
messaging.

Uploading/dowloading data to/from the host involves a data transport
(and http REST is the most commonly used). Which is exactly what you
need here, and it was in fact designed for this use case.

 this should be core feature, it cannot be implemented as plugin within the
 current monolithic implementation.

+1

-- 
Federico
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Alon Bar-Lev


- Original Message -
 From: Federico Simoncelli fsimo...@redhat.com
 To: Alon Bar-Lev alo...@redhat.com
 Cc: Nir Soffer nsof...@redhat.com, devel devel@ovirt.org, 李建盛 
 jiansheng...@eayun.com, Lucas Vandroux
 lucas.vandr...@eayun.com, Liron Aravot lara...@redhat.com, 潘礼洋 
 liyang@eayun.com, Michal Skrivanek
 mskri...@redhat.com
 Sent: Monday, December 15, 2014 7:09:11 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 - Original Message -
  From: Alon Bar-Lev alo...@redhat.com
  To: Nir Soffer nsof...@redhat.com
  Cc: devel devel@ovirt.org, 李建盛 jiansheng...@eayun.com, Lucas
  Vandroux lucas.vandr...@eayun.com, Liron
  Aravot lara...@redhat.com, 潘礼洋 liyang@eayun.com, Michal
  Skrivanek mskri...@redhat.com
  Sent: Monday, December 15, 2014 5:52:36 PM
  Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
  
  - Original Message -
   From: Nir Soffer nsof...@redhat.com
   To: Alon Bar-Lev alo...@redhat.com
   Cc: Itamar Heim ih...@redhat.com, devel devel@ovirt.org, Lucas
   Vandroux lucas.vandr...@eayun.com, 李建盛
   jiansheng...@eayun.com, 潘礼洋 liyang@eayun.com, Michal
   Skrivanek
   mskri...@redhat.com, Liron Aravot
   lara...@redhat.com
   Sent: Monday, December 15, 2014 6:47:44 PM
   Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
   
   - Original Message -
From: Alon Bar-Lev alo...@redhat.com
To: Itamar Heim ih...@redhat.com
Cc: devel devel@ovirt.org, Lucas Vandroux
lucas.vandr...@eayun.com,
李建盛 jiansheng...@eayun.com, 潘礼洋
liyang@eayun.com, Michal Skrivanek mskri...@redhat.com
Sent: Sunday, December 14, 2014 11:47:26 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files


using ssh and/or nfs to send artifacts to hosts is something we should
avoid
so using iso/image uploader tools are not a solution.
vdsm should support uploading images using its own protocol based on
the
authentication between engine and vdsm, is it already?
   
   Vdsm does support upload over http/https directly to storage.
   
   This feature is used to store ovf backups on storage domains, and
   probably
   not very efficient, but may be good enough for now.
   
   See vdsm/rpc/BindingXMLRPC.py (do_PUT)
  
  thanks.
  I hear this function is not supported by the new jsonrpc, nor will it be
  supported when messaging will be used... so not sure if it is a good idea
  to
  add functionality on top of this one.
 
 The format (xmlrpc/jsonrpc) here is not much interesting, the interesting
 part is the transport. In fact current download/uploadImage uses REST
 GET/PUT for downloading/uploading the images (only the errors/messages are
 reported with xmlrpc or eventually they could be reported with jsonrpc).
 
 By design they were not mixed with the other control APIs (because it's
 not control, it's data). And indeed it cannot be transported with
 messaging.
 
 Uploading/dowloading data to/from the host involves a data transport
 (and http REST is the most commonly used). Which is exactly what you
 need here, and it was in fact designed for this use case.

there are plans to use messaging/broker to access hosts.
this should abstract the physical location of the host.
using direct connection to host in addition to broker is something that should 
be avoided.
once solution may be to obtain connection information from the control channel, 
but one of the problems that will nice to be solved is to stop initiating 
connections from manager.

 
  this should be core feature, it cannot be implemented as plugin within the
  current monolithic implementation.
 
 +1
 
 --
 Federico
 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Federico Simoncelli


- Original Message -
 From: Alon Bar-Lev alo...@redhat.com
 To: Federico Simoncelli fsimo...@redhat.com
 Cc: Nir Soffer nsof...@redhat.com, devel devel@ovirt.org, 李建盛 
 jiansheng...@eayun.com, Lucas Vandroux
 lucas.vandr...@eayun.com, Liron Aravot lara...@redhat.com, 潘礼洋 
 liyang@eayun.com, Michal Skrivanek
 mskri...@redhat.com
 Sent: Monday, December 15, 2014 6:15:53 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 
 
 - Original Message -
  From: Federico Simoncelli fsimo...@redhat.com
  To: Alon Bar-Lev alo...@redhat.com
  Cc: Nir Soffer nsof...@redhat.com, devel devel@ovirt.org, 李建盛
  jiansheng...@eayun.com, Lucas Vandroux
  lucas.vandr...@eayun.com, Liron Aravot lara...@redhat.com, 潘礼洋
  liyang@eayun.com, Michal Skrivanek
  mskri...@redhat.com
  Sent: Monday, December 15, 2014 7:09:11 PM
  Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
  
  - Original Message -
   From: Alon Bar-Lev alo...@redhat.com
   To: Nir Soffer nsof...@redhat.com
   Cc: devel devel@ovirt.org, 李建盛 jiansheng...@eayun.com, Lucas
   Vandroux lucas.vandr...@eayun.com, Liron
   Aravot lara...@redhat.com, 潘礼洋 liyang@eayun.com, Michal
   Skrivanek mskri...@redhat.com
   Sent: Monday, December 15, 2014 5:52:36 PM
   Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
   
   - Original Message -
From: Nir Soffer nsof...@redhat.com
To: Alon Bar-Lev alo...@redhat.com
Cc: Itamar Heim ih...@redhat.com, devel devel@ovirt.org, Lucas
Vandroux lucas.vandr...@eayun.com, 李建盛
jiansheng...@eayun.com, 潘礼洋 liyang@eayun.com, Michal
Skrivanek
mskri...@redhat.com, Liron Aravot
lara...@redhat.com
Sent: Monday, December 15, 2014 6:47:44 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files

- Original Message -
 From: Alon Bar-Lev alo...@redhat.com
 To: Itamar Heim ih...@redhat.com
 Cc: devel devel@ovirt.org, Lucas Vandroux
 lucas.vandr...@eayun.com,
 李建盛 jiansheng...@eayun.com, 潘礼洋
 liyang@eayun.com, Michal Skrivanek mskri...@redhat.com
 Sent: Sunday, December 14, 2014 11:47:26 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 
 using ssh and/or nfs to send artifacts to hosts is something we
 should
 avoid
 so using iso/image uploader tools are not a solution.
 vdsm should support uploading images using its own protocol based on
 the
 authentication between engine and vdsm, is it already?

Vdsm does support upload over http/https directly to storage.

This feature is used to store ovf backups on storage domains, and
probably
not very efficient, but may be good enough for now.

See vdsm/rpc/BindingXMLRPC.py (do_PUT)
   
   thanks.
   I hear this function is not supported by the new jsonrpc, nor will it be
   supported when messaging will be used... so not sure if it is a good idea
   to
   add functionality on top of this one.
  
  The format (xmlrpc/jsonrpc) here is not much interesting, the interesting
  part is the transport. In fact current download/uploadImage uses REST
  GET/PUT for downloading/uploading the images (only the errors/messages are
  reported with xmlrpc or eventually they could be reported with jsonrpc).
  
  By design they were not mixed with the other control APIs (because it's
  not control, it's data). And indeed it cannot be transported with
  messaging.
  
  Uploading/dowloading data to/from the host involves a data transport
  (and http REST is the most commonly used). Which is exactly what you
  need here, and it was in fact designed for this use case.
 
 there are plans to use messaging/broker to access hosts.
 this should abstract the physical location of the host.
 using direct connection to host in addition to broker is something that
 should be avoided.

Agreed, although when we discussed how to coherently move large data
from/to the hosts it became obvious that messaging/broker is not the
right tool.

 once solution may be to obtain connection information from the control
 channel, but one of the problems that will nice to be solved is to stop
 initiating connections from manager.

Retrieving/pushing the REST GET/PUT endpoints from the control path
was exactly what we had in mind. WRT the direction I have no preference
(even though at the moment it's from manager to host and for simplicity
I wouldn't change it right away).

I am not sure if the current implementation is misleading but the fact
that download/uploadImage are something that don't belong to the control
path (regular APIs) should be clear for everyone.

-- 
Federico
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Itamar Heim

On 12/16/2014 12:27 AM, Federico Simoncelli wrote:



- Original Message -

From: Itamar Heim ih...@redhat.com
To: Federico Simoncelli fsimo...@redhat.com, Alon Bar-Lev 
alo...@redhat.com
Cc: devel devel@ovirt.org, 李建盛 jiansheng...@eayun.com, Lucas Vandroux 
lucas.vandr...@eayun.com, Liron
Aravot lara...@redhat.com, 潘礼洋 liyang@eayun.com, Michal Skrivanek 
mskri...@redhat.com
Sent: Monday, December 15, 2014 10:34:51 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files

On 12/15/2014 07:33 PM, Federico Simoncelli wrote:



- Original Message -

From: Alon Bar-Lev alo...@redhat.com
To: Federico Simoncelli fsimo...@redhat.com
Cc: Nir Soffer nsof...@redhat.com, devel devel@ovirt.org, 李建盛
jiansheng...@eayun.com, Lucas Vandroux
lucas.vandr...@eayun.com, Liron Aravot lara...@redhat.com, 潘礼洋
liyang@eayun.com, Michal Skrivanek
mskri...@redhat.com
Sent: Monday, December 15, 2014 6:15:53 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files



- Original Message -

From: Federico Simoncelli fsimo...@redhat.com
To: Alon Bar-Lev alo...@redhat.com
Cc: Nir Soffer nsof...@redhat.com, devel devel@ovirt.org, 李建盛
jiansheng...@eayun.com, Lucas Vandroux
lucas.vandr...@eayun.com, Liron Aravot lara...@redhat.com, 潘礼洋
liyang@eayun.com, Michal Skrivanek
mskri...@redhat.com
Sent: Monday, December 15, 2014 7:09:11 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files

- Original Message -

From: Alon Bar-Lev alo...@redhat.com
To: Nir Soffer nsof...@redhat.com
Cc: devel devel@ovirt.org, 李建盛 jiansheng...@eayun.com, Lucas
Vandroux lucas.vandr...@eayun.com, Liron
Aravot lara...@redhat.com, 潘礼洋 liyang@eayun.com, Michal
Skrivanek mskri...@redhat.com
Sent: Monday, December 15, 2014 5:52:36 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files

- Original Message -

From: Nir Soffer nsof...@redhat.com
To: Alon Bar-Lev alo...@redhat.com
Cc: Itamar Heim ih...@redhat.com, devel devel@ovirt.org, Lucas
Vandroux lucas.vandr...@eayun.com, 李建盛
jiansheng...@eayun.com, 潘礼洋 liyang@eayun.com, Michal
Skrivanek
mskri...@redhat.com, Liron Aravot
lara...@redhat.com
Sent: Monday, December 15, 2014 6:47:44 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files

- Original Message -

From: Alon Bar-Lev alo...@redhat.com
To: Itamar Heim ih...@redhat.com
Cc: devel devel@ovirt.org, Lucas Vandroux
lucas.vandr...@eayun.com,
李建盛 jiansheng...@eayun.com, 潘礼洋
liyang@eayun.com, Michal Skrivanek mskri...@redhat.com
Sent: Sunday, December 14, 2014 11:47:26 PM
Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files


using ssh and/or nfs to send artifacts to hosts is something we
should
avoid
so using iso/image uploader tools are not a solution.
vdsm should support uploading images using its own protocol based on
the
authentication between engine and vdsm, is it already?


Vdsm does support upload over http/https directly to storage.

This feature is used to store ovf backups on storage domains, and
probably
not very efficient, but may be good enough for now.

See vdsm/rpc/BindingXMLRPC.py (do_PUT)


thanks.
I hear this function is not supported by the new jsonrpc, nor will it be
supported when messaging will be used... so not sure if it is a good
idea
to
add functionality on top of this one.


The format (xmlrpc/jsonrpc) here is not much interesting, the interesting
part is the transport. In fact current download/uploadImage uses REST
GET/PUT for downloading/uploading the images (only the errors/messages
are
reported with xmlrpc or eventually they could be reported with jsonrpc).

By design they were not mixed with the other control APIs (because it's
not control, it's data). And indeed it cannot be transported with
messaging.

Uploading/dowloading data to/from the host involves a data transport
(and http REST is the most commonly used). Which is exactly what you
need here, and it was in fact designed for this use case.


there are plans to use messaging/broker to access hosts.
this should abstract the physical location of the host.
using direct connection to host in addition to broker is something that
should be avoided.


Agreed, although when we discussed how to coherently move large data
from/to the hosts it became obvious that messaging/broker is not the
right tool.


once solution may be to obtain connection information from the control
channel, but one of the problems that will nice to be solved is to stop
initiating connections from manager.


Retrieving/pushing the REST GET/PUT endpoints from the control path
was exactly what we had in mind. WRT the direction I have no preference
(even though at the moment it's from manager to host and for simplicity
I wouldn't change it right away).

I am not sure if the current implementation is misleading but the fact
that download/uploadImage are something that don't belong to the control
path (regular APIs) should be clear for everyone.



iirc, the current verbs allow vdsm to download or upload, not the
engine. not sure

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Federico Simoncelli
- Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Federico Simoncelli fsimo...@redhat.com
 Cc: Alon Bar-Lev alo...@redhat.com, devel devel@ovirt.org, 李建盛 
 jiansheng...@eayun.com, Lucas Vandroux
 lucas.vandr...@eayun.com, Liron Aravot lara...@redhat.com, 潘礼洋 
 liyang@eayun.com, Michal Skrivanek
 mskri...@redhat.com
 Sent: Monday, December 15, 2014 11:32:47 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 On 12/16/2014 12:27 AM, Federico Simoncelli wrote:
 
 
  iirc, the current verbs allow vdsm to download or upload, not the
  engine. not sure if this will work with streaming the images via the
  engine, vs. having to 'store and forward' them.
 
  Using download would require 'store and forward'. On the other hand
  upload should support streaming because engine (or its separate component)
  would be just a (streaming) proxy between the client and vdsm.
 
 
 I thought 'upload' is vdsm pushing to say, glance. not accepting from
 engine?

Yeah it's confusing. Those are different upload/download verbs that
are not involving data to/from engine at all.

Basically those are regular tasks: upload or download an image to/from
an http server. They go through control path because data path is
between vdsm and the remote http server.

The REST api that we're discussing here instead is the (let's say)
glance-like interface that was designed exactly for this download
and upload image use-case from the calling endpoint (it is used for
the ovf files and I think hosted-engine is using it to inject the
engine appliance).

In mind we had streaming, not sure how much of that ended up in
real code (schedule was extremely tight at that time).

-- 
Federico
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-15 Thread Itamar Heim

On 12/15/2014 07:59 PM, Keith Robertson wrote:


On 12/11/14 10:15 PM, Lucas Vandroux wrote:

Dear all,

I'm actually working to create a custom user interface plugin for oVirt
web administration application to let user upload iso files.

I'm in the very first stage of the project. I'm planning to use
angularjs with the ng-flow module https://github.com/flowjs/ng-flow on
the client-side and a java servlet using the ovirt-iso-uploader
http://www.ovirt.org/OVirt_engine_tools#ovirt-iso-uploader engine tool
on the server-side.

All my code is going to be on Github in the following repository
: iso-uploader-plugin
https://github.com/ovirt-china/iso-uploader-plugin. You can also check
a more detailed version of the specifications on my wiki
https://github.com/ovirt-china/iso-uploader-plugin/wiki/Specifications.

I'm writing to you guys to know if there is a way for us to collaborate
as you may also want to develop something like this to be integrated in
the oVirt Engine.

Best regards,

- Lucas Vandroux (冯凯)



Lucas,

I am glad to see someone finally picking this up.

I don't maintain the ISO uploader anymore but I can offer you some
suggestions that may help you with your effort.

1) The mount command which is used by the 'ovirt-iso-uploader' requires
root.  Hence, the calling process must be root.  If you shell out to the
'ovirt-iso-uploader' from a servlet running in Ovirt Engine the calling
process will not be root.  Instead, it will be the user 'ovirt'.  As
such, uploading a file will fail because the mount command will fail.

To circumvent this issue, I suggest you investigate creating a
consolehelper script to call the program as the user 'ovirt' and elevate
your privileges to root.  I would avoid sudo for this purpose.

2) The plug-in you are creating will only work if Ovirt Engine can
actually reach the NFS server.  AIUI, there are valid configurations
where Hypervisors can reach the NFS server but the Engine cannot.  I am
not sure if this is still the case but you will want to gracefully
handle this.

3) SSH is easier.  If you can SSH to the NFS server and upload the files
that way then you will not have to shell out to the
'ovirt-iso-uploader'.  You can use the Java SSH library that ships with
oVirt.  I would suggest you review lines in [1] so that you understand
the necessary file permissions.  It is important to note; however, that
not all NFS server support SSH so this should not be the default
uploading mechanism.



though as mentioned on the thread, we'd really love to see this done via 
upload to storage via vdsm communication.



HTH,
Keith

[1] http://goo.gl/0ihn8W





___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel



___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] UI Plugin to Upload ISO Files

2014-12-14 Thread Lucas Vandroux
Dear all,

I'm actually working to create a custom user interface plugin for oVirt web 
administration application to let user upload iso files.

I'm in the very first stage of the project. I'm planning to use angularjs with 
the ng-flow module on the client-side and a java servlet using the 
ovirt-iso-uploader engine tool on the server-side.

All my code is going to be on Github in the following repository : 
iso-uploader-plugin. You can also check a more detailed version of the 
specifications on my wiki.

I'm writing to you guys to know if there is a way for us to collaborate as you 
may also want to develop something like this to be integrated in the oVirt 
Engine.

Best regards,

- Lucas Vandroux (冯凯)___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-14 Thread Itamar Heim

On 12/12/2014 05:15 AM, Lucas Vandroux wrote:

Dear all,

I'm actually working to create a custom user interface plugin for oVirt
web administration application to let user upload iso files.

I'm in the very first stage of the project. I'm planning to use
angularjs with the ng-flow module https://github.com/flowjs/ng-flow on
the client-side and a java servlet using the ovirt-iso-uploader
http://www.ovirt.org/OVirt_engine_tools#ovirt-iso-uploader engine tool
on the server-side.

All my code is going to be on Github in the following repository :
iso-uploader-plugin
https://github.com/ovirt-china/iso-uploader-plugin. You can also check
a more detailed version of the specifications on my wiki
https://github.com/ovirt-china/iso-uploader-plugin/wiki/Specifications.

I'm writing to you guys to know if there is a way for us to collaborate
as you may also want to develop something like this to be integrated in
the oVirt Engine.

Best regards,

- Lucas Vandroux (冯凯)




looks very nice.

This is actually very interesting (and requested by multiple folks) but 
i'd like to see if we should focus on the more simple upload iso's or 
it doing more than just upload ISO's, but also VMs.
for the latter, architecture would probably be a servlet[1] on the 
engine and stream to vdsm to write to storage, so both vm disks and/or 
iso's could be uploaded/downloaded.


one thing to consider with above architecture is need to handle 
ticketing between servlet to validate end user is allowed to do this, 
and for more advanced - some throttling.


another concern is choice of technologies to see if can be properly 
packaged to be provided as more than just an optional ui plugin.


a question - i couldn't easily locate a screenshot/explanation how your 
plugin dialog for upload looks like / how you handle authentication in 
it now?


thanks,
   Itamar

[1] servlet to can be moved to another host and to not hog engine 
backend itself.


___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] UI Plugin to Upload ISO Files

2014-12-14 Thread Alon Bar-Lev


- Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Lucas Vandroux lucas.vandr...@eayun.com, devel devel@ovirt.org, 
 潘礼洋 liyang@eayun.com, 李建盛
 jiansheng...@eayun.com
 Cc: Michal Skrivanek mskri...@redhat.com, Scott Herold 
 sher...@redhat.com, Allon Mureinik
 amure...@redhat.com, Federico Simoncelli fsimo...@redhat.com, Barak 
 Azulay bazu...@redhat.com, Brian
 Proffitt bprof...@redhat.com, Alon Bar-Lev alo...@redhat.com
 Sent: Sunday, December 14, 2014 11:35:40 PM
 Subject: Re: [ovirt-devel] UI Plugin to Upload ISO Files
 
 On 12/12/2014 05:15 AM, Lucas Vandroux wrote:
  Dear all,
 
  I'm actually working to create a custom user interface plugin for oVirt
  web administration application to let user upload iso files.
 
  I'm in the very first stage of the project. I'm planning to use
  angularjs with the ng-flow module https://github.com/flowjs/ng-flow on
  the client-side and a java servlet using the ovirt-iso-uploader
  http://www.ovirt.org/OVirt_engine_tools#ovirt-iso-uploader engine tool
  on the server-side.
 
  All my code is going to be on Github in the following repository :
  iso-uploader-plugin
  https://github.com/ovirt-china/iso-uploader-plugin. You can also check
  a more detailed version of the specifications on my wiki
  https://github.com/ovirt-china/iso-uploader-plugin/wiki/Specifications.
 
  I'm writing to you guys to know if there is a way for us to collaborate
  as you may also want to develop something like this to be integrated in
  the oVirt Engine.
 
  Best regards,
 
  - Lucas Vandroux (冯凯)
 
 
 
 looks very nice.
 
 This is actually very interesting (and requested by multiple folks) but
 i'd like to see if we should focus on the more simple upload iso's or
 it doing more than just upload ISO's, but also VMs.
 for the latter, architecture would probably be a servlet[1] on the
 engine and stream to vdsm to write to storage, so both vm disks and/or
 iso's could be uploaded/downloaded.

using ssh and/or nfs to send artifacts to hosts is something we should avoid so 
using iso/image uploader tools are not a solution.
vdsm should support uploading images using its own protocol based on the 
authentication between engine and vdsm, is it already?
this should be a core feature not an add-on within the current monolithic 
implementation, as there is no access to vdsm interaction from any of the 
interfaces.

Alon
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel