Re: [ovirt-users] Disk image upload pausing

[spfma . tech]

Hi,
   I get this :ImageProxyAddress:hosted-ovirt-engine.domain.loc:54323 
version: general
   Regards 

 Le 21-Feb-2018 10:16:23 +0100, gianluca.cec...@gmail.com a crit:   
   On Tue, Feb 20, 2018 at 2:33 PM,  wrote:

 Hi,
   Here are lines I have found for my last faulty try : Just to be sure 
it is not a problem I got before and inherited by an update from a previous 
version (btw: version and history of this install?): if you execute this on 
engine what do you get   engine-config -g ImageProxyAddress   Gianluca 

-
FreeMail powered by mail.fr
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Disk image upload pausing

[Gianluca Cecchi]

On Tue, Feb 20, 2018 at 2:33 PM,  wrote:

> Hi,
>
> Here are lines I have found for my last faulty try :
>
>
Just to be sure it is not a problem I got before and inherited by an update
from a previous version (btw: version and history of this install?):
if you execute this on engine what do you get

engine-config -g ImageProxyAddress

Gianluca
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Disk image upload pausing

[Nir Soffer]

The vdsm and ovirt-imageio-daemon logs on the host selected for the upload
can help to understand this issue.

Nir

בתאריך יום ד׳, 21 בפבר׳ 2018, 7:10, מאת Idan Shaby ‏:

> On Tue, Feb 20, 2018 at 3:33 PM,  wrote:
>
>> Hi,
>>
>> Here are lines I have found for my last faulty try :
>>
>> ENGINE
>> 2018-02-19 17:52:27,283+01 INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand]
>> (EE-ManagedThreadFactory-engineScheduled-Thread-3) [1320afb0] Running
>> command: TransferImageStatusCommand internal: true. Entities affected :
>> ID: aaa0----123456789aaa Type: SystemAction group
>> CREATE_DISK with role type USER
>> 2018-02-19 17:52:27,290+01 INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand]
>> (EE-ManagedThreadFactory-engineScheduled-Thread-3) [1320afb0] Lock freed to
>> object 'EngineLock:{exclusiveLocks='', sharedLocks='[]'}'
>> 2018-02-19 17:52:28,658+01 INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand]
>> (default task-14) [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] Running command:
>> TransferImageStatusCommand internal: false. Entities affected :  ID:
>> aaa0----123456789aaa Type: SystemAction group CREATE_DISK
>> with role type USER
>> 2018-02-19 17:52:28,659+01 INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.ImageTransferUpdater]
>> (default task-14) [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] Updating image
>> transfer 1c55d561-45bf-4e57-b3b6-8fcbf3734a28 (image
>> af5997c5-ae69-4677-9d86-30a978cf83a5) phase to Paused by System (message:
>> 'Sent 405200MB')
>> 2018-02-19 17:52:28,665+01 WARN
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> (default task-14) [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] EVENT_ID:
>> UPLOAD_IMAGE_NETWORK_ERROR(1,038), Unable to upload image to disk
>> af5997c5-ae69-4677-9d86-30a978cf83a5 due to a network error. Make sure
>> ovirt-imageio-proxy service is installed and configured, and ovirt-engine's
>> certificate is registered as a valid CA in the browser. The certificate can
>> be fetched from https://
>> /ovirt-engine/services/pki-resource?resource=ca-certificate=X509-PEM-CA
>> 2018-02-19 17:52:32,624+01 INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand]
>> (default task-28) [0eba65e6-cca8-46ee-9038-fb29838ead47] Running command:
>> TransferImageStatusCommand internal: false. Entities affected :  ID:
>> aaa0----123456789aaa Type: SystemAction group CREATE_DISK
>> with role type USER
>> 2018-02-19 17:52:36,679+01 INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand]
>> (default task-16) [5d662615-a4e7-412b-8ecf-45be03c7e49f] Running command:
>> TransferImageStatusCommand internal: false. Entities affected :  ID:
>> aaa0----123456789aaa Type: SystemAction group CREATE_DISK
>> with role type USER
>> 2018-02-19 17:52:37,304+01 INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
>> (EE-ManagedThreadFactory-engineScheduled-Thread-40)
>> [6f21de96-d3bd-4cf0-b922-394be7389d3b] Transfer was paused by system.
>> Upload disk 'pfm-serv-pdc_Disk1' (id '----')
>>
>> PROXY
>> (Thread-4087) ERROR 2018-02-19 17:52:28,644
>> images:143:root:(make_imaged_request) Failed communicating with host: A
>> Connection error occurred.
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py",
>> line 134, in make_imaged_request
>> timeout=timeout, stream=stream)
>>   File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576,
>> in send
>> r = adapter.send(request, **kwargs)
>>   File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415,
>> in send
>> raise ConnectionError(err, request=request)
>> ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))
>> (Thread-4087) ERROR 2018-02-19 17:52:28,645 web:112:web:(log_error) ERROR
>> [10.100.0.184] PUT /images/f64acb43-d153-485d-b441-9f5d42773a03: [503]
>> Failed communicating with host: A Connection error occurred. (0.01s)
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_common/web.py",
>> line 64, in __call__
>> resp = self.dispatch(request)
>>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_common/web.py",
>> line 91, in dispatch
>> return method(*match.groups())
>>   File
>> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/http_helper.py", line
>> 104, in wrapper
>> return func(self, *args)
>>   File
>> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/http_helper.py", line
>> 60, in wrapper
>> ret = func(self, *args)
>>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py",
>> line 97, in put
>> self.request.method, imaged_url, headers, body, stream)
>>   File 

Re: [ovirt-users] Disk image upload pausing

[Idan Shaby]

On Tue, Feb 20, 2018 at 3:33 PM,  wrote:

> Hi,
>
> Here are lines I have found for my last faulty try :
>
> ENGINE
> 2018-02-19 17:52:27,283+01 INFO  [org.ovirt.engine.core.bll.
> storage.disk.image.TransferImageStatusCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-3)
> [1320afb0] Running command: TransferImageStatusCommand internal: true.
> Entities affected :  ID: aaa0----123456789aaa Type:
> SystemAction group CREATE_DISK with role type USER
> 2018-02-19 17:52:27,290+01 INFO  [org.ovirt.engine.core.bll.
> storage.disk.image.TransferImageStatusCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-3)
> [1320afb0] Lock freed to object 'EngineLock:{exclusiveLocks='',
> sharedLocks='[]'}'
> 2018-02-19 17:52:28,658+01 INFO  [org.ovirt.engine.core.bll.
> storage.disk.image.TransferImageStatusCommand] (default task-14)
> [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] Running command:
> TransferImageStatusCommand internal: false. Entities affected :  ID:
> aaa0----123456789aaa Type: SystemAction group CREATE_DISK
> with role type USER
> 2018-02-19 17:52:28,659+01 INFO  [org.ovirt.engine.core.bll.
> storage.disk.image.ImageTransferUpdater] (default task-14)
> [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] Updating image transfer
> 1c55d561-45bf-4e57-b3b6-8fcbf3734a28 (image 
> af5997c5-ae69-4677-9d86-30a978cf83a5)
> phase to Paused by System (message: 'Sent 405200MB')
> 2018-02-19 17:52:28,665+01 WARN  [org.ovirt.engine.core.dal.
> dbbroker.auditloghandling.AuditLogDirector] (default task-14)
> [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] EVENT_ID:
> UPLOAD_IMAGE_NETWORK_ERROR(1,038), Unable to upload image to disk
> af5997c5-ae69-4677-9d86-30a978cf83a5 due to a network error. Make sure
> ovirt-imageio-proxy service is installed and configured, and ovirt-engine's
> certificate is registered as a valid CA in the browser. The certificate can
> be fetched from https:///ovirt-engine/services/pki-resource?
> resource=ca-certificate=X509-PEM-CA
> 2018-02-19 17:52:32,624+01 INFO  [org.ovirt.engine.core.bll.
> storage.disk.image.TransferImageStatusCommand] (default task-28)
> [0eba65e6-cca8-46ee-9038-fb29838ead47] Running command:
> TransferImageStatusCommand internal: false. Entities affected :  ID:
> aaa0----123456789aaa Type: SystemAction group CREATE_DISK
> with role type USER
> 2018-02-19 17:52:36,679+01 INFO  [org.ovirt.engine.core.bll.
> storage.disk.image.TransferImageStatusCommand] (default task-16)
> [5d662615-a4e7-412b-8ecf-45be03c7e49f] Running command:
> TransferImageStatusCommand internal: false. Entities affected :  ID:
> aaa0----123456789aaa Type: SystemAction group CREATE_DISK
> with role type USER
> 2018-02-19 17:52:37,304+01 INFO  [org.ovirt.engine.core.bll.
> storage.disk.image.TransferDiskImageCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-40)
> [6f21de96-d3bd-4cf0-b922-394be7389d3b] Transfer was paused by system.
> Upload disk 'pfm-serv-pdc_Disk1' (id '----
> ')
>
> PROXY
> (Thread-4087) ERROR 2018-02-19 17:52:28,644 
> images:143:root:(make_imaged_request)
> Failed communicating with host: A Connection error occurred.
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py",
> line 134, in make_imaged_request
> timeout=timeout, stream=stream)
>   File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576,
> in send
> r = adapter.send(request, **kwargs)
>   File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415,
> in send
> raise ConnectionError(err, request=request)
> ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))
> (Thread-4087) ERROR 2018-02-19 17:52:28,645 web:112:web:(log_error) ERROR
> [10.100.0.184] PUT /images/f64acb43-d153-485d-b441-9f5d42773a03: [503]
> Failed communicating with host: A Connection error occurred. (0.01s)
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_common/web.py",
> line 64, in __call__
> resp = self.dispatch(request)
>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_common/web.py",
> line 91, in dispatch
> return method(*match.groups())
>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/http_helper.py",
> line 104, in wrapper
> return func(self, *args)
>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/http_helper.py",
> line 60, in wrapper
> ret = func(self, *args)
>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py",
> line 97, in put
> self.request.method, imaged_url, headers, body, stream)
>   File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py",
> line 144, in make_imaged_request
> raise exc.HTTPServiceUnavailable(s)
>
> I saw new updates so I applied them to the whole cluster and the engine
> VM, and finally rebooted everything.
>
> My new try this morning was OK.
>
> Maybe one service 

Re: [ovirt-users] Disk image upload pausing

[spfma . tech]

Hi,
   Here are lines I have found for my last faulty try :ENGINE 2018-02-19 
17:52:27,283+01 INFO 
[org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand] 
(EE-ManagedThreadFactory-engineScheduled-Thread-3) [1320afb0] Running command: 
TransferImageStatusCommand internal: true. Entities affected : ID: 
aaa0----123456789aaa Type: SystemAction group CREATE_DISK with 
role type USER
2018-02-19 17:52:27,290+01 INFO 
[org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand] 
(EE-ManagedThreadFactory-engineScheduled-Thread-3) [1320afb0] Lock freed to 
object 'EngineLock:{exclusiveLocks='', sharedLocks='[]'}'
2018-02-19 17:52:28,658+01 INFO 
[org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand] 
(default task-14) [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] Running command: 
TransferImageStatusCommand internal: false. Entities affected : ID: 
aaa0----123456789aaa Type: SystemAction group CREATE_DISK with 
role type USER
2018-02-19 17:52:28,659+01 INFO 
[org.ovirt.engine.core.bll.storage.disk.image.ImageTransferUpdater] (default 
task-14) [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] Updating image transfer 
1c55d561-45bf-4e57-b3b6-8fcbf3734a28 (image 
af5997c5-ae69-4677-9d86-30a978cf83a5) phase to Paused by System (message: 'Sent 
405200MB')
2018-02-19 17:52:28,665+01 WARN 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default 
task-14) [32ba50ad-f7cd-4b7e-87b9-f7ad8a73a946] EVENT_ID: 
UPLOAD_IMAGE_NETWORK_ERROR(1,038), Unable to upload image to disk 
af5997c5-ae69-4677-9d86-30a978cf83a5 due to a network error. Make sure 
ovirt-imageio-proxy service is installed and configured, and ovirt-engine's 
certificate is registered as a valid CA in the browser. The certificate can be 
fetched from 
https:///ovirt-engine/services/pki-resource?resource=ca-certificate font-size: 
10pt; color: #00;">PROXY (Thread-4087) ERROR 2018-02-19 17:52:28,644 
images:143:root:(make_imaged_request) Failed communicating with host: A 
Connection error occurred.
Traceback (most recent call last):
 File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py", line 
134, in make_imaged_request
 timeout=timeout, stream=stream)
 File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
 r = adapter.send(request, **kwargs)
 File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
 raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))
(Thread-4087) ERROR 2018-02-19 17:52:28,645 web:112:web:(log_error) ERROR 
[10.100.0.184] PUT /images/f64acb43-d153-485d-b441-9f5d42773a03: [503] Failed 
communicating with host: A Connection error occurred. (0.01s)
Traceback (most recent call last):
 File "/usr/lib/python2.7/site-packages/ovirt_imageio_common/web.py", line 64, 
in __call__
 resp = self.dispatch(request)
 File "/usr/lib/python2.7/site-packages/ovirt_imageio_common/web.py", line 91, 
in dispatch
 return method(*match.groups())
 File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/http_helper.py", 
line 104, in wrapper
 return func(self, *args)
 File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/http_helper.py", 
line 60, in wrapper
 ret = func(self, *args)
 File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py", line 
97, in put
 self.request.method, imaged_url, headers, body, stream)
 File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/images.py", line 
144, in make_imaged_request
 raise exc.HTTPServiceUnavailable(s)
   I saw new updates so I applied them to the whole cluster and the engine VM, 
and finally rebooted everything.   My new try this morning was OK.   Maybe one 
service just needed a restart ?   Regards 

 Le 20-Feb-2018 06:54:49 +0100, ish...@redhat.com a crit:   
  Hi,

 Can you please attach the engine, vdsm, daemon and proxy logs?

Regards, Idan   
 On Mon, Feb 19, 2018 at 11:17 AM,  wrote:

   Hi, I am trying to build a new vm based on a vhd image coming from a windows 
machine. I converted the image to raw, and I am now trying to import it in the 
engine. After setting up the CA in my browser, the import process starts but 
stops after a while with "paused by system" status. I can resume it, but it 
pauses without transferring more. The engine logs don't explain much, I see a 
line for the start and the next one for the pause. My network seems to work 
correctly, and I have plenty of space in the storage domain. What can cause the 
process to pause ? Regards 

-
FreeMail powered by mail.fr 
___
 Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

-
FreeMail powered by mail.fr

Re: [ovirt-users] Disk image upload pausing

[Idan Shaby]

Hi,

Can you please attach the engine, vdsm, daemon and proxy logs?


Regards,
Idan

On Mon, Feb 19, 2018 at 11:17 AM,  wrote:

>
> Hi,
> I am trying to build a new vm based on a vhd image coming from a windows
> machine. I converted the image to raw, and I am now trying to import it in
> the engine.
> After setting up the CA in my browser, the import process starts but stops
> after a while with "paused by system" status. I can resume it, but it
> pauses without transferring more.
> The engine logs don't explain much, I see a line for the start and the
> next one for the pause.
> My network seems to work correctly, and I have plenty of space in the
> storage domain.
> What can cause the process to pause ?
> Regards
>
> --
> FreeMail powered by mail.fr
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Disk image upload pausing

[spfma . tech]

Hi, I am trying to build a new vm based on a vhd image coming from a windows 
machine. I converted the image to raw, and I am now trying to import it in the 
engine. After setting up the CA in my browser, the import process starts but 
stops after a while with "paused by system" status. I can resume it, but it 
pauses without transferring more. The engine logs don't explain much, I see a 
line for the start and the next one for the pause. My network seems to work 
correctly, and I have plenty of space in the storage domain. What can cause the 
process to pause ? Regards 

-
FreeMail powered by mail.fr
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Disk image upload via CLI?

[Nir Soffer]

On Thu, Sep 14, 2017 at 5:45 PM Matthias Leopold <
matthias.leop...@meduniwien.ac.at> wrote:

> Hi Daniel and other friendly contributors,
>
> finally i sorted out how to set provisioned_size/initial_size correctly
> in upload_disk.py and my error is gone. It wasn't so easy, but maybe i
> took an awkard route when starting with a preallocated qcow2 image. In
> this special case you have to set provisioned_size to st_size, whereas
> with sparse images provisioned_size is "virtual size" from "qemu-img
> info". This may seem obvious to others, i took the hard route.
>

Just to make this more clear - when you create a disk, you want to use the
virtual size for the disk size - this is the size that the guest will see.

For example:

$ qemu-img create -f qcow2 test.qcow2 20g
Formatting 'test.qcow2', fmt=qcow2 size=21474836480 encryption=off
cluster_size=65536 lazy_refcounts=off refcount_bits=16

$ ls -lh test.qcow2
-rw-r--r--. 1 nsoffer nsoffer 193K Sep 15 18:15 test.qcow2

The disk size should be exactly 20g.

The initial size of the disk depends on the actual file size, in this
example,
anything bigger than 193k will be fine.

When using raw format, the disk size is always the file size.

When using preallocation, the actual file size may be more than the
virtual size, since the qcow file format needs extra space for image
metadata.

$ qemu-img create -f qcow2 test.qcow2 -o preallocation=full 1g
Formatting 'test.qcow2', fmt=qcow2 size=1073741824 encryption=off
cluster_size=65536 preallocation=full lazy_refcounts=off refcount_bits=16

$ ls -lht test.qcow2
-rw-r--r--. 1 nsoffer nsoffer 1.1G Sep 15 18:20 test.qcow2

$ python -c 'import os; print os.path.getsize("test.qcow2")'
1074135040

In this case the initial size must be 1074135040.

I guess the example upload code should be improved so it works
out of the box for preallocated images.


> My approach stems from my desire to repeat the exact example in
> upload_disk.py (which uses a qcow image) and my actual use case, which
> is uploading a rather large image converted from vmdk (i only tested
> this with raw format yet), so i wanted to have some "real large" data to
> upload.
>

I don't know any good reason to do full or falloc preallocation since this
means preallocating the entire image - in this case you should use raw
image and get better performance.

To simulate upload of a real image you can upload some dvd iso, or
create raw disk with some data and convert the disk to qcow2:

$ time dd if=/dev/zero bs=8M count=128 | tr "\0" "\1" > test.raw

$ qemu-img convert -f raw -O qcow2 test.raw test.qcow2

$ qemu-img info test.qcow2
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false

Nir


>
> @nsoffer:
> I'll open a bug for better ovirt-imageio-daemon as soon as i can.
>
> thanks a lot for help
> matthias
>
> Am 2017-09-13 um 16:49 schrieb Daniel Erez:
> > Hi Matthias,
> >
> > The 403 response from the daemon means the ticket can't be authenticated
> > (for some reason). I assume that the issue here is the initial size of
> > the disk.
> > When uploading/downloading a qcow image, you should specify the apparent
> > size of the file (see 'st_size' in [1]). You can get it simply by 'ls
> > -l' [2] (which is
> > a different value from 'disk size' of qemu-img info [3]).
> > btw, why are you creating a preallocated qcow disk? For what use-case?
> >
> > [1] https://linux.die.net/man/2/stat
> >
> > [2] $ ls -l test.qcow2
> > -rw-r--r--. 1 user user 1074135040 Sep 13 16:50 test.qcow2
> >
> > [3]
> > $ qemu-img create -f qcow2 -o preallocation=full test.qcow2 1g
> > $ qemu-img info test.qcow2
> > image: test.qcow2
> > file format: qcow2
> > virtual size: 1.0G (1073741824 bytes)
> > disk size: 1.0G
> > cluster_size: 65536
> > Format specific information:
> >  compat: 1.1
> >  lazy refcounts: false
> >  refcount bits: 16
> >  corrupt: false
> >
> >
> >
> > On Wed, Sep 13, 2017 at 5:03 PM Matthias Leopold
> >  > > wrote:
> >
> > i tried it again twice:
> >
> > when using upload_disk.py from the ovirt engine host itself the disk
> > upload succeeds (despite an "503 Service Unavailable Completed 100%"
> in
> > script output in the end)
> >
> > another try was from an ovirt-sdk installation on my ubuntu desktop
> > itself (yesterday i tried it from a centos VM on my desktop machine).
> > this failed again, this time with "socket.error: [Errno 32] Broken
> pipe"
> > after reaching "200 OK Completed 100%". in imageio-proxy log i have
> > again the 403 error in this moment
> >
> > what's the difference between accessing the API from the engine host
> and
> > from "outside" in this case?
> >
> > thx
> > matthias
> >
> > Am 2017-09-12 um 16:42 schrieb Matthias 

Re: [ovirt-users] Disk image upload via CLI?

[Matthias Leopold]

Hi Daniel and other friendly contributors,

finally i sorted out how to set provisioned_size/initial_size correctly 
in upload_disk.py and my error is gone. It wasn't so easy, but maybe i 
took an awkard route when starting with a preallocated qcow2 image. In 
this special case you have to set provisioned_size to st_size, whereas 
with sparse images provisioned_size is "virtual size" from "qemu-img 
info". This may seem obvious to others, i took the hard route.


My approach stems from my desire to repeat the exact example in 
upload_disk.py (which uses a qcow image) and my actual use case, which 
is uploading a rather large image converted from vmdk (i only tested 
this with raw format yet), so i wanted to have some "real large" data to 
upload.


@nsoffer:
I'll open a bug for better ovirt-imageio-daemon as soon as i can.

thanks a lot for help
matthias

Am 2017-09-13 um 16:49 schrieb Daniel Erez:

Hi Matthias,

The 403 response from the daemon means the ticket can't be authenticated
(for some reason). I assume that the issue here is the initial size of 
the disk.

When uploading/downloading a qcow image, you should specify the apparent
size of the file (see 'st_size' in [1]). You can get it simply by 'ls 
-l' [2] (which is

a different value from 'disk size' of qemu-img info [3]).
btw, why are you creating a preallocated qcow disk? For what use-case?

[1] https://linux.die.net/man/2/stat

[2] $ ls -l test.qcow2
-rw-r--r--. 1 user user 1074135040 Sep 13 16:50 test.qcow2

[3]
$ qemu-img create -f qcow2 -o preallocation=full test.qcow2 1g
$ qemu-img info test.qcow2
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G
cluster_size: 65536
Format specific information:
 compat: 1.1
 lazy refcounts: false
 refcount bits: 16
 corrupt: false



On Wed, Sep 13, 2017 at 5:03 PM Matthias Leopold 
> wrote:


i tried it again twice:

when using upload_disk.py from the ovirt engine host itself the disk
upload succeeds (despite an "503 Service Unavailable Completed 100%" in
script output in the end)

another try was from an ovirt-sdk installation on my ubuntu desktop
itself (yesterday i tried it from a centos VM on my desktop machine).
this failed again, this time with "socket.error: [Errno 32] Broken pipe"
after reaching "200 OK Completed 100%". in imageio-proxy log i have
again the 403 error in this moment

what's the difference between accessing the API from the engine host and
from "outside" in this case?

thx
matthias

Am 2017-09-12 um 16:42 schrieb Matthias Leopold:
 > Thanks, i tried this script and it _almost_ worked ;-)
 >
 > i uploaded two images i created with
 > qemu-img create -f qcow2 -o preallocation=full
 > and
 > qemu-img create -f qcow2 -o preallocation=falloc
 >
 > for initial_size and provisioned_size i took the value reported by
 > "qemu-img info" in "virtual size" (same as "disk size" in this case)
 >
 > the upload goes to 100% and then fails with
 >
 > 200 OK Completed 100%
 > Traceback (most recent call last):
 >File "./upload_disk.py", line 157, in 
 >  headers=upload_headers,
 >File "/usr/lib64/python2.7/httplib.py", line 1017, in request
 >  self._send_request(method, url, body, headers)
 >File "/usr/lib64/python2.7/httplib.py", line 1051, in
_send_request
 >  self.endheaders(body)
 >File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
 >  self._send_output(message_body)
 >File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
 >  self.send(msg)
 >File "/usr/lib64/python2.7/httplib.py", line 840, in send
 >  self.sock.sendall(data)
 >File "/usr/lib64/python2.7/ssl.py", line 746, in sendall
 >  v = self.send(data[count:])
 >File "/usr/lib64/python2.7/ssl.py", line 712, in send
 >  v = self._sslobj.write(data)
 > socket.error: [Errno 104] Connection reset by peer
 >
 > in web GUI the disk stays in Status: "Transferring via API"
 > it can only be removed when manually unlocking it (unlock_entity.sh)
 >
 > engine.log tells nothing interesting
 >
 > i attached the last lines of ovirt-imageio-proxy/image-proxy.log and
 > ovirt-imageio-daemon/daemon.log (from the executing node)
 >
 > the HTTP status 403 in ovirt-imageio-daemon/daemon.log doesn't
look too
 > nice to me
 >
 > can you explain what happens?
 >
 > ovirt engine is 4.1.5
 > ovirt node is 4.1.3 (is that a problem?)
 >
 > thx
 > matthias
 >
 >
 >
 > Am 2017-09-12 um 13:15 schrieb Fred Rolland:
 >> Hi,
 >>
 >> You can check this example:
 >>

https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py
 >>
  

Re: [ovirt-users] Disk image upload via CLI?

[Nir Soffer]

On Wed, Sep 13, 2017 at 5:03 PM Matthias Leopold <
matthias.leop...@meduniwien.ac.at> wrote:

> i tried it again twice:
>
> when using upload_disk.py from the ovirt engine host itself the disk
> upload succeeds (despite an "503 Service Unavailable Completed 100%" in
> script output in the end)
>
> another try was from an ovirt-sdk installation on my ubuntu desktop
> itself (yesterday i tried it from a centos VM on my desktop machine).
> this failed again, this time with "socket.error: [Errno 32] Broken pipe"
> after reaching "200 OK Completed 100%". in imageio-proxy log i have
> again the 403 error in this moment
>
> what's the difference between accessing the API from the engine host and
> from "outside" in this case?
>

There should be no difference - did you run the same code for uploading?


>
> thx
> matthias
>
> Am 2017-09-12 um 16:42 schrieb Matthias Leopold:
> > Thanks, i tried this script and it _almost_ worked ;-)
> >
> > i uploaded two images i created with
> > qemu-img create -f qcow2 -o preallocation=full
> > and
> > qemu-img create -f qcow2 -o preallocation=falloc
> >
> > for initial_size and provisioned_size i took the value reported by
> > "qemu-img info" in "virtual size" (same as "disk size" in this case)
> >
> > the upload goes to 100% and then fails with
> >
> > 200 OK Completed 100%
> > Traceback (most recent call last):
> >File "./upload_disk.py", line 157, in 
> >  headers=upload_headers,
> >File "/usr/lib64/python2.7/httplib.py", line 1017, in request
> >  self._send_request(method, url, body, headers)
> >File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
> >  self.endheaders(body)
> >File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
> >  self._send_output(message_body)
> >File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
> >  self.send(msg)
> >File "/usr/lib64/python2.7/httplib.py", line 840, in send
> >  self.sock.sendall(data)
> >File "/usr/lib64/python2.7/ssl.py", line 746, in sendall
> >  v = self.send(data[count:])
> >File "/usr/lib64/python2.7/ssl.py", line 712, in send
> >  v = self._sslobj.write(data)
> > socket.error: [Errno 104] Connection reset by peer
> >
> > in web GUI the disk stays in Status: "Transferring via API"
> > it can only be removed when manually unlocking it (unlock_entity.sh)
> >
> > engine.log tells nothing interesting
> >
> > i attached the last lines of ovirt-imageio-proxy/image-proxy.log and
> > ovirt-imageio-daemon/daemon.log (from the executing node)
> >
> > the HTTP status 403 in ovirt-imageio-daemon/daemon.log doesn't look too
> > nice to me
> >
> > can you explain what happens?
> >
> > ovirt engine is 4.1.5
> > ovirt node is 4.1.3 (is that a problem?)
> >
> > thx
> > matthias
> >
> >
> >
> > Am 2017-09-12 um 13:15 schrieb Fred Rolland:
> >> Hi,
> >>
> >> You can check this example:
> >>
> https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py
> >>
> >>
> >> Regards,
> >> Fred
> >>
> >> On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold
> >>  >> > wrote:
> >>
> >> Hi,
> >>
> >> is there a way to upload disk images (not OVF files, not ISO files)
> >> to oVirt storage domains via CLI? I need to upload a 800GB file and
> >> this is not really comfortable via browser. I looked at ovirt-shell
> >> and
> >>
> >>
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> >>
> >>
> >> <
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> >,
> >>
> >> but i didn't find an option in either of them.
> >>
> >> thx
> >> matthias
> >>
> >> ___
> >> Users mailing list
> >> Users@ovirt.org 
> >> http://lists.ovirt.org/mailman/listinfo/users
> >> 
> >>
> >>
> >
>
> --
> Matthias Leopold
> IT Systems & Communications
> Medizinische Universität Wien
> Spitalgasse 23 / BT 88 /Ebene 00
> A-1090 Wien
> Tel: +43 1 40160-21241 <+43%201%204016021241>
> Fax: +43 1 40160-921200 <+43%201%2040160921200>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Disk image upload via CLI?

[Nir Soffer]

On Tue, Sep 12, 2017 at 5:45 PM Matthias Leopold <
matthias.leop...@meduniwien.ac.at> wrote:

> Thanks, i tried this script and it _almost_ worked ;-)
>
> i uploaded two images i created with
> qemu-img create -f qcow2 -o preallocation=full
> and
> qemu-img create -f qcow2 -o preallocation=falloc
>
> for initial_size and provisioned_size i took the value reported by
> "qemu-img info" in "virtual size" (same as "disk size" in this case)

the upload goes to 100% and then fails with
>
> 200 OK Completed 100%
> Traceback (most recent call last):
>File "./upload_disk.py", line 157, in 
>  headers=upload_headers,
>File "/usr/lib64/python2.7/httplib.py", line 1017, in request
>  self._send_request(method, url, body, headers)
>File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
>  self.endheaders(body)
>File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
>  self._send_output(message_body)
>File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
>  self.send(msg)
>File "/usr/lib64/python2.7/httplib.py", line 840, in send
>  self.sock.sendall(data)
>File "/usr/lib64/python2.7/ssl.py", line 746, in sendall
>  v = self.send(data[count:])
>File "/usr/lib64/python2.7/ssl.py", line 712, in send
>  v = self._sslobj.write(data)
> socket.error: [Errno 104] Connection reset by peer
>
> in web GUI the disk stays in Status: "Transferring via API"
> it can only be removed when manually unlocking it (unlock_entity.sh)
>
> engine.log tells nothing interesting
>
> i attached the last lines of ovirt-imageio-proxy/image-proxy.log and
> ovirt-imageio-daemon/daemon.log (from the executing node)
>
> the HTTP status 403 in ovirt-imageio-daemon/daemon.log doesn't look too
> nice to me
>
> can you explain what happens?
>

Hi Matthias,

I think qemu-img reports the wrong size in disk size, so the disk you
created
for the upload was smaller than the file you tried to upload. This will end
in
404 forbidden error when you try to write after the end of the specified
size.

In this case we send a detailed error to the proxy but it cannot get this
response
since it tries to send the data to the daemon, but the daemon will never
read
data from invalid request.

You should get the file size using os.path.getsize() instead of qemu-img
info.

That said, our logging is not good enough:

2017-09-12 16:07:13,188 WARNING (Thread-641) [web] xxx.yyy.215.2 - PUT
/1e12aa19-f122-4f6c-bfad-ce84abe2684e 403 138 (0.00s)

This log should explain why the operation is forbidden - for example:

Trying to write to offset x, maximum offset y

Can you file a bug for more detailed logging when upload/download request
is invalid?

Regarding the logs you submitted, please submit complete logs. For example
in this case we would see the ticket size in the log for adding the ticket
for this
upload, which makes the issue more clear.


>
> ovirt engine is 4.1.5
> ovirt node is 4.1.3 (is that a problem?)
>

No but you want to run latest code to get the all the fixes introduced
in 4.1.5.

Cheers,
Nir


>
> thx
> matthias
>
>
>
> Am 2017-09-12 um 13:15 schrieb Fred Rolland:
> > Hi,
> >
> > You can check this example:
> >
> https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py
> >
> > Regards,
> > Fred
> >
> > On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold
> >  > > wrote:
> >
> > Hi,
> >
> > is there a way to upload disk images (not OVF files, not ISO files)
> > to oVirt storage domains via CLI? I need to upload a 800GB file and
> > this is not really comfortable via browser. I looked at ovirt-shell
> > and
> >
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> > <
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> >,
> > but i didn't find an option in either of them.
> >
> > thx
> > matthias
> >
> > ___
> > Users mailing list
> > Users@ovirt.org 
> > http://lists.ovirt.org/mailman/listinfo/users
> > 
> >
> >
>
> --
> Matthias Leopold
> IT Systems & Communications
> Medizinische Universität Wien
> Spitalgasse 23 / BT 88 /Ebene 00
> A-1090 Wien
> Tel: +43 1 40160-21241 <+43%201%204016021241>
> Fax: +43 1 40160-921200 <+43%201%2040160921200>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Disk image upload via CLI?

[Daniel Erez]

Hi Matthias,

The 403 response from the daemon means the ticket can't be authenticated
(for some reason). I assume that the issue here is the initial size of the
disk.
When uploading/downloading a qcow image, you should specify the apparent
size of the file (see 'st_size' in [1]). You can get it simply by 'ls -l'
[2] (which is
a different value from 'disk size' of qemu-img info [3]).
btw, why are you creating a preallocated qcow disk? For what use-case?

[1] https://linux.die.net/man/2/stat

[2] $ ls -l test.qcow2
-rw-r--r--. 1 user user 1074135040 Sep 13 16:50 test.qcow2

[3]
$ qemu-img create -f qcow2 -o preallocation=full test.qcow2 1g
$ qemu-img info test.qcow2
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false



On Wed, Sep 13, 2017 at 5:03 PM Matthias Leopold <
matthias.leop...@meduniwien.ac.at> wrote:

> i tried it again twice:
>
> when using upload_disk.py from the ovirt engine host itself the disk
> upload succeeds (despite an "503 Service Unavailable Completed 100%" in
> script output in the end)
>
> another try was from an ovirt-sdk installation on my ubuntu desktop
> itself (yesterday i tried it from a centos VM on my desktop machine).
> this failed again, this time with "socket.error: [Errno 32] Broken pipe"
> after reaching "200 OK Completed 100%". in imageio-proxy log i have
> again the 403 error in this moment
>
> what's the difference between accessing the API from the engine host and
> from "outside" in this case?
>
> thx
> matthias
>
> Am 2017-09-12 um 16:42 schrieb Matthias Leopold:
> > Thanks, i tried this script and it _almost_ worked ;-)
> >
> > i uploaded two images i created with
> > qemu-img create -f qcow2 -o preallocation=full
> > and
> > qemu-img create -f qcow2 -o preallocation=falloc
> >
> > for initial_size and provisioned_size i took the value reported by
> > "qemu-img info" in "virtual size" (same as "disk size" in this case)
> >
> > the upload goes to 100% and then fails with
> >
> > 200 OK Completed 100%
> > Traceback (most recent call last):
> >File "./upload_disk.py", line 157, in 
> >  headers=upload_headers,
> >File "/usr/lib64/python2.7/httplib.py", line 1017, in request
> >  self._send_request(method, url, body, headers)
> >File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
> >  self.endheaders(body)
> >File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
> >  self._send_output(message_body)
> >File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
> >  self.send(msg)
> >File "/usr/lib64/python2.7/httplib.py", line 840, in send
> >  self.sock.sendall(data)
> >File "/usr/lib64/python2.7/ssl.py", line 746, in sendall
> >  v = self.send(data[count:])
> >File "/usr/lib64/python2.7/ssl.py", line 712, in send
> >  v = self._sslobj.write(data)
> > socket.error: [Errno 104] Connection reset by peer
> >
> > in web GUI the disk stays in Status: "Transferring via API"
> > it can only be removed when manually unlocking it (unlock_entity.sh)
> >
> > engine.log tells nothing interesting
> >
> > i attached the last lines of ovirt-imageio-proxy/image-proxy.log and
> > ovirt-imageio-daemon/daemon.log (from the executing node)
> >
> > the HTTP status 403 in ovirt-imageio-daemon/daemon.log doesn't look too
> > nice to me
> >
> > can you explain what happens?
> >
> > ovirt engine is 4.1.5
> > ovirt node is 4.1.3 (is that a problem?)
> >
> > thx
> > matthias
> >
> >
> >
> > Am 2017-09-12 um 13:15 schrieb Fred Rolland:
> >> Hi,
> >>
> >> You can check this example:
> >>
> https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py
> >>
> >>
> >> Regards,
> >> Fred
> >>
> >> On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold
> >>  >> > wrote:
> >>
> >> Hi,
> >>
> >> is there a way to upload disk images (not OVF files, not ISO files)
> >> to oVirt storage domains via CLI? I need to upload a 800GB file and
> >> this is not really comfortable via browser. I looked at ovirt-shell
> >> and
> >>
> >>
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> >>
> >>
> >> <
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> >,
> >>
> >> but i didn't find an option in either of them.
> >>
> >> thx
> >> matthias
> >>
> >> ___
> >> Users mailing list
> >> Users@ovirt.org 
> >> http://lists.ovirt.org/mailman/listinfo/users
> >> 
> >>
> >>
> >
>
> --
> Matthias Leopold
> IT Systems & Communications
> Medizinische Universität Wien
> Spitalgasse 23 / BT 88 /Ebene 00
> A-1090 Wien
> Tel: +43 1 40160-21241 

Re: [ovirt-users] Disk image upload via CLI?

[Matthias Leopold]

i tried it again twice:

when using upload_disk.py from the ovirt engine host itself the disk 
upload succeeds (despite an "503 Service Unavailable Completed 100%" in 
script output in the end)


another try was from an ovirt-sdk installation on my ubuntu desktop 
itself (yesterday i tried it from a centos VM on my desktop machine). 
this failed again, this time with "socket.error: [Errno 32] Broken pipe" 
after reaching "200 OK Completed 100%". in imageio-proxy log i have 
again the 403 error in this moment


what's the difference between accessing the API from the engine host and 
from "outside" in this case?


thx
matthias

Am 2017-09-12 um 16:42 schrieb Matthias Leopold:

Thanks, i tried this script and it _almost_ worked ;-)

i uploaded two images i created with
qemu-img create -f qcow2 -o preallocation=full
and
qemu-img create -f qcow2 -o preallocation=falloc

for initial_size and provisioned_size i took the value reported by 
"qemu-img info" in "virtual size" (same as "disk size" in this case)


the upload goes to 100% and then fails with

200 OK Completed 100%
Traceback (most recent call last):
   File "./upload_disk.py", line 157, in 
 headers=upload_headers,
   File "/usr/lib64/python2.7/httplib.py", line 1017, in request
 self._send_request(method, url, body, headers)
   File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
 self.endheaders(body)
   File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
 self._send_output(message_body)
   File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
 self.send(msg)
   File "/usr/lib64/python2.7/httplib.py", line 840, in send
 self.sock.sendall(data)
   File "/usr/lib64/python2.7/ssl.py", line 746, in sendall
 v = self.send(data[count:])
   File "/usr/lib64/python2.7/ssl.py", line 712, in send
 v = self._sslobj.write(data)
socket.error: [Errno 104] Connection reset by peer

in web GUI the disk stays in Status: "Transferring via API"
it can only be removed when manually unlocking it (unlock_entity.sh)

engine.log tells nothing interesting

i attached the last lines of ovirt-imageio-proxy/image-proxy.log and 
ovirt-imageio-daemon/daemon.log (from the executing node)


the HTTP status 403 in ovirt-imageio-daemon/daemon.log doesn't look too 
nice to me


can you explain what happens?

ovirt engine is 4.1.5
ovirt node is 4.1.3 (is that a problem?)

thx
matthias



Am 2017-09-12 um 13:15 schrieb Fred Rolland:

Hi,

You can check this example:
https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py 



Regards,
Fred

On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold 
> wrote:


Hi,

is there a way to upload disk images (not OVF files, not ISO files)
to oVirt storage domains via CLI? I need to upload a 800GB file and
this is not really comfortable via browser. I looked at ovirt-shell
and

https://www.ovirt.org/develop/release-management/features/storage/image-upload/ 


, 


but i didn't find an option in either of them.

thx
matthias

___
Users mailing list
Users@ovirt.org 
http://lists.ovirt.org/mailman/listinfo/users







--
Matthias Leopold
IT Systems & Communications
Medizinische Universität Wien
Spitalgasse 23 / BT 88 /Ebene 00
A-1090 Wien
Tel: +43 1 40160-21241
Fax: +43 1 40160-921200
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Disk image upload via CLI?

[Matthias Leopold]

Thanks, i tried this script and it _almost_ worked ;-)

i uploaded two images i created with
qemu-img create -f qcow2 -o preallocation=full
and
qemu-img create -f qcow2 -o preallocation=falloc

for initial_size and provisioned_size i took the value reported by 
"qemu-img info" in "virtual size" (same as "disk size" in this case)


the upload goes to 100% and then fails with

200 OK Completed 100%
Traceback (most recent call last):
  File "./upload_disk.py", line 157, in 
headers=upload_headers,
  File "/usr/lib64/python2.7/httplib.py", line 1017, in request
self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 840, in send
self.sock.sendall(data)
  File "/usr/lib64/python2.7/ssl.py", line 746, in sendall
v = self.send(data[count:])
  File "/usr/lib64/python2.7/ssl.py", line 712, in send
v = self._sslobj.write(data)
socket.error: [Errno 104] Connection reset by peer

in web GUI the disk stays in Status: "Transferring via API"
it can only be removed when manually unlocking it (unlock_entity.sh)

engine.log tells nothing interesting

i attached the last lines of ovirt-imageio-proxy/image-proxy.log and 
ovirt-imageio-daemon/daemon.log (from the executing node)


the HTTP status 403 in ovirt-imageio-daemon/daemon.log doesn't look too 
nice to me


can you explain what happens?

ovirt engine is 4.1.5
ovirt node is 4.1.3 (is that a problem?)

thx
matthias



Am 2017-09-12 um 13:15 schrieb Fred Rolland:

Hi,

You can check this example:
https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py

Regards,
Fred

On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold 
> wrote:


Hi,

is there a way to upload disk images (not OVF files, not ISO files)
to oVirt storage domains via CLI? I need to upload a 800GB file and
this is not really comfortable via browser. I looked at ovirt-shell
and

https://www.ovirt.org/develop/release-management/features/storage/image-upload/

,
but i didn't find an option in either of them.

thx
matthias

___
Users mailing list
Users@ovirt.org 
http://lists.ovirt.org/mailman/listinfo/users





--
Matthias Leopold
IT Systems & Communications
Medizinische Universität Wien
Spitalgasse 23 / BT 88 /Ebene 00
A-1090 Wien
Tel: +43 1 40160-21241
Fax: +43 1 40160-921200
2017-09-12 16:07:10,046 INFO(Thread-632) [web] xxx.yyy.215.2 - PUT /1e12aa19-f122-4f6c-bfad-ce84abe2684e 200 0 (0.28s)
2017-09-12 16:07:10,171 INFO(Thread-633) [images] Writing 8388608 bytes at offset 5301600256 to /rhev/data-center/0001-0001-0001-0001-0311/ebb620c9-6dfe-43a8-9867-20b9a93c76b5/images/54b6da51-1c67-42e9-b128-0a218fa1e8b7/e0c1ab33-a817-4207-b1f9-32f1aa4e46be for ticket 1e12aa19-f122-4f6c-bfad-ce84abe2684e
2017-09-12 16:07:10,439 INFO(Thread-633) [web] xxx.yyy.215.2 - PUT /1e12aa19-f122-4f6c-bfad-ce84abe2684e 200 0 (0.27s)
2017-09-12 16:07:10,556 INFO(Thread-634) [images] Writing 8388608 bytes at offset 5309988864 to /rhev/data-center/0001-0001-0001-0001-0311/ebb620c9-6dfe-43a8-9867-20b9a93c76b5/images/54b6da51-1c67-42e9-b128-0a218fa1e8b7/e0c1ab33-a817-4207-b1f9-32f1aa4e46be for ticket 1e12aa19-f122-4f6c-bfad-ce84abe2684e
2017-09-12 16:07:10,819 INFO(Thread-634) [web] xxx.yyy.215.2 - PUT /1e12aa19-f122-4f6c-bfad-ce84abe2684e 200 0 (0.26s)
2017-09-12 16:07:10,924 INFO(Thread-635) [images] Writing 8388608 bytes at offset 5318377472 to /rhev/data-center/0001-0001-0001-0001-0311/ebb620c9-6dfe-43a8-9867-20b9a93c76b5/images/54b6da51-1c67-42e9-b128-0a218fa1e8b7/e0c1ab33-a817-4207-b1f9-32f1aa4e46be for ticket 1e12aa19-f122-4f6c-bfad-ce84abe2684e
2017-09-12 16:07:11,219 INFO(Thread-635) [web] xxx.yyy.215.2 - PUT /1e12aa19-f122-4f6c-bfad-ce84abe2684e 200 0 (0.30s)
2017-09-12 16:07:11,336 INFO(Thread-636) [images] Writing 8388608 bytes at offset 5326766080 to /rhev/data-center/0001-0001-0001-0001-0311/ebb620c9-6dfe-43a8-9867-20b9a93c76b5/images/54b6da51-1c67-42e9-b128-0a218fa1e8b7/e0c1ab33-a817-4207-b1f9-32f1aa4e46be for ticket 1e12aa19-f122-4f6c-bfad-ce84abe2684e
2017-09-12 16:07:11,595 INFO(Thread-636) [web] xxx.yyy.215.2 - PUT /1e12aa19-f122-4f6c-bfad-ce84abe2684e 200 0 (0.26s)
2017-09-12 16:07:11,711 INFO(Thread-637) [images] Writing 8388608 bytes at offset 5335154688 to 

Re: [ovirt-users] Disk image upload via CLI?

[Yaniv Kaul]

On Tue, Sep 12, 2017 at 2:15 PM, Fred Rolland  wrote:

> Hi,
>
> You can check this example:
> https://github.com/oVirt/ovirt-engine-sdk/blob/master/
> sdk/examples/upload_disk.py
>

Or via Ansibe:
https://github.com/oVirt/ovirt-ansible/blob/master/roles/ovirt-image-template/README.md

Y.


>
> Regards,
> Fred
>
> On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold <
> matthias.leop...@meduniwien.ac.at> wrote:
>
>> Hi,
>>
>> is there a way to upload disk images (not OVF files, not ISO files) to
>> oVirt storage domains via CLI? I need to upload a 800GB file and this is
>> not really comfortable via browser. I looked at ovirt-shell and
>> https://www.ovirt.org/develop/release-management/features/st
>> orage/image-upload/, but i didn't find an option in either of them.
>>
>> thx
>> matthias
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Disk image upload via CLI?

[Fred Rolland]

Hi,

You can check this example:
https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py

Regards,
Fred

On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold <
matthias.leop...@meduniwien.ac.at> wrote:

> Hi,
>
> is there a way to upload disk images (not OVF files, not ISO files) to
> oVirt storage domains via CLI? I need to upload a 800GB file and this is
> not really comfortable via browser. I looked at ovirt-shell and
> https://www.ovirt.org/develop/release-management/features/st
> orage/image-upload/, but i didn't find an option in either of them.
>
> thx
> matthias
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Disk image upload via CLI?

[Matthias Leopold]

Hi,

is there a way to upload disk images (not OVF files, not ISO files) to 
oVirt storage domains via CLI? I need to upload a 800GB file and this is 
not really comfortable via browser. I looked at ovirt-shell and 
https://www.ovirt.org/develop/release-management/features/storage/image-upload/, 
but i didn't find an option in either of them.


thx
matthias

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Disk image upload

[Anantha Raghava]

Hi,

I am trying to upload a Disk image using Web Admin UI. It is pausing the 
process and throwing up this error.


"Unable to upload image to disk 3e71c278-e03b-40e5-afaa-a13bed115229 due 
to a network error. Make sure ovirt-imageio-proxy service is installed 
and configured, and ovirt-engine's certificate is registered as a valid 
CA in the browser. The certificate can be fetched from 
https:///ovirt-engine/services/pki-resource?resource=ca-certificate=X509-PEM-CA


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users