[ovirt-users] Re: NVMe disk cause kernel panic in installer
Unfortunately, it didn't help. ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2YSRWPT536PAEWIJE6K5DPLZ7BDV24YJ/
[ovirt-users] Re: NVMe disk cause kernel panic in installer
I have seen similar behaviour in Anakonda even with regular disks.Please try to boot from the instalaltion media and go for troubleshooting.Then , just get a console (Ctrl + Alt + Fx - where x is between 1 and 12) and wipe the beginning of your NVMe.Something like:dd if=/dev/zero of=/dev/nvme0n1 bs=4M count=100 status=progress Then reboot and try again. I assume that you do not have anything on it. Best Regards,Strahil Nikolov В четвъртък, 18 юли 2019 г., 19:53:57 ч. Гринуич+3, bossma...@gmail.com написа: While installer is loading. ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NBNQE62UUKZ34ANXYIDB2SUOW7EY2KA3/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QI2CMCXLDO2QIBKW3STJWJUFNJLGJL47/
[ovirt-users] Re: Storage domain 'Inactive' but still functional
Hi Martin, First check what went wrong with the VG -as it could be something simple.vgcfgbackup -f VGname will create a file which you can use to compare current metadata with a previous version. If you have Linux boxes - you can add disks from another storage and then pvmove the data inside the VM. Of course , you will need to reinstall grub on the new OS disk , or you won't be able to boot afterwards.If possible, try with a test VM before proceeding with important ones. Backing up the VMs is very important , because working on LVM metadata is quite risky.Last time I had such an issue , I was working on clustered LVs which got their PVs "Missing". For me , restore from VG backup fixed the issue - but that might not be always the case. Just get the vgcfgbackup's output and compare with diff or vimdiff and check what is different. Sadly, I think that this is more a Linux problem , than an oVirt problem. Best Regards,Strahil Nikolov В четвъртък, 18 юли 2019 г., 18:51:32 ч. Гринуич+3, Martijn Grendelman написа: Hi! Thanks. Like I wrote, I have metadata backups from /etc/lvm/backup and -/archive, and I also have the current metadata as it exists on disk. What I'm most concerned about, is the proposed procedure. I would create a backup of the VG, but I'm not sure what would be the most sensible way to do it. I could make a new iSCSI target and simply 'dd' the whole disk over, but that would take quite some time (it's 2,5 TB) and there are VMs that can't really be down for that long. And I'm not even sure that dd'ing the disk like that is a sensible strategy. Moving disks out of the domain is currently not possible. oVirt says 'Source Storage Domain is not active'. Thanks, Martijn. Op 18-7-2019 om 17:44 schreef Strahil Nikolov: Can you check the /etc/lvm/backup and /etc/lvm/archive on your SPM host (check the other hosts, just in case you find anything useful) ?Usually LVM makes backup of everything. I would recommend you to:1. Create a backup of the problematic VG2. Compare the backup file and a file from backup/archive folders for the same VGCheck what is different with diff/vimdiff . It might give you a clue. I had some issues (non-related to oVirt) and restoring the VG from older backup did help me .Still ,any operation on block devices should be considered risky and a proper backup is needed.You could try to move a less important VM's disks out of this storage domain to another one. If it succeeds - then you can evacuate all VMs away before you can start "breaking" the storage domain. Best Regards,Strahil Nikolov В четвъртък, 18 юли 2019 г., 16:59:46 ч. Гринуич+3, Martijn Grendelman написа: Hi, It appears that O365 has trouble delivering mails to this list, so two earlier mails of mine are still somewhere in a queue and may yet be delivered. This mail has all of the content of 3 successive mails. I apologize for this format. Op 18-7-2019 om 11:20 schreef Martijn Grendelman: Op 18-7-2019 om 10:16 schreef Martijn Grendelman: Hi, For the first time in many months I have run into some trouble with oVirt (4.3.4.3) and I need some help. Yesterday, I noticed one of my iSCSI storage domains was almost full, and tried to move a disk image off of it, to another domain. This failed, and somewhere in the process, the whole storage domain went to status 'Inactive'. >From engine.log: 2019-07-17 16:30:35,319+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-1836383) [] starting processDomainRecovery for domain '875847b6-29a4-4419-be92-9315f4435429:HQST0_ISCSI02'. 2019-07-17 16:30:35,337+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-1836383) [] Domain '875847b6-29a4-4419-be92-9315f4435429:HQST0_ISCSI02' was reported by all hosts in status UP as problematic. Moving the domain to NonOperational. 2019-07-17 16:30:35,410+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1836383) [5f6fd35e] EVENT_ID: SYSTEM_DEACTIVATED_STORAGE_DOMAIN(970), Storage Domain HQST0_ISCSI02 (Data Center ISAAC01) was deactivated by system because it's not visible by any of the hosts. The thing is, the domain is still functional on all my hosts. It carries over 50 disks, and all involved VMs are up and running, and don't seem to have any problems. Also, 'iscsiadm' on all hosts seems to indiciate that everything is fine with this specific target and reading from the device with dd, or getting its size with 'blockdev' all works without issue. When I try to reactivate the domain, these errors are logged: 2019-07-18 09:34:53,631+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-43475) [79e386e] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command ActivateStorageDomainVDS failed: Storage domain does not exist:
[ovirt-users] Re: Error exporting into ova
On Fri, Jul 19, 2019 at 4:14 PM Gianluca Cecchi wrote: > On Fri, Jul 19, 2019 at 3:15 PM Gianluca Cecchi > wrote: > >> >> >> In engine.log the first error I see is 30 minutes after start >> >> 2019-07-19 12:25:31,563+02 ERROR >> [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] >> (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible >> playbook execution failed: Timeout occurred while executing Ansible >> playbook. >> > > In the mean time, as the playbook seems this one ( I run the job from > engine) : /usr/share/ovirt-engine/playbooks/ovirt-ova-export.yml > > Based on what described in bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=1697301 I created at the moment the file /etc/ovirt-engine/engine.conf.d/99-ansible-playbook-timeout.conf with ANSIBLE_PLAYBOOK_EXEC_DEFAULT_TIMEOUT=80 and restarted the engine and the python script to verify Just to see if it completes, even if in my case with a 30Gb preallocated disk, the source problem is qemu-img convert command very slow in I/O. It reads from iscsi multipath (2 paths) with 2x3MB/s and it writes on nfs If I run a dd command from iscsi device mapper device to an nfs file I have 140MB/s rate that is what expected based on my storage array performances and my network. Not understood why the qemu-img command is so slow The question still applies in case I have to do an appliance from a VM with a very big disk, where the copy could potentially have an elapsed of more that 30 minutes... Gianluca ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DMUARHYE34JLJRAX3BLEPV54EK2CS5SD/
[ovirt-users] Re: Error exporting into ova
On Fri, Jul 19, 2019 at 3:15 PM Gianluca Cecchi wrote: > > > In engine.log the first error I see is 30 minutes after start > > 2019-07-19 12:25:31,563+02 ERROR > [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] > (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible > playbook execution failed: Timeout occurred while executing Ansible > playbook. > In the mean time, as the playbook seems this one ( I run the job from engine) : /usr/share/ovirt-engine/playbooks/ovirt-ova-export.yml - hosts: all remote_user: root gather_facts: no roles: - ovirt-ova-export-pre-pack - ovirt-ova-pack - ovirt-ova-export-post-pack and it seems async is not supported, I temporary changed the main.yml inside ovirt-ova-pack role from -- - name: Run packing script script: > pack_ova.py ... to --- - name: Copy packing script copy: src: pack_ova.py dest: /tmp/pack_ova.py mode: '0755' - name: Run packing script command: timeout 4800 /tmp/pack_ova.py ... just to check... see how it goes now ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/3QSRA47E7STVEFX3WLPFPUC3YNNSCPWR/
[ovirt-users] Error exporting into ova
Hello,
I'm playing with export_vm_as_ova.py downloaded from the examples github:
https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/export_vm_as_ova.py
My environment is oVirt 4.3.3.7 with iSCSI storage domain.
It fails leaving an ova.tmp file
In webadmin gui:
Starting to export Vm enginecopy1 as a Virtual Appliance
7/19/1911:55:12 AM
VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical
Volume: ('General Storage Exception: ("5 [] [\' Logical volume
fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b
in
use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\',
\'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',)
7/19/1912:25:36 PM
Failed to export Vm enginecopy1 as a Virtual Appliance to path
/save_ova/base/dump/myvm2.ova on Host ov301
7/19/1912:25:37 PM
During export I have this qemu-img process creating the disk over the loop
device:
root 30878 30871 0 11:55 pts/200:00:00 su -p -c qemu-img convert
-T none -O qcow2
'/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b'
'/dev/loop1' vdsm
vdsm 30882 30878 10 11:55 ?00:00:00 qemu-img convert -T none -O
qcow2
/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b
/dev/loop1
The ova.tmp file is getting filled while command runs
eg:
[root@ov301 ]# du -sh /save_ova/base/dump/myvm2.ova.tmp
416M /save_ova/base/dump/myvm2.ova.tmp
[root@ov301 sysctl.d]#
[root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp
911M /save_ova/base/dump/myvm2.ova.tmp
[root@ov301 ]#
and the final generated / not completed file is in this state:
[root@ov301 ]# qemu-img info /save_ova/base/dump/myvm2.ova.tmp
image: /save_ova/base/dump/myvm2.ova.tmp
file format: raw
virtual size: 30G (32217446400 bytes)
disk size: 30G
[root@ov301 sysctl.d]#
But I notice that the timestamp of the file is about 67 minutes after start
of job and well after the notice of its failure
[root@ov301 sysctl.d]# ll /save_ova/base/dump/
total 30963632
-rw---. 1 root root 32217446400 Jul 19 13:02 myvm2.ova.tmp
[root@ov301 sysctl.d]#
[root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp
30G /save_ova/base/dump/myvm2.ova.tmp
[root@ov301 sysctl.d]#
In engine.log the first error I see is 30 minutes after start
2019-07-19 12:25:31,563+02 ERROR
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible
playbook execution failed: Timeout occurred while executing Ansible
playbook.
2019-07-19 12:25:31,563+02 INFO
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible
playbook command has exited with value: 1
2019-07-19 12:25:31,564+02 ERROR
[org.ovirt.engine.core.bll.CreateOvaCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Failed to
create OVA. Please check logs for more details:
/var/log/ovirt-engine/ova/ovirt-export-ova-ansible-20190719115531-ov301-2001ddf4.log
2019-07-19 12:25:31,565+02 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] START,
TeardownImageVDSCommand(HostName = ov301,
ImageActionsVDSCommandParameters:{hostId='8ef1ce6f-4e38-486c-b3a4-58235f1f1d06'}),
log id: 3d2246f7
2019-07-19 12:25:36,569+02 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] EVENT_ID:
VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov301 command TeardownImageVDS
failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5
[] [\' Logical volume
fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b
in
use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\',
\'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',)
In ansible playbook suggested log file I don't see anything useful.
It ends with timestamps when the script has been launched.
Last lines are:
2019-07-19 11:55:33,877 p=5699 u=ovirt | TASK [ovirt-ova-export-pre-pack :
Retrieving the temporary path for the OVA file] ***
2019-07-19 11:55:34,198 p=5699 u=ovirt | changed: [ov301] => {
"changed": true,
"dest": "/save_ova/base/dump/myvm2.ova.tmp",
"gid": 0,
"group": "root",
"mode": "0600",
"owner": "root",
"secontext": "system_u:object_r:nfs_t:s0",
"size": 32217446912,
"state": "file",
"uid": 0
}
2019-07-19 11:55:34,204 p=5699 u=ovirt | TASK [ovirt-ova-pack : Run
packing script] *
It seems 30 minutes... for timeout? About what, ansible job?
Or possibly implicit user session created when running the python script?
The snapshot has been correctly deleted (as I see also in engine.log), I
don't
[ovirt-users] Re: Stuck in "Finalizing" disk upload phase
Hi Daniel,
Is it possible that upgrade from 4.3.3 to 4.3.4 affected/reset/changed the
certificate for imageio?
Reason why I am asking is that, while running 4.3.3 version, I was uploading
Ubuntu 16-18, CentOS 7 cloud images, and CentOS6 no issues.
Now issue is with CentOS6.
I have tried to upload centos6.5 image via Ansible ovirt_disk and than one
ended stuck in “Finalizing” state.
Than I tried with to Upload same image from UI and that ended in “Paused” state.
I will try Test Connection and let you know.
Sent from my iPhone
> On 18 Jul 2019, at 16:13, Daniel Erez wrote:
>
> Hi Marko,
>
> According to the log, it seems there's an issue with a secured
> connection to the proxy.
> Can you please try the 'Test Connection' button on upload disk dialog,
> to ensure the certificate is properly installed?
> Also, is the disk currently in 'Paused by System' status?
>
> Thanks,
> Daniel
>
> On Thu, Jul 18, 2019 at 2:35 PM Vrgotic, Marko
> wrote:
>>
>> Hi Nir,
>>
>>
>>
>> Since this is already compressed file, I have added output of grep for
>> moment this command was issue and following 300lines.
>>
>>
>>
>> [root@ovirt-engine ovirt-engine]# zgrep "2019-07-17 13:36:11,121Z"
>> engine.log-20190* -A300
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,121Z INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] Running command:
>> TransferDiskImageCommand internal: false. Entities affected : ID:
>> 3452459d-aec6-430e-9509-1d9ca815b2d8 Type: DiskAction group
>> EDIT_DISK_PROPERTIES with role type USER
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,121Z INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] Creating
>> ImageTransfer entity for command '28eda0c2-e36b-4e70-91ea-2ecf4a030d19'
>>
>> gzip: -A300.gz: No such file or directory
>>
>> [root@ovirt-engine ovirt-engine]# zgrep -A 300 "2019-07-17 13:36:11,121Z"
>> engine.log-20190*
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,121Z INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] Running command:
>> TransferDiskImageCommand internal: false. Entities affected : ID:
>> 3452459d-aec6-430e-9509-1d9ca815b2d8 Type: DiskAction group
>> EDIT_DISK_PROPERTIES with role type USER
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,121Z INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] Creating
>> ImageTransfer entity for command '28eda0c2-e36b-4e70-91ea-2ecf4a030d19'
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,142Z INFO
>> [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] Successfully added
>> Upload disk 'av-07-centos-65-base' (disk id:
>> '3452459d-aec6-430e-9509-1d9ca815b2d8', image id:
>> 'b44659a9-607a-4eeb-a255-99532fd4fce4') for image transfer command
>> '28eda0c2-e36b-4e70-91ea-2ecf4a030d19'
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,188Z INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (default
>> task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] START,
>> PrepareImageVDSCommand(HostName = ovirt-hv-01.avinity.tv,
>> PrepareImageVDSCommandParameters:{hostId='e7e3f1dc-8037-4e74-a44c-442bdb02197d'}),
>> log id: 3bd84b38
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,351Z INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (default
>> task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] FINISH,
>> PrepareImageVDSCommand, return: PrepareImageReturn:{status='Status [code=0,
>> message=Done]'}, log id: 3bd84b38
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,356Z INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeLegalityVDSCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] START,
>> SetVolumeLegalityVDSCommand(
>> SetVolumeLegalityVDSCommandParameters:{storagePoolId='10525aa0-839d-11e9-a016-00163e4f2a6d',
>> ignoreFailoverLimit='false',
>> storageDomainId='644aacaa-12e1-4fcd-b3aa-941678cf95bd',
>> imageGroupId='3452459d-aec6-430e-9509-1d9ca815b2d8',
>> imageId='b44659a9-607a-4eeb-a255-99532fd4fce4'}), log id: 5e0f7654
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,390Z INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeLegalityVDSCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] FINISH,
>> SetVolumeLegalityVDSCommand, return: , log id: 5e0f7654
>>
>> engine.log-20190718.gz:2019-07-17 13:36:11,394Z INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.AddImageTicketVDSCommand]
>> (default task-376) [a43180ec-afc7-429e-9f30-9e851eaf7ce7] START,
>> AddImageTicketVDSCommand(HostName = ovirt-hv-01.avinity.tv,
>>
