Hi!
This is from ansible logs:
2021-04-27 22:20:38,286+0300 ERROR ansible failed {
"ansible_host": "localhost",
"ansible_playbook":
"/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
"ansible_result": {
"_ansible_no_log": false,
"changed": false,
"exception": "Traceback (most recent call last):\n File
\"/tmp/ansible_ovirt_storage_domain_payload_hlz7j9sk/ansible_ovirt_storage_domain_payload.zip/ansible_collections/ovirt/ovirt/plugins/modules/ovirt_storage_domain.py\",
line 804, in main\n File
\"/tmp/ansible_ovirt_storage_domain_payload_hlz7j9sk/ansible_ovirt_storage_domain_payload.zip/ansible_collections/ovirt/ovirt/plugins/module_utils/ovirt.py\",
line 651, in create\n **kwargs\n File
\"/usr/lib64/python3.6/site-packages/ovirtsdk4/services.py\", line
26182, in add\n return self._internal_add(storage_domain, headers,
query, wait)\n File
\"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 232,
in _internal_add\n return future.wait() if wait else future\n File
\"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 55,
in wait\n return self._code(response)\n File
\"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 229,
in callback\n self._check_fault(response)\n File
\"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 132,
in _check_fault\n self._raise_error(response, body)\n File
\"/usr/lib64/python3.6/site-packages/ovirtsdk4/service.py\", line 118,
in _raise_error\n raise error\novirtsdk4.Error: Fault reason is
\"Operation Failed\". Fault detail is \"[Physical device
initialization failed. Please check that the device is empty and
accessible by the host.]\". HTTP response code is 400.\n",
"invocation": {
"module_args": {
"backup": null,
"comment": null,
"critical_space_action_blocker": null,
"data_center": "Default",
"description": null,
"destroy": null,
"discard_after_delete": true,
"domain_function": "data",
"fcp": {
"lun_id": "36000d31005b4f6000000000000000029"
},
"fetch_nested": false,
"format": null,
"glusterfs": null,
"host": "ovirt02.virt.local",
"id": null,
"iscsi": null,
"localfs": null,
"managed_block_storage": null,
"name": "hosted_storage",
"nested_attributes": [],
"nfs": null,
"poll_interval": 3,
"posixfs": null,
"state": "unattached",
"timeout": 180,
"wait": true,
"warning_low_space": null,
"wipe_after_delete": null
}
},
"msg": "Fault reason is \"Operation Failed\". Fault detail is
\"[Physical device initialization failed. Please check that the device
is empty and accessible by the host.]\". HTTP response code is 400."
},
"ansible_task": "Add Fibre Channel storage domain",
"ansible_type": "task",
"status": "FAILED",
This seem to happen same time:
2021-04-27 22:20:37,576+0300 INFO (jsonrpc/5) [vdsm.api] FINISH
getDeviceList return={'devList': [{'GUID':
'36000d31005b4f6000000000000000029', 'capacity': '536870912000',
'devtype': 'FCP', 'fwrev': '0704', 'logicalblocksize': '512',
'pathlist': [], 'pathstatus': [{'physdev': 'sdu', 'state': 'active',
'capacity': '536870912000', 'lun': '3', 'type': 'FCP'}, {'physdev':
'sds', 'state': 'active', 'capacity': '536870912000', 'lun': '3',
'type': 'FCP'}, {'physdev': 'sdv', 'state': 'active', 'capacity':
'536870912000', 'lun': '3', 'type': 'FCP'}, {'physdev': 'sdt',
'state': 'active', 'capacity': '536870912000', 'lun': '3', 'type':
'FCP'}], 'physicalblocksize': '4096', 'productID': 'Compellent Vol',
'pvUUID': '', 'pvsize': '', 'serial':
'SCOMPELNTCompellent_Vol_0005b4f6-00000029', 'vendorID': 'COMPELNT',
'vgUUID': '', 'discard_max_bytes': 268435456, 'discard_zeroes_data':
0, 'status': 'unknown'}]} from=::ffff:192.168.222.89,57472,
flow_id=676e5ced-7a4f-4a36-8842-cb98a38f5f4b,
task_id=792a93ab-5129-4bf2-8ff0-70b5cdf19221 (api:54)
2021-04-27 22:20:37,641+0300 INFO (jsonrpc/4) [vdsm.api] START
createVG(vgname='bd3efbef-0178-4a21-9a13-2a293482fdd5',
devlist=['36000d31005b4f6000000000000000029'], force=False,
options=None) from=::ffff:192.168.222.89,57472, flow_id=6dce1780,
task_id=4b269432-9f27-406b-a03b-6dfaaa2d1f67 (api:48)
2021-04-27 22:20:37,729+0300 WARN (jsonrpc/4) [storage.LVM] Command
with specific filter failed or returned no data, retrying with a wider
filter, cmd=['/sbin/lvm', 'pvcreate', '--config', 'devices {
preferred_names=["^/dev/mapper/"] ignore_suspended_devices=1
write_cache_state=0 disable_after_error_count=3
filter=["a|^/dev/mapper/36000d31005b4f6000000000000000029$|", "r|.*|"]
hints="none" obtain_device_list_from_udev=0 } global {
locking_type=1 prioritise_write_locks=1 wait_for_locks=1
use_lvmetad=0 } backup { retain_min=50 retain_days=0 }',
'--metadatasize', '128m', '--metadatacopies', '2', '--metadataignore',
'y', '/dev/mapper/36000d31005b4f6000000000000000029'] rc=5 out=[]
err=[" Can't open /dev/mapper/36000d31005b4f6000000000000000029
exclusively. Mounted filesystem?", " Can't open
/dev/mapper/36000d31005b4f6000000000000000029 exclusively. Mounted
filesystem?"] (lvm:511)
2021-04-27 22:20:37,830+0300 WARN (jsonrpc/4) [storage.LVM] All 2
tries have failed: cmd=['/sbin/lvm', 'pvcreate', '--config', 'devices
{ preferred_names=["^/dev/mapper/"] ignore_suspended_devices=1
write_cache_state=0 disable_after_error_count=3
filter=["a|^/dev/mapper/360002ac00000000000000261000057b9$|^/dev/mapper/36000d31005b4f600000000000000001a$|^/dev/mapper/36000d31005b4f600000000000000001d$|^/dev/mapper/36000d31005b4f6000000000000000029$|",
"r|.*|"] hints="none" obtain_device_list_from_udev=0 } global {
locking_type=1 prioritise_write_locks=1 wait_for_locks=1
use_lvmetad=0 } backup { retain_min=50 retain_days=0 }',
'--metadatasize', '128m', '--metadatacopies', '2', '--metadataignore',
'y', '/dev/mapper/36000d31005b4f6000000000000000029'] rc=5 err=["
Can't open /dev/mapper/36000d31005b4f6000000000000000029 exclusively.
Mounted filesystem?", " Can't open
/dev/mapper/36000d31005b4f6000000000000000029 exclusively. Mounted
filesystem?"] (lvm:538)
2021-04-27 22:20:37,830+0300 ERROR (jsonrpc/4) [storage.LVM] pvcreate
failed with rc=5 (lvm:1142)
2021-04-27 22:20:37,830+0300 ERROR (jsonrpc/4) [storage.LVM] [], ["
Can't open /dev/mapper/36000d31005b4f6000000000000000029 exclusively.
Mounted filesystem?", " Can't open
/dev/mapper/36000d31005b4f6000000000000000029 exclusively. Mounted
filesystem?"] (lvm:1143)
2021-04-27 22:20:37,830+0300 INFO (jsonrpc/4) [vdsm.api] FINISH
createVG error=Failed to initialize physical device:
("['/dev/mapper/36000d31005b4f6000000000000000029']",)
from=::ffff:192.168.222.89,57472, flow_id=6dce1780,
task_id=4b269432-9f27-406b-a03b-6dfaaa2d1f67 (api:52)
2021-04-27 22:20:37,830+0300 ERROR (jsonrpc/4)
[storage.TaskManager.Task]
(Task='4b269432-9f27-406b-a03b-6dfaaa2d1f67') Unexpected error
(task:880)
2021-04-27 22:20:37,830+0300 INFO (jsonrpc/4)
[storage.TaskManager.Task]
(Task='4b269432-9f27-406b-a03b-6dfaaa2d1f67') aborting: Task is
aborted: 'value=Failed to initialize physical device:
("[\'/dev/mapper/36000d31005b4f6000000000000000029\']",)
abortedcode=601' (task:1190)
2021-04-27 22:20:37,830+0300 ERROR (jsonrpc/4) [storage.Dispatcher]
FINISH createVG error=Failed to initialize physical device:
("['/dev/mapper/36000d31005b4f6000000000000000029']",) (dispatcher:83)
I tried clearing the disk but from above it seems that something else
is wonky here.
Thanks,
Juhani
On Wed, Apr 28, 2021 at 8:30 AM Yedidyah Bar David <d...@redhat.com> wrote:
>
> On Tue, Apr 27, 2021 at 10:59 PM Juhani Rautiainen
> <juhani.rautiai...@gmail.com> wrote:
> >
> > Story of the problems continues. Finally shut everything down, got
> > storage domains to maintenance and then this happens:
> >
> > ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is
> > "[Physical device initialization failed. Please check that the device
> > is empty and accessible by the host.]". HTTP response code is 400.
> > [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg":
> > "Fault reason is \"Operation Failed\". Fault detail is \"[Physical
> > device initialization failed. Please check that the device is empty
> > and accessible by the host.]\". HTTP response code is 400."}
>
> Can you please check/share relevant parts of vdsm.log when this happens?
> Thanks.
>
> >
> > No amount zeroing device helped.
> >
> > Next plan: Find out if I can restore backup to standalone server. If
> > that fails that's the end of oVirt for me. It worked fine for a couple
> > of years but this update hassle is too much. Should have gone until
> > the end with 4.3.
> >
> > Thanks,
> > Juhani
> >
> > On Tue, Apr 27, 2021 at 11:47 AM Juhani Rautiainen
> > <juhani.rautiai...@gmail.com> wrote:
> > >
> > > Hmm. Is it possible that when the other node is still running v4.3
> > > this operation can't be completed as it doesn't know how to do it.
> > >
> > > Thanks,
> > > Juhani
> > >
> > > On Tue, Apr 27, 2021 at 11:07 AM Juhani Rautiainen
> > > <juhani.rautiai...@gmail.com> wrote:
> > > >
> > > > It seems that it is not supported in oVirt yet? I got this response
> > > > when I tried to change master with those storage domain that I have:
> > > >
> > > > <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> > > > <fault>
> > > > <detail>[Cannot switch master storage domain. Switch master
> > > > storage domain operation is not supported.]</detail>
> > > > <reason>Operation Failed</reason>
> > > > </fault>
> > > >
> > > > So is this really the only way to do this: shutdown everything and put
> > > > other storage domains into maintenance? It would have been nice if
> > > > this information was in the upgrade guide. It made this seem so easy
> > > > and simple...
> > > >
> > > > Thanks,
> > > > Juhani
> > > >
> > > > On Tue, Apr 27, 2021 at 10:47 AM Juhani Rautiainen
> > > > <juhani.rautiai...@gmail.com> wrote:
> > > > >
> > > > > Thanks this looks like what I'm looking for. I'm still wondering how
> > > > > to use this. I have LUN just for new hosted storage. Ansible created
> > > > > storage domain to it correctly but just can't activate it. So is the
> > > > > idea that I activate this unattached hosted_storage domain and try to
> > > > > use API to make it master? I attached a screen shot how it looks
> > > > > currently.
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Apr 27, 2021 at 10:41 AM Yedidyah Bar David <d...@redhat.com>
> > > > > wrote:
> > > > > >
> > > > > > On Tue, Apr 27, 2021 at 10:15 AM Juhani Rautiainen
> > > > > > <juhani.rautiai...@gmail.com> wrote:
> > > > > > >
> > > > > > > To continue. I noticed that another storage domain took the data
> > > > > > > (master) now. I saw one advice that you can force change by
> > > > > > > putting
> > > > > > > the storage domain to maintenance mode. Problem is that there are
> > > > > > > VM's
> > > > > > > running on these domains. How is this supposed to work during the
> > > > > > > restore?
> > > > > >
> > > > > > There is a recent change [1] by Shani (Cced) that should allow you
> > > > > > to choose another storage domain as master. So you can create a new
> > > > > > (temporary?) SD with the correct compatibility level and then set it
> > > > > > to master.
> > > > > >
> > > > > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1576923
> > > > > >
> > > > > > Best regards,
> > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Juhani
> > > > > > >
> > > > > > > On Tue, Apr 27, 2021 at 9:58 AM Juhani Rautiainen
> > > > > > > <juhani.rautiai...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Hi!
> > > > > > > >
> > > > > > > > I started the upgrade from 4.3->4.4. Now I'm stuck as restoring
> > > > > > > > the
> > > > > > > > backup fails to create the correct storage domain for Hosted
> > > > > > > > Engine.
> > > > > > > > How can I create one? Error from ansible task is:
> > > > > > > >
> > > > > > > > [ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed".
> > > > > > > > Fault
> > > > > > > > detail is "[Domain format is different from master storage
> > > > > > > > domain
> > > > > > > > format]". HTTP response code is 400.
> > > > > > > > [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false,
> > > > > > > > "msg":
> > > > > > > > "Fault reason is \"Operation Failed\". Fault detail is \"[Domain
> > > > > > > > format is different from master storage domain format]\". HTTP
> > > > > > > > response code is 400."}
> > > > > > > >
> > > > > > > > And from UI I can see that what has been created is a data
> > > > > > > > domain. Not
> > > > > > > > data (master) domain as old one was. Now I'm stuck here. This
> > > > > > > > is a
> > > > > > > > fibre channel system where I'm trying to do this if it is
> > > > > > > > relevant.
> > > > > > > >
> > > > > > > > What I could find from the logs is this:
> > > > > > > > 2021-04-27 09:36:06,925+0300 DEBUG
> > > > > > > > otopi.ovirt_hosted_engine_setup.ansible_utils
> > > > > > > > ansible_utils._process_output:105 storage_domain_details:
> > > > > > > > {'changed':
> > > > > > > > False, 'ovirt_storage_domains': [{'href':
> > > > > > > > '/ovirt-engine/api/storagedomains/dd52022b-7616-47f6-9534-6f1a4084fdf4',
> > > > > > > > 'comment': '', 'description': '', 'id':
> > > > > > > > 'dd52022b-7616-47f6-9534-6f1a4084fdf4', 'name':
> > > > > > > > 'hosted_storage',
> > > > > > > > 'available': 531502202880, 'backup': False, 'block_size': 512,
> > > > > > > > 'committed': 0, 'critical_space_action_blocker': 5,
> > > > > > > > 'discard_after_delete': True, 'disk_profiles': [],
> > > > > > > > 'disk_snapshots':
> > > > > > > > [], 'disks': [], 'external_status': 'ok', 'master': False,
> > > > > > > > 'permissions': [], 'status': 'unattached', 'storage': {'type':
> > > > > > > > 'fcp',
> > > > > > > > 'volume_group': {'id': 'HRLDCn-p7X2-5X2O-vm4h-1Wb9-wAMu-WkIwit',
> > > > > > > > 'logical_units': [{'discard_max_size': 268435456,
> > > > > > > > 'discard_zeroes_data': False, 'id':
> > > > > > > > '36000d31005b4f6000000000000000029', 'lun_mapping': 3, 'paths':
> > > > > > > > 0,
> > > > > > > > 'product_id': 'Compellent Vol', 'serial':
> > > > > > > > 'SCOMPELNTCompellent_Vol_0005b4f6-00000029', 'size':
> > > > > > > > 536870912000,
> > > > > > > > 'storage_domain_id': 'dd52022b-7616-47f6-9534-6f1a4084fdf4',
> > > > > > > > 'vendor_id': 'COMPELNT', 'volume_group_id':
> > > > > > > > 'HRLDCn-p7X2-5X2O-vm4h-1Wb9-wAMu-WkIwit'}]}},
> > > > > > > > 'storage_connections':
> > > > > > > > [], 'storage_format': 'v5', 'supports_discard': True,
> > > > > > > > 'supports_discard_zeroes_data': False, 'templates': [], 'type':
> > > > > > > > 'data', 'used': 4294967296, 'vms': [],
> > > > > > > > 'warning_low_space_indicator':
> > > > > > > > 10, 'wipe_after_delete': False}], 'failed': False}
> > > > > > > >
> > > > > > > > 'master': False? I'm not sure if this creation or check. I
> > > > > > > > tried this
> > > > > > > > opration twice. I removed the new hosted_storage and remove
> > > > > > > > also old
> > > > > > > > hosted_engine domain on second try to make sure that it doesn't
> > > > > > > > prevent creation of another master. No luck with that.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Juhani
> > > > > > > _______________________________________________
> > > > > > > Users mailing list -- users@ovirt.org
> > > > > > > To unsubscribe send an email to users-le...@ovirt.org
> > > > > > > Privacy Statement: https://www.ovirt.org/privacy-policy.html
> > > > > > > oVirt Code of Conduct:
> > > > > > > https://www.ovirt.org/community/about/community-guidelines/
> > > > > > > List Archives:
> > > > > > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/TZHLFDIYRBQFVDLEVG4SWVZPXSVO7SKK/
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Didi
> > > > > >
> >
>
>
> --
> Didi
>
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/53Q5AR3BLX6XUKN7D76ITIH65DLSFVAO/
