[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-02 Thread Yedidyah Bar David
On Thu, Feb 3, 2022 at 5:53 AM Ayansh Rocks
 wrote:
>
> Hi All,
>
> How to clean stale hosted engine from the database. I want to remove host id=3

Perhaps by using the options '--clean_metadata' and '--host-id'
of the command 'hosted-engine'.
You can check documentation, list archives, etc. for details.

Good luck and best regards,
-- 
Didi
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/XM74P2MKDW37GXX5ZTXHR5POQ3HE2IVJ/


[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-02 Thread Ayansh Rocks
Hi Yedidyah,

I am thinking to do below -

# hosted-engine --set-maintenance --mode=global
# hosted-engine --clean-metadata --host-id= --force-clean"

I hope it is safe and tested ?

Thanks

On Thu, Feb 3, 2022 at 12:17 PM Yedidyah Bar David  wrote:

> On Thu, Feb 3, 2022 at 5:53 AM Ayansh Rocks
>  wrote:
> >
> > Hi All,
> >
> > How to clean stale hosted engine from the database. I want to remove
> host id=3
>
> Perhaps by using the options '--clean_metadata' and '--host-id'
> of the command 'hosted-engine'.
> You can check documentation, list archives, etc. for details.
>
> Good luck and best regards,
> --
> Didi
>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z6JENPJHQ5TF5JXMPX3DZFR3ZAQETA53/


[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-02 Thread Yedidyah Bar David
On Thu, Feb 3, 2022 at 9:11 AM Ayansh Rocks
 wrote:
>
> Hi Yedidyah,
>
> I am thinking to do below -
>
> # hosted-engine --set-maintenance --mode=global
> # hosted-engine --clean-metadata --host-id= --force-clean"
>
> I hope it is safe and tested ?

It is tested, and it's "safe" as in "Does what it's designed to do".

It's definitely not "safe" as in "riskless" - if you make a mistake,
you risk damage. I didn't use it myself recently, and do not remember
details anymore. It's definitely not something that you are supposed
to use/need casually.

If unsure, and if your environment is important to you (not just some
test/dev/learn thing), I warmly suggest to try this first on a test
env (can be done with nested-kvm VMs, no need for physical machines
for this).

That said, others on this list (and elsewhere) did use it successfully,
and I recommend again to search for examples/reports/experiences.

Good luck and best regards,

>
> Thanks
>
> On Thu, Feb 3, 2022 at 12:17 PM Yedidyah Bar David  wrote:
>>
>> On Thu, Feb 3, 2022 at 5:53 AM Ayansh Rocks
>>  wrote:
>> >
>> > Hi All,
>> >
>> > How to clean stale hosted engine from the database. I want to remove host 
>> > id=3
>>
>> Perhaps by using the options '--clean_metadata' and '--host-id'
>> of the command 'hosted-engine'.
>> You can check documentation, list archives, etc. for details.
>>
>> Good luck and best regards,
>> --
>> Didi
>>


-- 
Didi
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/6RFT4X3OGJPSF4AMQO55XKMSQOVM2SKD/


[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-03 Thread Ayansh Rocks
Hi Yedidyah,

Thanks, After cleaning it from one host i am getting below error messages
on /var/log/messages

Feb  3 18:01:02 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:17 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:18 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes
Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
3: received 0 of 512 expected bytes


On Thu, Feb 3, 2022 at 1:04 PM Yedidyah Bar David  wrote:

> On Thu, Feb 3, 2022 at 9:11 AM Ayansh Rocks
>  wrote:
> >
> > Hi Yedidyah,
> >
> > I am thinking to do below -
> >
> > # hosted-engine --set-maintenance --mode=global
> > # hosted-engine --clean-metadata --host-id= --force-clean"
> >
> > I hope it is safe and tested ?
>
> It is tested, and it's "safe" as in "Does what it's designed to do".
>
> It's definitely not "safe" as in "riskless" - if you make a mistake,
> you risk damage. I didn't use it myself recently, and do not remember
> details anymore. It's definitely not something that you are supposed
> to use/need casually.
>
> If unsure, and if your environment is important to you (not just some
> test/dev/learn thing), I warmly suggest to try this first on a test
> env (can be done with nested-kvm VMs, no need for physical machines
> for this).
>
> That said, others on this list (and elsewhere) did use it successfully,
> and I recommend again to search for examples/reports/experiences.
>
> Good luck and best regards,
>
> >
> > Thanks
> >
> > On Thu, Feb 3, 2022 at 12:17 PM Yedidyah Bar David 
> wrote:
> >>
> >> On Thu, Feb 3, 2022 at 5:53 AM Ayansh Rocks
> >>  wrote:
> >> >
> >> > Hi All,
> >> >
> >> > How to clean stale hosted engine from the database. I want to remove
> host id=3
> >>
> >> Perhaps by using the options '--clean_metadata' and '--host-id'
> >> of the command 'hosted-engine'.
> >> You can check documentation, list archives, etc. for details.
> >>
> >> Good luck and best regards,
> >> --
> >> Didi
> >>
>
>
> --
> Didi
>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/OOOTBJUH2I27UF3QLU2CEECQIFRINP7F/


[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-03 Thread Ayansh Rocks
Do I need to clean it from every host which has the hosted engine ? Sinc
the other host still can see it.

On Thu, Feb 3, 2022 at 6:02 PM Ayansh Rocks 
wrote:

> Hi Yedidyah,
>
> Thanks, After cleaning it from one host i am getting below error messages
> on /var/log/messages
>
> Feb  3 18:01:02 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:17 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:18 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
> Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host
> 3: received 0 of 512 expected bytes
>
>
> On Thu, Feb 3, 2022 at 1:04 PM Yedidyah Bar David  wrote:
>
>> On Thu, Feb 3, 2022 at 9:11 AM Ayansh Rocks
>>  wrote:
>> >
>> > Hi Yedidyah,
>> >
>> > I am thinking to do below -
>> >
>> > # hosted-engine --set-maintenance --mode=global
>> > # hosted-engine --clean-metadata --host-id= --force-clean"
>> >
>> > I hope it is safe and tested ?
>>
>> It is tested, and it's "safe" as in "Does what it's designed to do".
>>
>> It's definitely not "safe" as in "riskless" - if you make a mistake,
>> you risk damage. I didn't use it myself recently, and do not remember
>> details anymore. It's definitely not something that you are supposed
>> to use/need casually.
>>
>> If unsure, and if your environment is important to you (not just some
>> test/dev/learn thing), I warmly suggest to try this first on a test
>> env (can be done with nested-kvm VMs, no need for physical machines
>> for this).
>>
>> That said, others on this list (and elsewhere) did use it successfully,
>> and I recommend again to search for examples/reports/experiences.
>>
>> Good luck and best regards,
>>
>> >
>> > Thanks
>> >
>> > On Thu, Feb 3, 2022 at 12:17 PM Yedidyah Bar David 
>> wrote:
>> >>
>> >> On Thu, Feb 3, 2022 at 5:53 AM Ayansh Rocks
>> >>  wrote:
>> >> >
>> >> > Hi All,
>> >> >
>> >> > How to clean stale hosted engine from the database. I want to remove
>> host id=3
>> >>
>> >> Perhaps by using the options '--clean_metadata' and '--host-id'
>> >> of the command 'hosted-engine'.
>> >> You can check documentation, list archives, etc. for details.
>> >>
>> >> Good luck and best regards,
>> >> --
>> >> Didi
>> >>
>>
>>
>> --
>> Didi
>>
>>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IR3SMGNMRRGEKXUXTMCZBINAX6XZDRM5/


[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-03 Thread Yedidyah Bar David
On Thu, Feb 3, 2022 at 2:48 PM Ayansh Rocks
 wrote:
>
> Do I need to clean it from every host which has the hosted engine ? Sinc the 
> other host still can see it.

Are you sure it's reading the same space? Perhaps there is some
misconfiguration or something? Caching?

>
> On Thu, Feb 3, 2022 at 6:02 PM Ayansh Rocks  
> wrote:
>>
>> Hi Yedidyah,
>>
>> Thanks, After cleaning it from one host i am getting below error messages on 
>> /var/log/messages
>>
>> Feb  3 18:01:02 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:17 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:18 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes
>> Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 
>> 3: received 0 of 512 expected bytes

Not sure. Perhaps try to restart the ha broker, agent or both.
If you do this in global maintenance, I think it should be safe -
wait until everything looks ok before exiting global maint.

Check also vdsm and ha logs.

Good luck and best regards,
-- 
Didi
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/6RNYC7C6GNWXS46IFNJLRBNOSLKDGD62/


[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-03 Thread Martin Sivak
On Thu, Feb 3, 2022 at 1:50 PM Ayansh Rocks 
wrote:

> Do I need to clean it from every host which has the hosted engine ? Sinc
> the other host still can see it.
>

No, the metadata "whiteboard" is shared. Just one copy that you removed.
Once node 3 start reporting again, it will simply put new content in there.

Not sure about vdsm, maybe some component (vdsm, engine) is still trying to
get status of node 3. I have been away for too long, so Didi will have to
pitch in.

Martin Sivak



>
> On Thu, Feb 3, 2022 at 6:02 PM Ayansh Rocks 
> wrote:
>
>> Hi Yedidyah,
>>
>> Thanks, After cleaning it from one host i am getting below error messages
>> on /var/log/messages
>>
>> Feb  3 18:01:02 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:17 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:18 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>> Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>> host 3: received 0 of 512 expected bytes
>>
>>
>> On Thu, Feb 3, 2022 at 1:04 PM Yedidyah Bar David 
>> wrote:
>>
>>> On Thu, Feb 3, 2022 at 9:11 AM Ayansh Rocks
>>>  wrote:
>>> >
>>> > Hi Yedidyah,
>>> >
>>> > I am thinking to do below -
>>> >
>>> > # hosted-engine --set-maintenance --mode=global
>>> > # hosted-engine --clean-metadata --host-id= --force-clean"
>>> >
>>> > I hope it is safe and tested ?
>>>
>>> It is tested, and it's "safe" as in "Does what it's designed to do".
>>>
>>> It's definitely not "safe" as in "riskless" - if you make a mistake,
>>> you risk damage. I didn't use it myself recently, and do not remember
>>> details anymore. It's definitely not something that you are supposed
>>> to use/need casually.
>>>
>>
It should be safe in global maintenance, the only risk is when you remove
the report of a node that currently runs the engine. As other nodes might
decide that the engine needs restarting.



> If unsure, and if your environment is important to you (not just some
>>> test/dev/learn thing), I warmly suggest to try this first on a test
>>> env (can be done with nested-kvm VMs, no need for physical machines
>>> for this).
>>>
>>> That said, others on this list (and elsewhere) did use it successfully,
>>> and I recommend again to search for examples/reports/experiences.
>>>
>>> Good luck and best regards,
>>>
>>> >
>>> > Thanks
>>> >
>>> > On Thu, Feb 3, 2022 at 12:17 PM Yedidyah Bar David 
>>> wrote:
>>> >>
>>> >> On Thu, Feb 3, 2022 at 5:53 AM Ayansh Rocks
>>> >>  wrote:
>>> >> >
>>> >> > Hi All,
>>> >> >
>>> >> > How to clean stale hosted engine from the database. I want to
>>> remove host id=3
>>> >>
>>> >> Perhaps by using the options '--clean_metadata' and '--host-id'
>>> >> of the command 'hosted-engine'.
>>> >> You can check documentation, list archives, etc. for details.
>>> >>
>>> >> Good luck and best regards,
>>> >> --
>>> >> Didi
>>> >>
>>>
>>>
>>> --
>>> Didi
>>>
>>> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/IR3SMGNMRRGEKXUXTMCZBINAX6XZDRM5/
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/C5QZMCH5VSXZRWAG65FOWGX6MV3YZYSO/


[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-03 Thread Ayansh Rocks
VDSM Logs -

ual': True}} from=internal, task_id=b017c5d2-a941-4ccd-af02-3fac0a767df5
(api:54)
2022-02-03 18:53:53,368+0530 ERROR (periodic/3)
[ovirt_hosted_engine_ha.client.client.HAClient] Malformed metadata for host
3: received 0 of 512 expected bytes (client:137)
2022-02-03 18:53:54,529+0530 INFO  (jsonrpc/5) [api.host] START getStats()
from=::1,38990 (api:48)
2022-02-03 18:53:54,543+0530 INFO  (jsonrpc/5) [vdsm.api] START
repoStats(domains=()) from=::1,38990,
task_id=3864f22e-98cc-426f-a4d5-2e2254f34aae (api:48)
2022-02-03 18:53:54,543+0530 INFO  (jsonrpc/5) [vdsm.api] FINISH repoStats
return={'8ab8b572-39a3-4603-b0e7-2b6fe71c4d73': {'code': 0, 'lastCheck':
'6.3', 'delay': '0.000359743', 'valid': True, 'version': 5, 'acquired':
True, 'actual': True}, 'f3114b74-8b12-4493-9faf-0f514afa3a16': {'code': 0,
'lastCheck': '2.1', 'delay': '0.000447464', 'valid': True, 'version': 5,
'acquired': True, 'actual': True}, '26bfbe5d-e427-47a9-9111-cca8a628fea8':
{'code': 0, 'lastCheck': '2.0', 'delay': '0.00034851', 'valid': True,
'version': 5, 'acquired': True, 'actual': True},
'23540fa8-4de2-4715-97e1-4ce9fe4f66cc': {'code': 0, 'lastCheck': '2.1',
'delay': '0.000352501', 'valid': True, 'version': 5, 'acquired': True,
'actual': True}, '8fa0a28e-480f-444c-9cf2-bef536d3c52f': {'code': 0,
'lastCheck': '2.0', 'delay': '0.000320087', 'valid': True, 'version': 5,
'acquired': True, 'actual': True}, 'd1dba7f5-cb30-4571-8fe6-5fb5005bcdb2':
{'code': 0, 'lastCheck': '2.6', 'delay': '0.000452836', 'valid': True,
'version': 0, 'acquired': True, 'actual': True},
'88b94c9a-6855-4199-9d37-1f88d21f211c': {'code': 0, 'lastCheck': '2.0',
'delay': '0.000299592', 'valid': True, 'version': 5, 'acquired': True,
'actual': True}, '2e0ec671-5972-4276-aa49-aefc306f5da4': {'code': 0,
'lastCheck': '2.0', 'delay': '0.000316513', 'valid': True, 'version': 5,
'acquired': True, 'actual': True}} from=::1,38990,
task_id=3864f22e-98cc-426f-a4d5-2e2254f34aae (api:54)
2022-02-03 18:53:54,543+0530 INFO  (jsonrpc/5) [vdsm.api] START
multipath_health() from=::1,38990,
task_id=7ecaad0c-aa5d-45c0-96a5-dffc78572248 (api:48)
2022-02-03 18:53:54,543+0530 INFO  (jsonrpc/5) [vdsm.api] FINISH
multipath_health return={} from=::1,38990,
task_id=7ecaad0c-aa5d-45c0-96a5-dffc78572248 (api:54)
2022-02-03 18:53:54,548+0530 ERROR (jsonrpc/5)
[ovirt_hosted_engine_ha.client.client.HAClient] Malformed metadata for host
3: received 0 of 512 expected bytes (client:137)
2022-02-03 18:53:54,549+0530 INFO  (jsonrpc/5) [api.host] FINISH getStats
return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)}
from=::1,38990 (api:54)
2022-02-03 18:53:56,038+0530 INFO  (jsonrpc/4) [vdsm.api] START
repoStats(domains=['8ab8b572-39a3-4603-b0e7-2b6fe71c4d73']) from=::1,38990,
task_id=8855f314-9334-4b06-88e3-7a1a3f577049 (api:48)
2022-02-03 18:53:56,038+0530 INFO  (jsonrpc/4) [vdsm.api] FINISH repoStats
return={'8ab8b572-39a3-4603-b0e7-2b6fe71c4d73': {'code': 0, 'lastCheck':
'1.3', 'delay': '0.000349788', 'valid': True, 'version': 5, 'acquired':
True, 'actual': True}} from=::1,38990,
task_id=8855f314-9334-4b06-88e3-7a1a3f577049 (api:54)



On Thu, Feb 3, 2022 at 6:42 PM Martin Sivak  wrote:

>
>
> On Thu, Feb 3, 2022 at 1:50 PM Ayansh Rocks 
> wrote:
>
>> Do I need to clean it from every host which has the hosted engine ? Sinc
>> the other host still can see it.
>>
>
> No, the metadata "whiteboard" is shared. Just one copy that you removed.
> Once node 3 start reporting again, it will simply put new content in there.
>
> Not sure about vdsm, maybe some component (vdsm, engine) is still trying
> to get status of node 3. I have been away for too long, so Didi will have
> to pitch in.
>
> Martin Sivak
>
>
>
>>
>> On Thu, Feb 3, 2022 at 6:02 PM Ayansh Rocks 
>> wrote:
>>
>>> Hi Yedidyah,
>>>
>>> Thanks, After cleaning it from one host i am getting below error
>>> messages on /var/log/messages
>>>
>>> Feb  3 18:01:02 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:17 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:18 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for
>>> host 3: received 0 of 512 expected bytes
>>> Feb  3 18:01:23 iondelsvr14 vdsm[1

[ovirt-users] Re: How to clean stale engine from "hosted-engine unknow stale-data"

2022-02-03 Thread Ayansh Rocks
I restarted the host and it seems like it's running fine...However do it
need to reboot other hosts or restart some services on other hosts as well
clean this up. Because they still can see the Host ID=3
[root@iondelsvr10 vdsm]# hosted-engine --vm-status


--== Host iondelsvr10.iontrading.com (id: 1) status ==--

Host ID: 1
Host timestamp : 2672943
Score  : 3400
Engine status  : {"vm": "down", "health": "bad",
"detail": "unknown", "reason": "vm not running on this host"}
Hostname   : iondelsvr10.iontrading.com
Local maintenance  : False
stopped: False
crc32  : b737573e
conf_on_shared_storage : True
local_conf_timestamp   : 2672943
Status up-to-date  : True
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=2672943 (Thu Feb  3 20:09:34 2022)
host-id=1
score=3400
vm_conf_refresh_time=2672943 (Thu Feb  3 20:09:34 2022)
conf_on_shared_storage=True
maintenance=False
state=EngineDown
stopped=False


--== Host iondelsvr11.iontrading.com (id: 2) status ==--

Host ID: 2
Host timestamp : 6255602
Score  : 3400
Engine status  : {"vm": "down", "health": "bad",
"detail": "unknown", "reason": "vm not running on this host"}
Hostname   : iondelsvr11.iontrading.com
Local maintenance  : False
stopped: False
crc32  : 5ff71c1a
conf_on_shared_storage : True
local_conf_timestamp   : 6255602
Status up-to-date  : True
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=6255602 (Thu Feb  3 20:09:24 2022)
host-id=2
score=3400
vm_conf_refresh_time=6255602 (Thu Feb  3 20:09:24 2022)
conf_on_shared_storage=True
maintenance=False
state=EngineDown
stopped=False


--== Host iondelsvr12.iontrading.com (id: 3) status ==--

Host ID: 3
Host timestamp : 734
Score  : 1800
Engine status  : unknown stale-data
Hostname   : iondelsvr12.iontrading.com
Local maintenance  : False
stopped: False
crc32  : 062e92e9
conf_on_shared_storage : True
local_conf_timestamp   : 734
Status up-to-date  : False
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=734 (Fri Jan 28 17:35:18 2022)
host-id=3
score=1800
vm_conf_refresh_time=734 (Fri Jan 28 17:35:18 2022)
conf_on_shared_storage=True
maintenance=False
state=EngineDown
stopped=False


--== Host iondelsvr14.iontrading.com (id: 4) status ==--

Host ID: 4
Host timestamp : 2670
Score  : 3400
Engine status  : {"vm": "up", "health": "good",
"detail": "Up"}
Hostname   : iondelsvr14.iontrading.com
Local maintenance  : False
stopped: False
crc32  : fdf46b17
conf_on_shared_storage : True
local_conf_timestamp   : 2670
Status up-to-date  : True
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=2670 (Thu Feb  3 20:09:30 2022)
host-id=4
score=3400
vm_conf_refresh_time=2670 (Thu Feb  3 20:09:30 2022)
conf_on_shared_storage=True
maintenance=False
state=EngineUp
stopped=False
[root@iondelsvr10 vdsm]#

On Thu, Feb 3, 2022 at 6:54 PM Ayansh Rocks 
wrote:

> VDSM Logs -
>
> ual': True}} from=internal, task_id=b017c5d2-a941-4ccd-af02-3fac0a767df5
> (api:54)
> 2022-02-03 18:53:53,368+0530 ERROR (periodic/3)
> [ovirt_hosted_engine_ha.client.client.HAClient] Malformed metadata for host
> 3: received 0 of 512 expected bytes (client:137)
> 2022-02-03 18:53:54,529+0530 INFO  (jsonrpc/5) [api.host] START getStats()
> from=::1,38990 (api:48)
> 2022-02-03 18:53:54,543+0530 INFO  (jsonrpc/5) [vdsm.api] START
> repoStats(domains=()) from=::1,38990,
> task_id=3864f22e-98cc-426f-a4d5-2e2254f34aae (api:48)
> 2022-02-03 18:53:54,543+0530 INFO  (jsonrpc/5) [vdsm.api] FINISH repoStats
> return={'8ab8b572-39a3-4603-b0e7-2b6fe71c4d73': {'code': 0, 'lastCheck':
> '6.3', 'delay': '0.000359743', 'valid': True, 'version': 5, 'acquired':
> Tr