[ovirt-devel] Re: please stop merging

[Dafna Ron]

ok :)
I think a decision has been made so we'll push forward and once everything
in moved to rhel8 we will start flooding the issues and try to get it all
fixed as quickly as possible.

Thanks everyone
Dafna


On Thu, Feb 13, 2020 at 10:18 AM Yedidyah Bar David  wrote:

> On Thu, Feb 13, 2020 at 12:09 PM Dafna Ron  wrote:
> >
> >
> >
> > On Wed, Feb 12, 2020 at 11:37 PM Michal Skrivanek <
> michal.skriva...@redhat.com> wrote:
> >>
> >>
> >>
> >> On 12 Feb 2020, at 13:48, Dafna Ron  wrote:
> >>
> >>
> >>
> >> On Wed, Feb 12, 2020 at 6:38 PM Martin Perina 
> wrote:
> >>>
> >>>
> >>>
> >>> On Wed, Feb 12, 2020 at 6:33 PM Dafna Ron  wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Feb 12, 2020 at 5:03 PM Martin Perina 
> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I'm probably missing something here, so please correct me if I'm
> wrong:
> >>>>>
> >>>>> 1. CQ is running OST to detect failures and if OST will not pass,
> RPM with the change will not land in tested repo, right?
> >>>>
> >>>> unless someone manually moves packages into the tested repo - which
> caused all projects to fail on engine deploy.
> >>>>>
> >>>>>
> >>>>> 2. OST is still running engine on EL7, right?
> >>>>
> >>>> yes
> >>>>>
> >>>>>
> >>>>> 3. We have dropped EL7 builds from engine on Feb 6th
> https://gerrit.ovirt.org/106795
> >>>>> So no matter what we push to engine repo, it will not pass the
> CQ, because it will still be testing the old broken version (because new
> builds will produce packages only for FC30 and EL8 repos). Am I right or
> missing something?
> >>>>>
> >>>> Actually, engine has been failing for more then 4 weeks already on
> unrelated issues. currently its failing on a package which is deprecated
> because the change has not passed CQ and the package that is tested is
> based on some un-merged change. so fixing the issues now, have nothing to
> do with the missing el8 packages and the longer you wait to fix the
> failures, the more regressions you will introduce and harder it will be to
> fix it.
> >>>>
> >>>> At this point, I need to have patch
> https://gerrit.ovirt.org/#/c/106809/ run in CQ so I can see why its
> failing.
> >>>
> >>>
> >>> Well, there are other fixes after this patch which fixes known issues,
> so I'm not sure if only this fix will make the OST to run successfully.
> >>>
> >>>>
> >>>>> Right now we are doing our best to make engine running on CentOS 8
> (my personal estimation is Friday), so after that we should be able to
> merge OST patch https://gerrit.ovirt.org/106824 so OST will use EL8 for
> engine.
> >>>>>
> >>>>>
> >>>>> So unless I'm missing something it doesn't make sense to block
> merging to engine and its dependencies, because we need to make them
> running on EL8 and switch engine on EL8 in OST to unblock CQ. Am I right or
> am I missing something?
> >>>>
> >>>>
> >>>> lets first get to a point where we are blocked by rl8 related issues
> instead of merging more changes which we later on cannot track.
> >>>> Lets see what the change is failing as for now, you are blocking all
> other projects (not just engine and its deps).
> >>>
> >>>
> >>> Well maybe, but again I don't believe that we are able to stabilize
> OST before we get to engine EL8, because since last week there were merged
> several patches which might have help OST but also several fixes which
> broke engine on EL7. And all those fixes are mixed.
> >>>
> >>> So from my point right now the biggest priority is to make engine
> running on EL8 asap and only afterwards focus on OST stabilization and
> fixing other less important dependencies
> >>
> >>
> >> However we should (engine maintainers should) be cautious about merging
> other stuff unrelated to el8. If it’s not essential I would suggest to
> postpone merging exactly for that reason that we don’t have any validation
> beyond unit tests.
> >>
> >>
> >> OST is not related at all to this and its not about stabilizing OST but
> about stabilizing engine as these are code issues and not random OST
> failur

[ovirt-devel] Re: please stop merging

[Dafna Ron]

On Wed, Feb 12, 2020 at 11:37 PM Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
>
> On 12 Feb 2020, at 13:48, Dafna Ron  wrote:
>
>
>
> On Wed, Feb 12, 2020 at 6:38 PM Martin Perina  wrote:
>
>>
>>
>> On Wed, Feb 12, 2020 at 6:33 PM Dafna Ron  wrote:
>>
>>>
>>>
>>> On Wed, Feb 12, 2020 at 5:03 PM Martin Perina 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm probably missing something here, so please correct me if I'm wrong:
>>>>
>>>> 1. CQ is running OST to detect failures and if OST will not pass, RPM
>>>> with the change will not land in tested repo, right?
>>>>
>>> unless someone manually moves packages into the tested repo - which
>>> caused all projects to fail on engine deploy.
>>>
>>>>
>>>> 2. OST is still running engine on EL7, right?
>>>>
>>> yes
>>>
>>>>
>>>> 3. We have dropped EL7 builds from engine on Feb 6th
>>>> https://gerrit.ovirt.org/106795
>>>> So no matter what we push to engine repo, it will not pass the CQ,
>>>> because it will still be testing the old broken version (because new builds
>>>> will produce packages only for FC30 and EL8 repos). Am I right or missing
>>>> something?
>>>>
>>>> Actually, engine has been failing for more then 4 weeks already on
>>> unrelated issues. currently its failing on a package which is deprecated
>>> because the change has not passed CQ and the package that is tested is
>>> based on some un-merged change. so fixing the issues now, have nothing to
>>> do with the missing el8 packages and the longer you wait to fix the
>>> failures, the more regressions you will introduce and harder it will be to
>>> fix it.
>>>
>>> At this point, I need to have patch https://gerrit.ovirt.org/#/c/106809/
>>> run in CQ so I can see why its failing.
>>>
>>
>> Well, there are other fixes after this patch which fixes known issues, so
>> I'm not sure if only this fix will make the OST to run successfully.
>>
>>
>>> Right now we are doing our best to make engine running on CentOS 8 (my
>>>> personal estimation is Friday), so after that we should be able to merge
>>>> OST patch https://gerrit.ovirt.org/106824 so OST will use EL8 for
>>>> engine.
>>>>
>>>
>>>> So unless I'm missing something it doesn't make sense to block merging
>>>> to engine and its dependencies, because we need to make them running on EL8
>>>> and switch engine on EL8 in OST to unblock CQ. Am I right or am I missing
>>>> something?
>>>>
>>>
>>> lets first get to a point where we are blocked by rl8 related issues
>>> instead of merging more changes which we later on cannot track.
>>> Lets see what the change is failing as for now, you are blocking all
>>> other projects (not just engine and its deps).
>>>
>>
>> Well maybe, but again I don't believe that we are able to stabilize OST
>> before we get to engine EL8, because since last week there were merged
>> several patches which might have help OST but also several fixes which
>> broke engine on EL7. And all those fixes are mixed.
>>
>> So from my point right now the biggest priority is to make engine running
>> on EL8 asap and only afterwards focus on OST stabilization and fixing other
>> less important dependencies
>>
>
> However we should (engine maintainers should) be cautious about merging
> other stuff unrelated to el8. If it’s not essential I would suggest to
> postpone merging exactly for that reason that we don’t have any validation
> beyond unit tests.
>
>
> OST is not related at all to this and its not about stabilizing OST but
> about stabilizing engine as these are code issues and not random OST
> failures.
>
>
> It’s related in a sense that as soon as we get OST on el8 running we can
> sleep a bit better when merging engine patches.
>
> The failures in cq are engine related and the rest of the projects are
> also failing because a faulty engine package has been introduced to the
> rest of the projects so everyone are merging "blind" because of the current
> failures.
>
>
> You’re right about the other projects, but we can as well just go back to
> a previous build then. Just remove the last ovirt-engine manually?
>

We do not have a roll back on the repo but perhaps Anton would have an
idea.


> you can continue merging

[ovirt-devel] Re: please stop merging

[Dafna Ron]

On Wed, Feb 12, 2020 at 6:38 PM Martin Perina  wrote:

>
>
> On Wed, Feb 12, 2020 at 6:33 PM Dafna Ron  wrote:
>
>>
>>
>> On Wed, Feb 12, 2020 at 5:03 PM Martin Perina  wrote:
>>
>>> Hi,
>>>
>>> I'm probably missing something here, so please correct me if I'm wrong:
>>>
>>> 1. CQ is running OST to detect failures and if OST will not pass, RPM
>>> with the change will not land in tested repo, right?
>>>
>> unless someone manually moves packages into the tested repo - which
>> caused all projects to fail on engine deploy.
>>
>>>
>>> 2. OST is still running engine on EL7, right?
>>>
>> yes
>>
>>>
>>> 3. We have dropped EL7 builds from engine on Feb 6th
>>> https://gerrit.ovirt.org/106795
>>> So no matter what we push to engine repo, it will not pass the CQ,
>>> because it will still be testing the old broken version (because new builds
>>> will produce packages only for FC30 and EL8 repos). Am I right or missing
>>> something?
>>>
>>> Actually, engine has been failing for more then 4 weeks already on
>> unrelated issues. currently its failing on a package which is deprecated
>> because the change has not passed CQ and the package that is tested is
>> based on some un-merged change. so fixing the issues now, have nothing to
>> do with the missing el8 packages and the longer you wait to fix the
>> failures, the more regressions you will introduce and harder it will be to
>> fix it.
>>
>> At this point, I need to have patch https://gerrit.ovirt.org/#/c/106809/
>> run in CQ so I can see why its failing.
>>
>
> Well, there are other fixes after this patch which fixes known issues, so
> I'm not sure if only this fix will make the OST to run successfully.
>
>
>> Right now we are doing our best to make engine running on CentOS 8 (my
>>> personal estimation is Friday), so after that we should be able to merge
>>> OST patch https://gerrit.ovirt.org/106824 so OST will use EL8 for
>>> engine.
>>>
>>
>>> So unless I'm missing something it doesn't make sense to block merging
>>> to engine and its dependencies, because we need to make them running on EL8
>>> and switch engine on EL8 in OST to unblock CQ. Am I right or am I missing
>>> something?
>>>
>>
>> lets first get to a point where we are blocked by rl8 related issues
>> instead of merging more changes which we later on cannot track.
>> Lets see what the change is failing as for now, you are blocking all
>> other projects (not just engine and its deps).
>>
>
> Well maybe, but again I don't believe that we are able to stabilize OST
> before we get to engine EL8, because since last week there were merged
> several patches which might have help OST but also several fixes which
> broke engine on EL7. And all those fixes are mixed.
>
> So from my point right now the biggest priority is to make engine running
> on EL8 asap and only afterwards focus on OST stabilization and fixing other
> less important dependencies
>

OST is not related at all to this and its not about stabilizing OST but
about stabilizing engine as these are code issues and not random OST
failures. The failures in cq are engine related and the rest of the
projects are also failing because a faulty engine package has been
introduced to the rest of the projects so everyone are merging "blind"
because of the current failures.
you can continue merging if you like, but note that all the projects in CQ
have no testing coverage at all at this time and you are merging without
any CI testing.



> Regards,
> Martin
>
>
> On Wed, Feb 12, 2020 at 4:03 PM Dafna Ron  wrote:
>
>> Hi all,
>>
>> Engine has been failing for a while and someone has also moved the broken
>> package manually to the tested repo which broke all projects in cq.
>> At this time, until we stabilise engine and get a new package into
>> tested, all projects will continue failing cq and no new packages will be
>> moved to the stable repo (tested).
>> if you continue to merge, we not only risk adding more regression but it
>> also makes it more difficult to debug and fix as there are a lot of changes
>> in the queue waiting to be tested and very little resources to run it.
>>
>> Please stop merging on all projects until this is resolved.
>>
>> Thanks,
>> Dafna
>>
>>
>>
>>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
>

>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/OJXQXGU4KG6WLFKNK2UAYPYXK4UQTGR6/

[ovirt-devel] Re: please stop merging

[Dafna Ron]

On Wed, Feb 12, 2020 at 5:03 PM Martin Perina  wrote:

> Hi,
>
> I'm probably missing something here, so please correct me if I'm wrong:
>
> 1. CQ is running OST to detect failures and if OST will not pass, RPM with
> the change will not land in tested repo, right?
>
unless someone manually moves packages into the tested repo - which caused
all projects to fail on engine deploy.

>
> 2. OST is still running engine on EL7, right?
>
yes

>
> 3. We have dropped EL7 builds from engine on Feb 6th
> https://gerrit.ovirt.org/106795
> So no matter what we push to engine repo, it will not pass the CQ,
> because it will still be testing the old broken version (because new builds
> will produce packages only for FC30 and EL8 repos). Am I right or missing
> something?
>
> Actually, engine has been failing for more then 4 weeks already on
unrelated issues. currently its failing on a package which is deprecated
because the change has not passed CQ and the package that is tested is
based on some un-merged change. so fixing the issues now, have nothing to
do with the missing el8 packages and the longer you wait to fix the
failures, the more regressions you will introduce and harder it will be to
fix it.

At this point, I need to have patch https://gerrit.ovirt.org/#/c/106809/
run in CQ so I can see why its failing.

Right now we are doing our best to make engine running on CentOS 8 (my
> personal estimation is Friday), so after that we should be able to merge
> OST patch https://gerrit.ovirt.org/106824 so OST will use EL8 for engine.
>

> So unless I'm missing something it doesn't make sense to block merging to
> engine and its dependencies, because we need to make them running on EL8
> and switch engine on EL8 in OST to unblock CQ. Am I right or am I missing
> something?
>

lets first get to a point where we are blocked by rl8 related issues
instead of merging more changes which we later on cannot track.
Lets see what the change is failing as for now, you are blocking all other
projects (not just engine and its deps).



> Regards,
> Martin
>
>
> On Wed, Feb 12, 2020 at 4:03 PM Dafna Ron  wrote:
>
>> Hi all,
>>
>> Engine has been failing for a while and someone has also moved the broken
>> package manually to the tested repo which broke all projects in cq.
>> At this time, until we stabilise engine and get a new package into
>> tested, all projects will continue failing cq and no new packages will be
>> moved to the stable repo (tested).
>> if you continue to merge, we not only risk adding more regression but it
>> also makes it more difficult to debug and fix as there are a lot of changes
>> in the queue waiting to be tested and very little resources to run it.
>>
>> Please stop merging on all projects until this is resolved.
>>
>> Thanks,
>> Dafna
>>
>>
>>
>>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/JAD2GZ3MQQ7U3NXWUHPZWUW2LVZEJDZS/

[ovirt-devel] please stop merging

[Dafna Ron]

Hi all,

Engine has been failing for a while and someone has also moved the broken
package manually to the tested repo which broke all projects in cq.
At this time, until we stabilise engine and get a new package into tested,
all projects will continue failing cq and no new packages will be moved to
the stable repo (tested).
if you continue to merge, we not only risk adding more regression but it
also makes it more difficult to debug and fix as there are a lot of changes
in the queue waiting to be tested and very little resources to run it.

Please stop merging on all projects until this is resolved.

Thanks,
Dafna
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/D76I2SGVNK4DEXGBMYE4DTRELJ4B56WU/

[ovirt-devel] Re: master CQ fails due to missing ovirt-host-deploy

[Dafna Ron]

The problem is that the package which is currently in tested is an
un-authorised package that is actually breaking all the projects and not
just engine.
it has been transferred manually as I cannot see a passing ovirt-engine cq
test for over a month and the package that is there is pointing at an
un-merged change.

We should not touch CQ manually ever as doing that breaks it completely and
makes it more difficult to find and fix any other issues and breaks all
other projects as well.

I have re-merged the fix that you pointed to to see what is causing it to
fail CQ and will update once it goes through cq and we can see what other
issues we have.

Thanks,
Dafna



On Wed, Feb 12, 2020 at 1:02 PM Martin Perina  wrote:

>
>
> On Wed, Feb 12, 2020 at 11:07 AM Dafna Ron  wrote:
>
>> Then another possibility is that the package is now suppose to be coming
>> from somewhere else and its not available?
>> where is it suppose to come from? from what I understood, it should be
>> deployed by ansible, is that correct?
>> I can see an abandoned patch but cannot see a patch that was merged for
>> ansible:
>> https://gerrit.ovirt.org/#/q/ovirt-host-deploy
>>
>> It seems patch: https://gerrit.ovirt.org/#/c/102959/ was abandoned for
>> no activity - was anything else actually merged on ansible side?
>>
>> Thanks,
>> Dafna
>>
>>
>> On Wed, Feb 12, 2020 at 9:47 AM Yedidyah Bar David 
>> wrote:
>>
>>> On Wed, Feb 12, 2020 at 11:39 AM Dafna Ron  wrote:
>>> >
>>> > this is the current package in tested:
>>> ovirt-engine-0:4.2.0-0.0.master.20170712084142.git63b81ef.el7.centos.noarch
>>> and since all projects are now failing on this, it means the engine package
>>> in tested is a broken package.
>>> > what I don't understand is how can CQ mark engine as failed since this
>>> patch: https://gerrit.ovirt.org/#/c/104391/ which is 4 weeks ago but I
>>> can see a new package in tested since 06-02-2020 with no passing cq test
>>> for ovirt-engine.
>>> > If anyone is moving packages manually to tested they are breaking all
>>> projects by introducing broken packages.
>>> >
>>> > As the package in tested is now broken, we should start fixing the
>>> code on each failing issue.
>>> > for now, the main thing that is failing is that we are looking for a
>>> package that no longer exists.
>>> > Didi, is it possible its the typo Marcin was commenting on in this
>>> patch: https://gerrit.ovirt.org/#/c/106797/
>>>
>>> I already addressed this,
>>>
>>
> https://gerrit.ovirt.org/106809 was a fix for that issue, if it still
> didn't pass the CQ, then let's pass the engine build this patch through CQ
> manually
>
>
>>> >
>>> > Thanks,
>>> > Dafna
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Tue, Feb 11, 2020 at 9:03 AM Yedidyah Bar David 
>>> wrote:
>>> >>
>>> >> On Tue, Feb 11, 2020 at 10:22 AM Dusan Fodor 
>>> wrote:
>>> >> >
>>> >> > Hello,
>>> >> > master CQ is failing on initialize_engine due to missing
>>> ovirt-host-deploy dependency.
>>> >> >
>>> >> > [ INFO  ] Checking for an update for Setup...
>>> >> > [ ERROR ] Yum Cannot queue package ovirt-host-deploy: Package
>>> ovirt-host-deploy cannot be found
>>> >> >
>>> >> >
>>> >> > I was under the impression that engine-setup should not require it
>>> anymore, but it seems the related patch [1] doesn't work as intended.
>>> Perhaps Marcin's comment there explains why? Can you please check this?
>>> >>
>>> >> How can I know which engine build was used for [2]?
>>> >>
>>> >> Checking lago.log there, I see:
>>> >>
>>> >> Installed:
>>> >>   net-snmp.x86_64 1:5.7.2-43.el7_7.3
>>> >>   ntp.x86_64 0:4.2.6p5-29.el7.centos
>>> >>   otopi-debug-plugins.noarch 0:1.9.0-1.el7
>>> >>   ovirt-engine.noarch 0:4.4.0-0.0.master.20200206140618.gitff1f0b4.el7
>>> >>   ovirt-engine-extension-aaa-ldap.noarch
>>> 0:1.3.11-0.0.master.gitb9a171f.el7
>>> >>   ovirt-engine-extension-aaa-ldap-setup.noarch
>>> >> 0:1.3.11-0.0.master.gitb9a171f.el7
>>> >>   ovirt-log-collector.noarch
>>> 0:4.4.0-0.0.master.20191122133512.gitfc29382.el7
>>> >>
>

[ovirt-devel] Re: OST network suite is failing

[Dafna Ron]

was the package created from this patch manually moved into tested repo?
I can see the package in tested and yet this patch
https://gerrit.ovirt.org/#/c/106794/ has not been merged

On Thu, Feb 6, 2020 at 6:22 PM Michal Skrivanek 
wrote:

> Just note that from this time on engine is going through breaking changes
> to bring it to el8. The CQ works in mysterious ways so in case anything new
> ends up in master repo it’s unlikely that you’ll get anywhere far in OST.
> Until the transition is finished
>
> On 6 Feb 2020, at 11:37, Andrej Krejcir  wrote:
>
> This patch should fix the NPE:
>
> https://gerrit.ovirt.org/#/c/106794/
>
> On Thu, 6 Feb 2020 at 10:59, Benny Zlotnik  wrote:
>
>> I think it was fixed[1], is the patch rebased?
>>
>> [1] https://gerrit.ovirt.org/#/c/106772/
>>
>> On Thu, Feb 6, 2020 at 11:53 AM Dominik Holler 
>> wrote:
>> >
>> > Hello Tal and Andrej,
>> > can you please have a look at the failing
>> >
>> https://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-master/1260/
>> >
>> > 2020-02-05 21:03:12,798-05 ERROR
>> [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (default task-2)
>> [03a09eb0-7ccd-44db-a8fd-c01dc8ce82c7] Error during ValidateFailure.:
>> java.lang.NullPointerException
>> > at
>> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.validator.storage.DiskVmElementValidator.isVirtIoScsiValid(DiskVmElementValidator.java:54)
>> > at
>> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.AddDiskCommand.checkIfImageDiskCanBeAdded(AddDiskCommand.java:299)
>> > at
>> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.AddDiskCommand.validate(AddDiskCommand.java:195)
>> >
>> >
>> >
>> > Thanks,
>> >
>> > Dominik
>> >
>> > ___
>> > Devel mailing list -- devel@ovirt.org
>> > To unsubscribe send an email to devel-le...@ovirt.org
>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> > oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> > List Archives:
>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/V42PCHW23M5XVSQMQ3IRJ4ZYTEPLMJQC/
>>
>> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GTVQZ7ERY4HKRA2WGZQ7E3HHX5UZB5ER/
>
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/KDSI6KRRGZNCBMGE23TME7FN7OJXPTWE/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ETRCDZ4PFM4JICWMWLSL3BDT3BYSKXAP/

[ovirt-devel] Re: master CQ fails due to missing ovirt-host-deploy

[Dafna Ron]

Then another possibility is that the package is now suppose to be coming
from somewhere else and its not available?
where is it suppose to come from? from what I understood, it should be
deployed by ansible, is that correct?
I can see an abandoned patch but cannot see a patch that was merged for
ansible:
https://gerrit.ovirt.org/#/q/ovirt-host-deploy

It seems patch: https://gerrit.ovirt.org/#/c/102959/ was abandoned for no
activity - was anything else actually merged on ansible side?

Thanks,
Dafna


On Wed, Feb 12, 2020 at 9:47 AM Yedidyah Bar David  wrote:

> On Wed, Feb 12, 2020 at 11:39 AM Dafna Ron  wrote:
> >
> > this is the current package in tested:
> ovirt-engine-0:4.2.0-0.0.master.20170712084142.git63b81ef.el7.centos.noarch
> and since all projects are now failing on this, it means the engine package
> in tested is a broken package.
> > what I don't understand is how can CQ mark engine as failed since this
> patch: https://gerrit.ovirt.org/#/c/104391/ which is 4 weeks ago but I
> can see a new package in tested since 06-02-2020 with no passing cq test
> for ovirt-engine.
> > If anyone is moving packages manually to tested they are breaking all
> projects by introducing broken packages.
> >
> > As the package in tested is now broken, we should start fixing the code
> on each failing issue.
> > for now, the main thing that is failing is that we are looking for a
> package that no longer exists.
> > Didi, is it possible its the typo Marcin was commenting on in this
> patch: https://gerrit.ovirt.org/#/c/106797/
>
> I already addressed this,
>
> >
> > Thanks,
> > Dafna
> >
> >
> >
> >
> >
> >
> >
> > On Tue, Feb 11, 2020 at 9:03 AM Yedidyah Bar David 
> wrote:
> >>
> >> On Tue, Feb 11, 2020 at 10:22 AM Dusan Fodor  wrote:
> >> >
> >> > Hello,
> >> > master CQ is failing on initialize_engine due to missing
> ovirt-host-deploy dependency.
> >> >
> >> > [ INFO  ] Checking for an update for Setup...
> >> > [ ERROR ] Yum Cannot queue package ovirt-host-deploy: Package
> ovirt-host-deploy cannot be found
> >> >
> >> >
> >> > I was under the impression that engine-setup should not require it
> anymore, but it seems the related patch [1] doesn't work as intended.
> Perhaps Marcin's comment there explains why? Can you please check this?
> >>
> >> How can I know which engine build was used for [2]?
> >>
> >> Checking lago.log there, I see:
> >>
> >> Installed:
> >>   net-snmp.x86_64 1:5.7.2-43.el7_7.3
> >>   ntp.x86_64 0:4.2.6p5-29.el7.centos
> >>   otopi-debug-plugins.noarch 0:1.9.0-1.el7
> >>   ovirt-engine.noarch 0:4.4.0-0.0.master.20200206140618.gitff1f0b4.el7
> >>   ovirt-engine-extension-aaa-ldap.noarch
> 0:1.3.11-0.0.master.gitb9a171f.el7
> >>   ovirt-engine-extension-aaa-ldap-setup.noarch
> >> 0:1.3.11-0.0.master.gitb9a171f.el7
> >>   ovirt-log-collector.noarch
> 0:4.4.0-0.0.master.20191122133512.gitfc29382.el7
> >>
> >> ff1f0b4 is a hash of a not-yet-merged patch:
> >>
> >> https://gerrit.ovirt.org/106794
> >>
> >> It should be rebased.
> >>
> >> Also not sure how the change-queue-tester uses a not-merged-yet engine.
> >>
> >> (And also, [1] is not enough, Marcin Sobczyk later pushed a fix for
> >> it, https://gerrit.ovirt.org/106809 ).
>
> here ^^. Already merged as well.
>
> >>
> >> > Job example [2]
> >> > Thanks
> >> >
> >> > [1] https://gerrit.ovirt.org/#/c/106797/
> >> > [2]
> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/19724/testReport/(root)/001_initialize_engine/initialize_engine/
> >>
> >>
> >>
> >> --
> >> Didi
> >> ___
> >> Devel mailing list -- devel@ovirt.org
> >> To unsubscribe send an email to devel-le...@ovirt.org
> >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> >> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> >> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/I5NOUPCLFXQDKDBHNIQC2XL56YZYLGOT/
>
>
>
> --
> Didi
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ZAXMQR773ODCQSDK7TK7QV62LRFGOYOV/

[ovirt-devel] Re: master CQ fails due to missing ovirt-host-deploy

[Dafna Ron]

this is the current package in tested:
ovirt-engine-0:4.2.0-0.0.master.20170712084142.git63b81ef.el7.centos.noarch
and since all projects are now failing on this, it means the engine package
in tested is a broken package.
what I don't understand is how can CQ mark engine as failed since this
patch: https://gerrit.ovirt.org/#/c/104391/ which is 4 weeks ago but I can
see a new package in tested since 06-02-2020 with no passing cq test for
ovirt-engine.
If anyone is moving packages manually to tested they are breaking all
projects by introducing broken packages.

As the package in tested is now broken, we should start fixing the code on
each failing issue.
for now, the main thing that is failing is that we are looking for a
package that no longer exists.
Didi, is it possible its the typo Marcin was commenting on in this patch:
https://gerrit.ovirt.org/#/c/106797/

Thanks,
Dafna







On Tue, Feb 11, 2020 at 9:03 AM Yedidyah Bar David  wrote:

> On Tue, Feb 11, 2020 at 10:22 AM Dusan Fodor  wrote:
> >
> > Hello,
> > master CQ is failing on initialize_engine due to missing
> ovirt-host-deploy dependency.
> >
> > [ INFO  ] Checking for an update for Setup...
> > [ ERROR ] Yum Cannot queue package ovirt-host-deploy: Package
> ovirt-host-deploy cannot be found
> >
> >
> > I was under the impression that engine-setup should not require it
> anymore, but it seems the related patch [1] doesn't work as intended.
> Perhaps Marcin's comment there explains why? Can you please check this?
>
> How can I know which engine build was used for [2]?
>
> Checking lago.log there, I see:
>
> Installed:
>   net-snmp.x86_64 1:5.7.2-43.el7_7.3
>   ntp.x86_64 0:4.2.6p5-29.el7.centos
>   otopi-debug-plugins.noarch 0:1.9.0-1.el7
>   ovirt-engine.noarch 0:4.4.0-0.0.master.20200206140618.gitff1f0b4.el7
>   ovirt-engine-extension-aaa-ldap.noarch 0:1.3.11-0.0.master.gitb9a171f.el7
>   ovirt-engine-extension-aaa-ldap-setup.noarch
> 0:1.3.11-0.0.master.gitb9a171f.el7
>   ovirt-log-collector.noarch
> 0:4.4.0-0.0.master.20191122133512.gitfc29382.el7
>
> ff1f0b4 is a hash of a not-yet-merged patch:
>
> https://gerrit.ovirt.org/106794
>
> It should be rebased.
>
> Also not sure how the change-queue-tester uses a not-merged-yet engine.
>
> (And also, [1] is not enough, Marcin Sobczyk later pushed a fix for
> it, https://gerrit.ovirt.org/106809 ).
>
> > Job example [2]
> > Thanks
> >
> > [1] https://gerrit.ovirt.org/#/c/106797/
> > [2]
> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/19724/testReport/(root)/001_initialize_engine/initialize_engine/
>
>
>
> --
> Didi
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/I5NOUPCLFXQDKDBHNIQC2XL56YZYLGOT/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/XZWUOLX4SCRKX53E3OXVBNAUAOUBOULA/

[ovirt-devel] Re: vdsm has been tagged (v4.30.37)

[Dafna Ron]

Thanks Milan :)


On Tue, Nov 12, 2019 at 2:32 PM Milan Zamazal  wrote:

> 4.3.7 should be 4.*.37, shouldn't it?
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NCQJ6HD2D4ZK6H6FL24DCH6DTP7AM3RS/

[ovirt-devel] Re: Please remove 4.2 from your stdci.yaml files

[Dafna Ron]

The failure reported here was resolved along with another package that was
effected but as the configuration is in the projects themselves and we
cannot really go over each bone to check I am afraid we cannot give a list
of packages at this time (until it fails).

On Wed, Oct 9, 2019 at 3:36 PM Sandro Bonazzola  wrote:

>
>
> Il giorno ven 20 set 2019 alle ore 12:14 Dafna Ron  ha
> scritto:
>
>> Hi,
>>
>> We have several unstable jobs which are caused because 4.2 is no longer
>> available for CQ.
>>
>> Please remove 4.2 from your stdci.yaml files to avoid getting unstable
>> jobs.
>>
>
> have we got a list of packages affected?
>
>
>>
>> Thanks,
>> Dafna
>>
>> ___
>> Infra mailing list -- in...@ovirt.org
>> To unsubscribe send an email to infra-le...@ovirt.org
>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>> https://lists.ovirt.org/archives/list/in...@ovirt.org/message/4SKBLVF7AFCAKWM6ZBB6BLDY6B3TDSTP/
>>
>
>
> --
>
> Sandro Bonazzola
>
> MANAGER, SOFTWARE ENGINEERING, EMEA R RHV
>
> Red Hat EMEA <https://www.redhat.com/>
>
> sbona...@redhat.com
> <https://www.redhat.com/>*Red Hat respects your work life balance.
> Therefore there is no need to answer this email out of your office hours.
> <https://mojo.redhat.com/docs/DOC-1199578>*
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PZTQF2ICD6TXXDC4MR3BX24VAG6ROKB3/

[ovirt-devel] Please remove 4.2 from your stdci.yaml files

[Dafna Ron]

Hi,

We have several unstable jobs which are caused because 4.2 is no longer
available for CQ.

Please remove 4.2 from your stdci.yaml files to avoid getting unstable
jobs.

Thanks,
Dafna
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/4SKBLVF7AFCAKWM6ZBB6BLDY6B3TDSTP/

[ovirt-devel] Re: URGENT - ovirt-engine master is failing for the past 9 days

[Dafna Ron]

We have a passing ovirt-engine build
Thanks everyone!
Dafna


On Mon, Sep 2, 2019 at 1:00 PM Tal Nisan  wrote:

> Done
>
> On Mon, Sep 2, 2019 at 2:46 PM Dafna Ron  wrote:
>
>> Thanks Andrej,
>> Tal, can you merge?
>>
>> Thanks,
>> Dafna
>>
>>
>> On Mon, Sep 2, 2019 at 2:42 PM Andrej Krejcir 
>> wrote:
>>
>>> This patch fixes the issue: https://gerrit.ovirt.org/#/c/103019/
>>>
>>> On Mon, 2 Sep 2019 at 10:16, Andrej Krejcir  wrote:
>>>
>>>> I will post a fix later today. Sorry for the delay, It took me a while
>>>> to figure out what is the problem.
>>>>
>>>> On Sun, 1 Sep 2019 at 14:58, Tal Nisan  wrote:
>>>>
>>>>> OST failing for 9 days is totally unacceptable, the only reason I'm
>>>>> not sending a revert patch is that it'll make a bloody mess with the
>>>>> upgrade scripts as we have two on top of it, this needs to be fixed ASAP
>>>>>
>>>>> On Sun, Sep 1, 2019 at 3:27 PM Dafna Ron  wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have been failing CQ master for project ovirt-engine for the past
>>>>>> 9 days.
>>>>>> this was reported last week and we have not yet seen a fix for this
>>>>>> issue.
>>>>>>
>>>>>> the patch reported by CQ is
>>>>>> https://gerrit.ovirt.org/#/c/101913/10 - core: Change CPU config to
>>>>>> secure/insecure concept
>>>>>>
>>>>>> logs for latest failure can be found here:
>>>>>>
>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15638/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/
>>>>>>
>>>>>> Error:
>>>>>> 2019-09-01 06:31:17,248-04 INFO
>>>>>>  [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-1)
>>>>>> [174bc463-9a98-441f-ad14-4d97fefde4fa] Running command:
>>>>>> UpdateClusterCommand internal: false. Entities affected :  ID:
>>>>>> 0407bc06-4bef-4c47-99e1-0e42f9ced996 Type: ClusterAction group
>>>>>> EDIT_CLUSTER_CONFIGURATION with role type ADMIN
>>>>>> 2019-09-01 06:31:17,263-04 INFO
>>>>>>  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>> (default task-1) [174bc463-9a98-441f-ad14-4d97fefde4fa] EVENT_ID:
>>>>>> CLUSTER_UPDATE_CPU_WHEN_DEPRECATED(9,029), Modified the CPU Type to Intel
>>>>>> Haswell-noTSX Family when upgrading the Compatibility Version of Cluster
>>>>>> test-cluster because the previous CPU Type was deprecated.
>>>>>> 2019-09-01 06:31:17,319-04 WARN
>>>>>>  [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [18fd7817]
>>>>>> Validation of action 'UpdateVm' failed for user admin@internal-authz.
>>>>>> Reasons:
>>>>>> VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE
>>>>>> 2019-09-01 06:31:17,329-04 WARN
>>>>>>  [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [62694a6a]
>>>>>> Validation of action 'UpdateVm' failed for user admin@internal-authz.
>>>>>> Reasons:
>>>>>> VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE
>>>>>> 2019-09-01 06:31:17,334-04 WARN
>>>>>>  [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [10e41cd3]
>>>>>> Validation of action 'UpdateVm' failed for user admin@internal-authz.
>>>>>> Reasons:
>>>>>> VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE
>>>>>> 2019-09-01 06:31:17,345-04 WARN
>>>>>>  [org.ovirt.engine.core.bll.UpdateVmTemplateCommand] (default task-1)
>>>>>> [26a471dd] Validation of action 'UpdateVmTemplate' failed for user
>>>>>> admin@internal-authz. Reasons:
>>>>>> VAR__ACTION__UPDATE,VAR__TYPE__VM_TEMPLATE,VMT_CANNOT_UPDATE_ILLEGAL_FIELD
>>>>>> 2019-09-01 06:31:17,352-04 WARN
>>>>>>  [org.ovirt.engine.core.bll.UpdateVmTemplateCommand] (default task-1)
>>>>>> [32fd89b5] Validation of action 'Upd

[ovirt-devel] URGENT - ovirt-engine master is failing for the past 9 days

[Dafna Ron]

Hi,

We have been failing CQ master for project ovirt-engine for the past 9
days.
this was reported last week and we have not yet seen a fix for this issue.

the patch reported by CQ is
https://gerrit.ovirt.org/#/c/101913/10 - core: Change CPU config to
secure/insecure concept

logs for latest failure can be found here:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15638/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/

Error:
2019-09-01 06:31:17,248-04 INFO
 [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-1)
[174bc463-9a98-441f-ad14-4d97fefde4fa] Running command:
UpdateClusterCommand internal: false. Entities affected :  ID:
0407bc06-4bef-4c47-99e1-0e42f9ced996 Type: ClusterAction group
EDIT_CLUSTER_CONFIGURATION with role type ADMIN
2019-09-01 06:31:17,263-04 INFO
 [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [174bc463-9a98-441f-ad14-4d97fefde4fa] EVENT_ID:
CLUSTER_UPDATE_CPU_WHEN_DEPRECATED(9,029), Modified the CPU Type to Intel
Haswell-noTSX Family when upgrading the Compatibility Version of Cluster
test-cluster because the previous CPU Type was deprecated.
2019-09-01 06:31:17,319-04 WARN
 [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [18fd7817]
Validation of action 'UpdateVm' failed for user admin@internal-authz.
Reasons:
VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE
2019-09-01 06:31:17,329-04 WARN
 [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [62694a6a]
Validation of action 'UpdateVm' failed for user admin@internal-authz.
Reasons:
VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE
2019-09-01 06:31:17,334-04 WARN
 [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [10e41cd3]
Validation of action 'UpdateVm' failed for user admin@internal-authz.
Reasons:
VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE
2019-09-01 06:31:17,345-04 WARN
 [org.ovirt.engine.core.bll.UpdateVmTemplateCommand] (default task-1)
[26a471dd] Validation of action 'UpdateVmTemplate' failed for user
admin@internal-authz. Reasons:
VAR__ACTION__UPDATE,VAR__TYPE__VM_TEMPLATE,VMT_CANNOT_UPDATE_ILLEGAL_FIELD
2019-09-01 06:31:17,352-04 WARN
 [org.ovirt.engine.core.bll.UpdateVmTemplateCommand] (default task-1)
[32fd89b5] Validation of action 'UpdateVmTemplate' failed for user
admin@internal-authz. Reasons:
VAR__ACTION__UPDATE,VAR__TYPE__VM_TEMPLATE,VMT_CANNOT_UPDATE_ILLEGAL_FIELD
2019-09-01 06:31:17,363-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [32fd89b5] EVENT_ID:
CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update
compatibility version of Vm/Template: [vm-with-iface], Message: [No Message]
2019-09-01 06:31:17,371-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [32fd89b5] EVENT_ID:
CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update
compatibility version of Vm/Template: [vm-with-iface-template], Message:
[No Message]
2019-09-01 06:31:17,380-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [32fd89b5] EVENT_ID:
CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update
compatibility version of Vm/Template: [vm0], Message: [No Message]
2019-09-01 06:31:17,388-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [32fd89b5] EVENT_ID:
CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update
compatibility version of Vm/Template: [vm1], Message: [No Message]
2019-09-01 06:31:17,405-04 INFO
 [org.ovirt.engine.core.bll.CommandCompensator] (default task-1) [32fd89b5]
Command [id=50898f00-4956-484a-94ff-f273adc2e93e]: Compensating
UPDATED_ONLY_ENTITY of
org.ovirt.engine.core.common.businessentities.Cluster; snapshot: Cluster
[test-cluster].
2019-09-01 06:31:17,422-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [32fd89b5] EVENT_ID: USER_UPDATE_CLUSTER_FAILED(812),
Failed to update Host cluster (User: admin@internal-authz)
2019-09-01 06:31:17,422-04 INFO
 [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-1)
[32fd89b5] Lock freed to object
'EngineLock:{exclusiveLocks='[5d3b2f0f-f05f-41c6-b61a-c517704f79fd=TEMPLATE,
728f5530-4e34-4c92-beef-ca494ec104b9=TEMPLATE]',
sharedLocks='[a107316a-a961-403f-adb2-d01f22f0b8f1=VM,
dc3a2ad8-6019-4e00-85e5-d9fba7390d4f=VM,
6348722f-61ae-40be-a2b8-bd9d086b06dc=VM]'}'
2019-09-01 06:31:17,424-04 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
task-1) [] Operation Failed: [Update of cluster compatibility version
failed because there are VMs/Templates [vm-with-iface,
vm-with-iface-template, vm0, vm1] with incorrect configuration. 

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

Thanks Andrej!

On Wed, Aug 28, 2019 at 10:40 AM Andrej Krejcir  wrote:

> This patch is not the cause, it only removes unused code. But there was
> another patch set merged recently, that is probably the cause. I'm looking
> into it.
>
>
> Andrej
>
> On Tue, 27 Aug 2019 at 16:25, Dafna Ron  wrote:
>
>> Adding Andrej Krejcir and Lucia =
>> Patch https://gerrit.ovirt.org/#/c/102839/ - engine:
>> UpdateClusterCommand cleanup is causing the regression in engine
>>
>> Lucia/Andrej, can you please issue a fix?
>>
>> Thanks for the quick report Dominik!
>> Dafna
>>
>>
>>
>> On Tue, Aug 27, 2019 at 5:20 PM Dafna Ron  wrote:
>>
>>> I see there is an ovirt-engine package in the master queue:
>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue/45420/
>>>
>>> we should expect it to fail
>>>
>>>
>>> On Tue, Aug 27, 2019 at 5:15 PM Dafna Ron  wrote:
>>>
>>>> its failing so I guess we do have some issue in the pre-merged packages
>>>> on cluster version
>>>>
>>>> On Tue, Aug 27, 2019 at 5:14 PM Dafna Ron  wrote:
>>>>
>>>>> odd I thought it passed...
>>>>>
>>>>>
>>>>> On Tue, Aug 27, 2019 at 5:11 PM Dominik Holler 
>>>>> wrote:
>>>>>
>>>>>> On Tue, 27 Aug 2019 16:36:45 +0300
>>>>>> Dafna Ron  wrote:
>>>>>>
>>>>>> > passed with latest package:
>>>>>> >
>>>>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5454/
>>>>>> >
>>>>>>
>>>>>> I don't get it. Isn't 5454 failing?
>>>>>>
>>>>>>
>>>>>> >
>>>>>> > On Tue, Aug 27, 2019 at 3:57 PM Dafna Ron  wrote:
>>>>>> >
>>>>>> > > Dominik,
>>>>>> > > seems it passes with no changes:
>>>>>> > >
>>>>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5452/
>>>>>> > > the issue if exists is in engine before merge (unless we will see
>>>>>> a failed
>>>>>> > > CQ soon)
>>>>>> > > I ran another manual test on the latest built package of engine
>>>>>> so lets
>>>>>> > > see if we have an issue that has not been identified yet.
>>>>>> > >
>>>>>> > > adding Dusan in case CQ starts failing
>>>>>> > >
>>>>>> > >
>>>>>> > > On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron 
>>>>>> wrote:
>>>>>> > >
>>>>>> > >> Running it without a new package to see if this is reproduces on
>>>>>> packages
>>>>>> > >> from tested with no changes.
>>>>>> > >>
>>>>>> > >>
>>>>>> > >> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler <
>>>>>> dhol...@redhat.com>
>>>>>> > >> wrote:
>>>>>> > >>
>>>>>> > >>> Hi,
>>>>>> > >>> for me OST upgrade-from-release-suite-master
>>>>>> update_cluster_versions
>>>>>> > >>> is failing with
>>>>>> > >>>
>>>>>> > >>> 004_basic_sanity.update_cluster_versions (from nosetests)
>>>>>> > >>> Failing for the past 1 build (Since Failed#5442 )
>>>>>> > >>> Took 1 min 16 sec.
>>>>>> > >>> add description
>>>>>> > >>> Error Message
>>>>>> > >>>
>>>>>> > >>> Fault reason is "Operation Failed". Fault detail is "[Update of
>>>>>> cluster
>>>>>> > >>> compatibility version failed because there are VMs/Templates
>>>>>> > >>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>>>>>> > >>> configuration. To fix the issue, please go to each of them,
>>>>>> edit, change
>>>>>> > >>> the Custom Compatibility Version of the VM/Template to the
>>>>>> cluster level
>>&

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

Adding Andrej Krejcir and Lucia =
Patch https://gerrit.ovirt.org/#/c/102839/ - engine: UpdateClusterCommand
cleanup is causing the regression in engine

Lucia/Andrej, can you please issue a fix?

Thanks for the quick report Dominik!
Dafna



On Tue, Aug 27, 2019 at 5:20 PM Dafna Ron  wrote:

> I see there is an ovirt-engine package in the master queue:
> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue/45420/
>
> we should expect it to fail
>
>
> On Tue, Aug 27, 2019 at 5:15 PM Dafna Ron  wrote:
>
>> its failing so I guess we do have some issue in the pre-merged packages
>> on cluster version
>>
>> On Tue, Aug 27, 2019 at 5:14 PM Dafna Ron  wrote:
>>
>>> odd I thought it passed...
>>>
>>>
>>> On Tue, Aug 27, 2019 at 5:11 PM Dominik Holler 
>>> wrote:
>>>
>>>> On Tue, 27 Aug 2019 16:36:45 +0300
>>>> Dafna Ron  wrote:
>>>>
>>>> > passed with latest package:
>>>> >
>>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5454/
>>>> >
>>>>
>>>> I don't get it. Isn't 5454 failing?
>>>>
>>>>
>>>> >
>>>> > On Tue, Aug 27, 2019 at 3:57 PM Dafna Ron  wrote:
>>>> >
>>>> > > Dominik,
>>>> > > seems it passes with no changes:
>>>> > >
>>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5452/
>>>> > > the issue if exists is in engine before merge (unless we will see a
>>>> failed
>>>> > > CQ soon)
>>>> > > I ran another manual test on the latest built package of engine so
>>>> lets
>>>> > > see if we have an issue that has not been identified yet.
>>>> > >
>>>> > > adding Dusan in case CQ starts failing
>>>> > >
>>>> > >
>>>> > > On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron  wrote:
>>>> > >
>>>> > >> Running it without a new package to see if this is reproduces on
>>>> packages
>>>> > >> from tested with no changes.
>>>> > >>
>>>> > >>
>>>> > >> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler >>> >
>>>> > >> wrote:
>>>> > >>
>>>> > >>> Hi,
>>>> > >>> for me OST upgrade-from-release-suite-master
>>>> update_cluster_versions
>>>> > >>> is failing with
>>>> > >>>
>>>> > >>> 004_basic_sanity.update_cluster_versions (from nosetests)
>>>> > >>> Failing for the past 1 build (Since Failed#5442 )
>>>> > >>> Took 1 min 16 sec.
>>>> > >>> add description
>>>> > >>> Error Message
>>>> > >>>
>>>> > >>> Fault reason is "Operation Failed". Fault detail is "[Update of
>>>> cluster
>>>> > >>> compatibility version failed because there are VMs/Templates
>>>> > >>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>>>> > >>> configuration. To fix the issue, please go to each of them, edit,
>>>> change
>>>> > >>> the Custom Compatibility Version of the VM/Template to the
>>>> cluster level
>>>> > >>> you want to update the cluster to and press OK. If the save does
>>>> not pass,
>>>> > >>> fix the dialog validation. After successful cluster update, you
>>>> can revert
>>>> > >>> your Custom Compatibility Version change.]". HTTP response code
>>>> is 500.
>>>> > >>>
>>>> > >>> Stacktrace
>>>> > >>>
>>>> > >>> Traceback (most recent call last):
>>>> > >>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>>>> > >>> testMethod()
>>>> > >>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197,
>>>> in
>>>> > >>> runTest
>>>> > >>> self.test(*self.arg)
>>>> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>>> line
>>>> 

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

I see there is an ovirt-engine package in the master queue:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue/45420/

we should expect it to fail


On Tue, Aug 27, 2019 at 5:15 PM Dafna Ron  wrote:

> its failing so I guess we do have some issue in the pre-merged packages on
> cluster version
>
> On Tue, Aug 27, 2019 at 5:14 PM Dafna Ron  wrote:
>
>> odd I thought it passed...
>>
>>
>> On Tue, Aug 27, 2019 at 5:11 PM Dominik Holler 
>> wrote:
>>
>>> On Tue, 27 Aug 2019 16:36:45 +0300
>>> Dafna Ron  wrote:
>>>
>>> > passed with latest package:
>>> >
>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5454/
>>> >
>>>
>>> I don't get it. Isn't 5454 failing?
>>>
>>>
>>> >
>>> > On Tue, Aug 27, 2019 at 3:57 PM Dafna Ron  wrote:
>>> >
>>> > > Dominik,
>>> > > seems it passes with no changes:
>>> > >
>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5452/
>>> > > the issue if exists is in engine before merge (unless we will see a
>>> failed
>>> > > CQ soon)
>>> > > I ran another manual test on the latest built package of engine so
>>> lets
>>> > > see if we have an issue that has not been identified yet.
>>> > >
>>> > > adding Dusan in case CQ starts failing
>>> > >
>>> > >
>>> > > On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron  wrote:
>>> > >
>>> > >> Running it without a new package to see if this is reproduces on
>>> packages
>>> > >> from tested with no changes.
>>> > >>
>>> > >>
>>> > >> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler 
>>> > >> wrote:
>>> > >>
>>> > >>> Hi,
>>> > >>> for me OST upgrade-from-release-suite-master
>>> update_cluster_versions
>>> > >>> is failing with
>>> > >>>
>>> > >>> 004_basic_sanity.update_cluster_versions (from nosetests)
>>> > >>> Failing for the past 1 build (Since Failed#5442 )
>>> > >>> Took 1 min 16 sec.
>>> > >>> add description
>>> > >>> Error Message
>>> > >>>
>>> > >>> Fault reason is "Operation Failed". Fault detail is "[Update of
>>> cluster
>>> > >>> compatibility version failed because there are VMs/Templates
>>> > >>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>>> > >>> configuration. To fix the issue, please go to each of them, edit,
>>> change
>>> > >>> the Custom Compatibility Version of the VM/Template to the cluster
>>> level
>>> > >>> you want to update the cluster to and press OK. If the save does
>>> not pass,
>>> > >>> fix the dialog validation. After successful cluster update, you
>>> can revert
>>> > >>> your Custom Compatibility Version change.]". HTTP response code is
>>> 500.
>>> > >>>
>>> > >>> Stacktrace
>>> > >>>
>>> > >>> Traceback (most recent call last):
>>> > >>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>>> > >>> testMethod()
>>> > >>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197,
>>> in
>>> > >>> runTest
>>> > >>> self.test(*self.arg)
>>> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>> line
>>> > >>> 142, in wrapped_test
>>> > >>> test()
>>> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>> line 60,
>>> > >>> in wrapper
>>> > >>> return func(get_test_prefix(), *args, **kwargs)
>>> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>> line 79,
>>> > >>> in wrapper
>>> > >>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>>> > >>>   File
>>> > >>>
>>> &q

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

its failing so I guess we do have some issue in the pre-merged packages on
cluster version

On Tue, Aug 27, 2019 at 5:14 PM Dafna Ron  wrote:

> odd I thought it passed...
>
>
> On Tue, Aug 27, 2019 at 5:11 PM Dominik Holler  wrote:
>
>> On Tue, 27 Aug 2019 16:36:45 +0300
>> Dafna Ron  wrote:
>>
>> > passed with latest package:
>> >
>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5454/
>> >
>>
>> I don't get it. Isn't 5454 failing?
>>
>>
>> >
>> > On Tue, Aug 27, 2019 at 3:57 PM Dafna Ron  wrote:
>> >
>> > > Dominik,
>> > > seems it passes with no changes:
>> > >
>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5452/
>> > > the issue if exists is in engine before merge (unless we will see a
>> failed
>> > > CQ soon)
>> > > I ran another manual test on the latest built package of engine so
>> lets
>> > > see if we have an issue that has not been identified yet.
>> > >
>> > > adding Dusan in case CQ starts failing
>> > >
>> > >
>> > > On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron  wrote:
>> > >
>> > >> Running it without a new package to see if this is reproduces on
>> packages
>> > >> from tested with no changes.
>> > >>
>> > >>
>> > >> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler 
>> > >> wrote:
>> > >>
>> > >>> Hi,
>> > >>> for me OST upgrade-from-release-suite-master update_cluster_versions
>> > >>> is failing with
>> > >>>
>> > >>> 004_basic_sanity.update_cluster_versions (from nosetests)
>> > >>> Failing for the past 1 build (Since Failed#5442 )
>> > >>> Took 1 min 16 sec.
>> > >>> add description
>> > >>> Error Message
>> > >>>
>> > >>> Fault reason is "Operation Failed". Fault detail is "[Update of
>> cluster
>> > >>> compatibility version failed because there are VMs/Templates
>> > >>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>> > >>> configuration. To fix the issue, please go to each of them, edit,
>> change
>> > >>> the Custom Compatibility Version of the VM/Template to the cluster
>> level
>> > >>> you want to update the cluster to and press OK. If the save does
>> not pass,
>> > >>> fix the dialog validation. After successful cluster update, you can
>> revert
>> > >>> your Custom Compatibility Version change.]". HTTP response code is
>> 500.
>> > >>>
>> > >>> Stacktrace
>> > >>>
>> > >>> Traceback (most recent call last):
>> > >>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>> > >>> testMethod()
>> > >>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in
>> > >>> runTest
>> > >>> self.test(*self.arg)
>> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
>> > >>> 142, in wrapped_test
>> > >>> test()
>> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>> line 60,
>> > >>> in wrapper
>> > >>> return func(get_test_prefix(), *args, **kwargs)
>> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>> line 79,
>> > >>> in wrapper
>> > >>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>> > >>>   File
>> > >>>
>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
>> > >>> line 184, in update_cluster_versions
>> > >>> minor=minor
>> > >>>   File
>> > >>>
>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
>> > >>> line 130, in _update_cluster_version
>> > >>> version=new_version
>> > >>>   File "/usr/l

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

odd I thought it passed...


On Tue, Aug 27, 2019 at 5:11 PM Dominik Holler  wrote:

> On Tue, 27 Aug 2019 16:36:45 +0300
> Dafna Ron  wrote:
>
> > passed with latest package:
> >
> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5454/
> >
>
> I don't get it. Isn't 5454 failing?
>
>
> >
> > On Tue, Aug 27, 2019 at 3:57 PM Dafna Ron  wrote:
> >
> > > Dominik,
> > > seems it passes with no changes:
> > >
> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5452/
> > > the issue if exists is in engine before merge (unless we will see a
> failed
> > > CQ soon)
> > > I ran another manual test on the latest built package of engine so lets
> > > see if we have an issue that has not been identified yet.
> > >
> > > adding Dusan in case CQ starts failing
> > >
> > >
> > > On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron  wrote:
> > >
> > >> Running it without a new package to see if this is reproduces on
> packages
> > >> from tested with no changes.
> > >>
> > >>
> > >> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler 
> > >> wrote:
> > >>
> > >>> Hi,
> > >>> for me OST upgrade-from-release-suite-master update_cluster_versions
> > >>> is failing with
> > >>>
> > >>> 004_basic_sanity.update_cluster_versions (from nosetests)
> > >>> Failing for the past 1 build (Since Failed#5442 )
> > >>> Took 1 min 16 sec.
> > >>> add description
> > >>> Error Message
> > >>>
> > >>> Fault reason is "Operation Failed". Fault detail is "[Update of
> cluster
> > >>> compatibility version failed because there are VMs/Templates
> > >>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
> > >>> configuration. To fix the issue, please go to each of them, edit,
> change
> > >>> the Custom Compatibility Version of the VM/Template to the cluster
> level
> > >>> you want to update the cluster to and press OK. If the save does not
> pass,
> > >>> fix the dialog validation. After successful cluster update, you can
> revert
> > >>> your Custom Compatibility Version change.]". HTTP response code is
> 500.
> > >>>
> > >>> Stacktrace
> > >>>
> > >>> Traceback (most recent call last):
> > >>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
> > >>> testMethod()
> > >>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in
> > >>> runTest
> > >>> self.test(*self.arg)
> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
> > >>> 142, in wrapped_test
> > >>> test()
> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
> 60,
> > >>> in wrapper
> > >>> return func(get_test_prefix(), *args, **kwargs)
> > >>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
> 79,
> > >>> in wrapper
> > >>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
> > >>>   File
> > >>>
> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
> > >>> line 184, in update_cluster_versions
> > >>> minor=minor
> > >>>   File
> > >>>
> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
> > >>> line 130, in _update_cluster_version
> > >>> version=new_version
> > >>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py",
> line
> > >>> 3943, in update
> > >>> return self._internal_update(cluster, headers, query, wait)
> > >>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",
> line
> > >>> 253, in _internal_update
> > >>> return future.wait() if wait else future
> > >>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py",
> line
> > 

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

passed with latest package:
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5454/


On Tue, Aug 27, 2019 at 3:57 PM Dafna Ron  wrote:

> Dominik,
> seems it passes with no changes:
> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5452/
> the issue if exists is in engine before merge (unless we will see a failed
> CQ soon)
> I ran another manual test on the latest built package of engine so lets
> see if we have an issue that has not been identified yet.
>
> adding Dusan in case CQ starts failing
>
>
> On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron  wrote:
>
>> Running it without a new package to see if this is reproduces on packages
>> from tested with no changes.
>>
>>
>> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler 
>> wrote:
>>
>>> Hi,
>>> for me OST upgrade-from-release-suite-master update_cluster_versions
>>> is failing with
>>>
>>> 004_basic_sanity.update_cluster_versions (from nosetests)
>>> Failing for the past 1 build (Since Failed#5442 )
>>> Took 1 min 16 sec.
>>> add description
>>> Error Message
>>>
>>> Fault reason is "Operation Failed". Fault detail is "[Update of cluster
>>> compatibility version failed because there are VMs/Templates
>>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>>> configuration. To fix the issue, please go to each of them, edit, change
>>> the Custom Compatibility Version of the VM/Template to the cluster level
>>> you want to update the cluster to and press OK. If the save does not pass,
>>> fix the dialog validation. After successful cluster update, you can revert
>>> your Custom Compatibility Version change.]". HTTP response code is 500.
>>>
>>> Stacktrace
>>>
>>> Traceback (most recent call last):
>>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>>> testMethod()
>>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in
>>> runTest
>>> self.test(*self.arg)
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
>>> 142, in wrapped_test
>>> test()
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60,
>>> in wrapper
>>> return func(get_test_prefix(), *args, **kwargs)
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79,
>>> in wrapper
>>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>>>   File
>>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
>>> line 184, in update_cluster_versions
>>> minor=minor
>>>   File
>>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
>>> line 130, in _update_cluster_version
>>> version=new_version
>>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
>>> 3943, in update
>>> return self._internal_update(cluster, headers, query, wait)
>>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>> 253, in _internal_update
>>> return future.wait() if wait else future
>>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>> 55, in wait
>>> return self._code(response)
>>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>> 250, in callback
>>> self._check_fault(response)
>>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>> 132, in _check_fault
>>> self._raise_error(response, body)
>>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>> 118, in _raise_error
>>> raise error
>>> Error: Fault reason is "Operation Failed". Fault detail is "[Update of
>>> cluster compatibility version failed because there are VMs/Templates
>>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>>> configuration. To fix the issue, please go to each of them, edit, change
>>> the Custom Compatibility Version of the VM/Template to the cluster level
>>> you want to update the cluster to and press OK. If the save does not pass,
>>> fix the dialog validation. After successful cluster update, you can revert
>>> your Custom Compatibility Version change.]". HTTP response code is 500.
>>>
>>> Please find an example run in
>>> https://jenkins.ovirt.org/view/oVirt system
>>> tests/job/ovirt-system-tests_manual/5442
>>>
>>> Is this a known error, or is someone already addressing this issue?
>>> Thanks
>>> Dominik
>>>
>>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/Y5F4K7PWJV75IMYGD35SG7AULXTKZBYK/

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

Dominik,
seems it passes with no changes:
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/5452/
the issue if exists is in engine before merge (unless we will see a failed
CQ soon)
I ran another manual test on the latest built package of engine so lets see
if we have an issue that has not been identified yet.

adding Dusan in case CQ starts failing


On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron  wrote:

> Running it without a new package to see if this is reproduces on packages
> from tested with no changes.
>
>
> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler  wrote:
>
>> Hi,
>> for me OST upgrade-from-release-suite-master update_cluster_versions
>> is failing with
>>
>> 004_basic_sanity.update_cluster_versions (from nosetests)
>> Failing for the past 1 build (Since Failed#5442 )
>> Took 1 min 16 sec.
>> add description
>> Error Message
>>
>> Fault reason is "Operation Failed". Fault detail is "[Update of cluster
>> compatibility version failed because there are VMs/Templates
>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>> configuration. To fix the issue, please go to each of them, edit, change
>> the Custom Compatibility Version of the VM/Template to the cluster level
>> you want to update the cluster to and press OK. If the save does not pass,
>> fix the dialog validation. After successful cluster update, you can revert
>> your Custom Compatibility Version change.]". HTTP response code is 500.
>>
>> Stacktrace
>>
>> Traceback (most recent call last):
>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>> testMethod()
>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in
>> runTest
>> self.test(*self.arg)
>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142,
>> in wrapped_test
>> test()
>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60,
>> in wrapper
>> return func(get_test_prefix(), *args, **kwargs)
>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79,
>> in wrapper
>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>>   File
>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
>> line 184, in update_cluster_versions
>> minor=minor
>>   File
>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
>> line 130, in _update_cluster_version
>> version=new_version
>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
>> 3943, in update
>> return self._internal_update(cluster, headers, query, wait)
>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 253, in _internal_update
>> return future.wait() if wait else future
>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 55, in wait
>> return self._code(response)
>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 250, in callback
>> self._check_fault(response)
>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 132, in _check_fault
>> self._raise_error(response, body)
>>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>> 118, in _raise_error
>> raise error
>> Error: Fault reason is "Operation Failed". Fault detail is "[Update of
>> cluster compatibility version failed because there are VMs/Templates
>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
>> configuration. To fix the issue, please go to each of them, edit, change
>> the Custom Compatibility Version of the VM/Template to the cluster level
>> you want to update the cluster to and press OK. If the save does not pass,
>> fix the dialog validation. After successful cluster update, you can revert
>> your Custom Compatibility Version change.]". HTTP response code is 500.
>>
>> Please find an example run in
>> https://jenkins.ovirt.org/view/oVirt system
>> tests/job/ovirt-system-tests_manual/5442
>>
>> Is this a known error, or is someone already addressing this issue?
>> Thanks
>> Dominik
>>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/UQ64ND2WQRLKZXWFWAX3SDMTW5XVT5JK/

[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

[Dafna Ron]

Running it without a new package to see if this is reproduces on packages
from tested with no changes.


On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler  wrote:

> Hi,
> for me OST upgrade-from-release-suite-master update_cluster_versions
> is failing with
>
> 004_basic_sanity.update_cluster_versions (from nosetests)
> Failing for the past 1 build (Since Failed#5442 )
> Took 1 min 16 sec.
> add description
> Error Message
>
> Fault reason is "Operation Failed". Fault detail is "[Update of cluster
> compatibility version failed because there are VMs/Templates
> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
> configuration. To fix the issue, please go to each of them, edit, change
> the Custom Compatibility Version of the VM/Template to the cluster level
> you want to update the cluster to and press OK. If the save does not pass,
> fix the dialog validation. After successful cluster update, you can revert
> your Custom Compatibility Version change.]". HTTP response code is 500.
>
> Stacktrace
>
> Traceback (most recent call last):
>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
> testMethod()
>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in
> runTest
> self.test(*self.arg)
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142,
> in wrapped_test
> test()
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60,
> in wrapper
> return func(get_test_prefix(), *args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79,
> in wrapper
> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>   File
> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
> line 184, in update_cluster_versions
> minor=minor
>   File
> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py",
> line 130, in _update_cluster_version
> version=new_version
>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
> 3943, in update
> return self._internal_update(cluster, headers, query, wait)
>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 253, in _internal_update
> return future.wait() if wait else future
>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55,
> in wait
> return self._code(response)
>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 250, in callback
> self._check_fault(response)
>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 132, in _check_fault
> self._raise_error(response, body)
>   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> 118, in _raise_error
> raise error
> Error: Fault reason is "Operation Failed". Fault detail is "[Update of
> cluster compatibility version failed because there are VMs/Templates
> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect
> configuration. To fix the issue, please go to each of them, edit, change
> the Custom Compatibility Version of the VM/Template to the cluster level
> you want to update the cluster to and press OK. If the save does not pass,
> fix the dialog validation. After successful cluster update, you can revert
> your Custom Compatibility Version change.]". HTTP response code is 500.
>
> Please find an example run in
> https://jenkins.ovirt.org/view/oVirt system
> tests/job/ovirt-system-tests_manual/5442
>
> Is this a known error, or is someone already addressing this issue?
> Thanks
> Dominik
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/F2OVG7LT2ANJNLAZSZHUH4UF7JQTPCUC/

[ovirt-devel] Re: [Ovirt] [CQ weekly status] [02-08-2019]

[Dafna Ron]

 I think I saw some projects still merging to 4.2

On Mon, Aug 5, 2019 at 5:44 PM Eyal Edri  wrote:

>
>
> On Mon, Aug 5, 2019 at 5:23 PM Dusan Fodor  wrote:
>
>>
>>
>> On Mon, Aug 5, 2019 at 11:22 AM Sandro Bonazzola 
>> wrote:
>>
>>>
>>>
>>> Il giorno ven 2 ago 2019 alle ore 22:50 Dusan Fodor 
>>> ha scritto:
>>>
 Hi,

 This mail is to provide the current status of CQ and allow people to
 review status before and after the weekend.
 Please refer to below colour map for further information on the meaning
 of the colours.

 *CQ-4.2*:  RED (#1)

 Last failure was on 01-08 for ovirt-ansible-hosted-engine-setup caused
 by missing dependency, patch is pending to fix this.

>>>
>>> I think we can close 4.2 CQ, 4.2 gone EOL a few months ago
>>>
>>
> Can we drop it in upstream?
> In downstream its running 4.2 EUS with 4.3 engine.
>
>
>>
>>>
>>>

 *CQ-4.3*:   RED (#1)

 Last failure was on 02-08 for vdsm caused by missing dependency, patch
 is pending to fix this.

>>>
>>> Have we got a bug for this? If not please open one
>>>
>> It was already resolved by https://gerrit.ovirt.org/#/c/102317/
>>
>>>
>>>

 *CQ-Master:*  RED (#1)

 Last failure was on 02-08 for ovirt-engine due failure in
 build-artifacts, which was caused by gerrit issue, which was reported
 Evgheni.

  Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be
 found here:

 [1]
 http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

 [2]
 https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

 [3]
 http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

 Have a nice week!
 Dusan



 ---
 COLOUR MAP

 Green = job has been passing successfully

 ** green for more than 3 days may suggest we need a review of our test
 coverage


1.

1-3 days   GREEN (#1)
2.

4-7 days   GREEN (#2)
3.

Over 7 days GREEN (#3)


 Yellow = intermittent failures for different projects but no lasting or
 current regressions

 ** intermittent would be a healthy project as we expect a number of
 failures during the week

 ** I will not report any of the solved failures or regressions.


1.

Solved job failuresYELLOW (#1)
2.

Solved regressions  YELLOW (#2)


 Red = job has been failing

 ** Active Failures. The colour will change based on the amount of time
 the project/s has been broken. Only active regressions would be reported.


1.

1-3 days  RED (#1)
2.

4-7 days  RED (#2)
3.

Over 7 days RED (#3)

 ___
 Devel mailing list -- devel@ovirt.org
 To unsubscribe send an email to devel-le...@ovirt.org
 Privacy Statement: https://www.ovirt.org/site/privacy-policy/
 oVirt Code of Conduct:
 https://www.ovirt.org/community/about/community-guidelines/
 List Archives:
 https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YCNCKRK3G4EJXA3OCYAUS4VMKRDA67F4/

>>>
>>>
>>> --
>>>
>>> Sandro Bonazzola
>>>
>>> MANAGER, SOFTWARE ENGINEERING, EMEA R RHV
>>>
>>> Red Hat EMEA 
>>>
>>> sbona...@redhat.com
>>> *Red Hat respects your work life balance.
>>> Therefore there is no need to answer this email out of your office hours.
>>> *
>>>
>>
>
> --
>
> Eyal edri
>
> He / Him / His
>
>
> MANAGER
>
> CONTINUOUS PRODUCTIZATION
>
> SYSTEM ENGINEERING
>
> Red Hat 
> 
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel)
> ___
> Infra mailing list -- in...@ovirt.org
> To unsubscribe send an email to infra-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/in...@ovirt.org/message/IUUPA6RTP5BMBQC4JYWEYTYKQC6BCN2H/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DPELS26ERWH464APDF3SRFGSMSXOK3QQ/

[ovirt-devel] Re: OST fails, cannot connect to repo

[Dafna Ron]

it seems like you are downloading from external mirror.
please use local mirrors (this fix should be done in you project)


On Thu, Jul 18, 2019 at 10:42 AM Vojtech Juranek 
wrote:

> Hi,
> OST fails with
>
> 09:47:03
> https://copr-be.cloud.fedoraproject.org/results/sac/gluster-ansible/
> epel-7-x86_64/repodata/repomd.xml
> :
> [Errno 14] curl#7 - "Failed connect to
> copr-be.cloud.fedoraproject.org:443; Connection refused"
>
> see e.g. [1] for full log. Stared to fail this morning.
> Can anyone take a look and fix it?
>
> Thanks in advance.
> Vojta
>
> [1] https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5132/console
> ___
> Infra mailing list -- in...@ovirt.org
> To unsubscribe send an email to infra-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/in...@ovirt.org/message/DXDNFFNWE3DC2IJTOH7CMXN7FD4PO4HU/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PAEWFBQKS2TP6XJSDCTUUHNBWANWEWLC/

[ovirt-devel] Re: HC suite is failing for 3 weeks on vm_run

[Dafna Ron]

Hi Eyal,

Please note these suites do not run on CQ and not monitored by the CI team
as they are team owned.
We also no longer run all suites for OST patches related to CQ as we found
it hard to get certain teams to fix issues quickly enough when we had a CQ
related issue and also, every check-patch took almost a full day to run due
to the amount of suites that were linked to the same repos.

Thanks,
Dafna


On Mon, Jul 15, 2019 at 2:32 PM Eyal Edri  wrote:

> FYI,
>
> Failing on vm_run test for 4.3:
>
> https://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.3/lastCompletedBuild/testReport/(root)/004_basic_sanity/vm_run/
>
> same on master:
>
> https://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/lastCompletedBuild/testReport/(root)/004_basic_sanity/vm_run/
>
>
> --
>
> Eyal edri
>
> He / Him / His
>
>
> MANAGER
>
> CONTINUOUS PRODUCTIZATION
>
> SYSTEM ENGINEERING
>
> Red Hat 
> 
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel)
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/RC5UKNIXICNMGERP3KDFMDZUMB33PCFR/

[ovirt-devel] [Ovirt] [CQ weekly status] [12-07-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last failure was on 02-07 for ovirt-provider-ovm due to failed test:
098_ovirt_provider_ovn.use_ovn_provide.
This was a code regression and was fixed by patch
https://gerrit.ovirt.org/#/c/97072/

*CQ-4.3*:  GREEN (#1)

Last failure was on 12-07 for ovirt-provider-ovm due to failed
build-artifacts which was caused due to a mirror issue.

*CQ-Master:*  RED (#1)

Last failure was on 12-07 for ovirt-provider-ovm due to failed
build-artifacts which was caused due to a mirror issue.

 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YCNCKRK3G4EJXA3OCYAUS4VMKRDA67F4/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 10-07-2019 ] [ 004_basic_sanity.vdsm_recovery ]

[Dafna Ron]

It seems there is a passing vdsm build after this one so whatever this was
is now fixed.
but I think the ksm path test should be fixed.
Adding Arik

On Wed, Jul 10, 2019 at 5:06 PM Dafna Ron  wrote:

>
>
> On Wed, Jul 10, 2019 at 4:18 PM Milan Zamazal  wrote:
>
>> Dafna Ron  writes:
>>
>> > Hi,
>> >
>> > We have a failure on test  004_basic_sanity.vdsm_recovery on basic
>> suite.
>> > the error seems to be an error in KSM (invalid arg)
>> >
>> > can you please have a look?
>> >
>> > Link and headline of suspected patches:
>> >
>> >
>> > cq identified this as the cause of failure:
>> >
>> > https://gerrit.ovirt.org/#/c/101603/ - localFsSD: Enable 4k block_size
>> and
>> > alignments
>> >
>> >
>> > However, I can see some py3 patches merged at the same time:
>> >
>> >
>> > py3: storage: Fix bytes x string in lvm locking type validation -
>> > https://gerrit.ovirt.org/#/c/101124/
>>
>> OST was successfully run on this patch before merging, so it's unlikely
>> to be the cause.
>>
>> > Link to Job:
>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14963/
>> >
>> > Link to all logs:
>> >
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14963/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/
>> >
>> > (Relevant) error snippet from the log:
>> >
>> > 
>> >
>> > s/da0eeccb-5dd8-47e5-9009-8a848fe17ea5.ovirt-guest-agent.0',) {}
>> > MainProcess|vm/da0eeccb::DEBUG::2019-07-10
>> > 07:53:41,003::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper)
>> > return prepareVmChannel with None
>> > MainProcess|jsonrpc/1::DEBUG::2019-07-10
>> > 07:54:05,580::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
>> > call ksmTune with ({u'pages_to_scan': 64, u'run': 1, u'sleep
>> > _millisecs': 89.25152465623417},) {}
>> > MainProcess|jsonrpc/1::ERROR::2019-07-10
>> > 07:54:05,581::supervdsm_server::103::SuperVdsm.ServerCallback::(wrapper)
>> > Error in ksmTune
>> > Traceback (most recent call last):
>> >   File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line
>> > 101, in wrapper
>> > res = func(*args, **kwargs)
>> >   File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_api/ksm.py",
>> line
>> > 45, in ksmTune
>> > f.write(str(v))
>>
>> This writes to files in /sys/kernel/mm/ksm/ and the values are checked
>> in the code.  It's also weird that Vdsm starts happily and then fails on
>> this in recovery.
>>
>> Can you exclude there is some problem with the system OST runs on?
>>
>
> I see there was a change in ksm patch 7 weeks ago which explains the
> failure we are seeing.
> However, I am not sure why its failing the test now and I am not seeing
> any other error that can cause this.
>
> Adding Ehud and Evgheni.
> Are the manual jobs running on containers or physical severs?
>
>
>
> https://gerrit.ovirt.org/#/c/95994/ - fix path of ksm files in a comment
>
>>
>> > IOError: [Errno 22] Invalid argument
>> > MainProcess|jsonrpc/5::DEBUG::2019-07-10
>> > 07:56:33,211::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
>> > call rmAppropriateMultipathRules with
>> > ('da0eeccb-5dd8-47e5-9009-8a848fe17ea5',) {}
>> >
>> >
>> > 
>>
>>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/VBEZWGXV4E2RUXVZVB2AZ6II3N4J763W/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 10-07-2019 ] [ 004_basic_sanity.vdsm_recovery ]

[Dafna Ron]

On Wed, Jul 10, 2019 at 4:18 PM Milan Zamazal  wrote:

> Dafna Ron  writes:
>
> > Hi,
> >
> > We have a failure on test  004_basic_sanity.vdsm_recovery on basic suite.
> > the error seems to be an error in KSM (invalid arg)
> >
> > can you please have a look?
> >
> > Link and headline of suspected patches:
> >
> >
> > cq identified this as the cause of failure:
> >
> > https://gerrit.ovirt.org/#/c/101603/ - localFsSD: Enable 4k block_size
> and
> > alignments
> >
> >
> > However, I can see some py3 patches merged at the same time:
> >
> >
> > py3: storage: Fix bytes x string in lvm locking type validation -
> > https://gerrit.ovirt.org/#/c/101124/
>
> OST was successfully run on this patch before merging, so it's unlikely
> to be the cause.
>
> > Link to Job:
> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14963/
> >
> > Link to all logs:
> >
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14963/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/
> >
> > (Relevant) error snippet from the log:
> >
> > 
> >
> > s/da0eeccb-5dd8-47e5-9009-8a848fe17ea5.ovirt-guest-agent.0',) {}
> > MainProcess|vm/da0eeccb::DEBUG::2019-07-10
> > 07:53:41,003::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper)
> > return prepareVmChannel with None
> > MainProcess|jsonrpc/1::DEBUG::2019-07-10
> > 07:54:05,580::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
> > call ksmTune with ({u'pages_to_scan': 64, u'run': 1, u'sleep
> > _millisecs': 89.25152465623417},) {}
> > MainProcess|jsonrpc/1::ERROR::2019-07-10
> > 07:54:05,581::supervdsm_server::103::SuperVdsm.ServerCallback::(wrapper)
> > Error in ksmTune
> > Traceback (most recent call last):
> >   File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line
> > 101, in wrapper
> > res = func(*args, **kwargs)
> >   File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_api/ksm.py", line
> > 45, in ksmTune
> > f.write(str(v))
>
> This writes to files in /sys/kernel/mm/ksm/ and the values are checked
> in the code.  It's also weird that Vdsm starts happily and then fails on
> this in recovery.
>
> Can you exclude there is some problem with the system OST runs on?
>

I see there was a change in ksm patch 7 weeks ago which explains the
failure we are seeing.
However, I am not sure why its failing the test now and I am not seeing any
other error that can cause this.

Adding Ehud and Evgheni.
Are the manual jobs running on containers or physical severs?



https://gerrit.ovirt.org/#/c/95994/ - fix path of ksm files in a comment

>
> > IOError: [Errno 22] Invalid argument
> > MainProcess|jsonrpc/5::DEBUG::2019-07-10
> > 07:56:33,211::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
> > call rmAppropriateMultipathRules with
> > ('da0eeccb-5dd8-47e5-9009-8a848fe17ea5',) {}
> >
> >
> > 
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NGGHPRTCFTJLLWYAITW7TDXV3BLGFM4Y/

[ovirt-devel] [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 10-07-2019 ] [ 004_basic_sanity.vdsm_recovery ]

[Dafna Ron]

Hi,

We have a failure on test  004_basic_sanity.vdsm_recovery on basic suite.
the error seems to be an error in KSM (invalid arg)

can you please have a look?

Link and headline of suspected patches:


cq identified this as the cause of failure:

https://gerrit.ovirt.org/#/c/101603/ - localFsSD: Enable 4k block_size and
alignments


However, I can see some py3 patches merged at the same time:


py3: storage: Fix bytes x string in lvm locking type validation -
https://gerrit.ovirt.org/#/c/101124/


Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14963/

Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14963/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/

(Relevant) error snippet from the log:



s/da0eeccb-5dd8-47e5-9009-8a848fe17ea5.ovirt-guest-agent.0',) {}
MainProcess|vm/da0eeccb::DEBUG::2019-07-10
07:53:41,003::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper)
return prepareVmChannel with None
MainProcess|jsonrpc/1::DEBUG::2019-07-10
07:54:05,580::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
call ksmTune with ({u'pages_to_scan': 64, u'run': 1, u'sleep
_millisecs': 89.25152465623417},) {}
MainProcess|jsonrpc/1::ERROR::2019-07-10
07:54:05,581::supervdsm_server::103::SuperVdsm.ServerCallback::(wrapper)
Error in ksmTune
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line
101, in wrapper
res = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_api/ksm.py", line
45, in ksmTune
f.write(str(v))
IOError: [Errno 22] Invalid argument
MainProcess|jsonrpc/5::DEBUG::2019-07-10
07:56:33,211::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
call rmAppropriateMultipathRules with
('da0eeccb-5dd8-47e5-9009-8a848fe17ea5',) {}



___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/2E2FERLTPLPSN3LZFHH3DYJTJGTKGKHK/

[ovirt-devel] Re: OST fail to install hosts: ​requires: device-mapper = 7:1.02.149-10.el7_6.8 installed: ​device-mapper-1.02.149-10.el7_6.7.x86_64

[Dafna Ron]

Galit merged the patch and jobs are now passing. you can try to run ost
again

On Mon, Jul 8, 2019 at 1:10 PM Dafna Ron  wrote:

> you can fix the tests and add a rhel update to all engine and hosts
> deployment
>
> On Mon, Jul 8, 2019 at 1:09 PM Nir Soffer  wrote:
>
>>
>>
>> On Mon, Jul 8, 2019, 10:28 Dafna Ron  wrote:
>>
>>> sorry, it has 7.6.7 packages instead of 7.6.6 packages and in order to
>>> resolve this in ost we either need to add rhel update to all machines in
>>> all suites (not just for engine in upgrade suites) or Galit has to do a new
>>> image which she is doing now
>>>
>>
>> This block merging in master, we don't merge without passing OST.
>>
>> How can we prevent this from happening in the next update (7.6.8)?
>>
>>
>>>
>>> On Mon, Jul 8, 2019 at 8:26 AM Dafna Ron  wrote:
>>>
>>>> you're right. its 7.6.7 instead of 7.6.6 :)
>>>>
>>>> On Mon, Jul 8, 2019 at 8:25 AM Sandro Bonazzola 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> Il giorno ven 5 lug 2019 alle ore 12:35 Dafna Ron 
>>>>> ha scritto:
>>>>>
>>>>>> The issue is that the images installed by ost are 7.6 which means the
>>>>>> packages are 7.6 packages
>>>>>>
>>>>>> templates:
>>>>>>   engine: el7.6-base-4
>>>>>>   host: el7.6-base-4
>>>>>>
>>>>>> however, the packages coming from the repo are 7.7
>>>>>>
>>>>>
>>>>> No, packages are not from 7.7. They are from 7.6 (el7_6)
>>>>> Errata is here:
>>>>> https://lists.centos.org/pipermail/centos-announce/2019-July/023356.html
>>>>> They were released in RHEL 7.6 batch #5 on June 4th. Sad it took a
>>>>> month to get into OST.
>>>>>
>>>>>
>>>>>>
>>>>>> Repository google-chrome is listed more than once in the configuration
>>>>>> pcp-pmda-systemd-0:4.1.0-5.el7_6.x86_64
>>>>>> systemd-0:219-62.el7_6.7.x86_64
>>>>>> systemd-devel-0:219-62.el7_6.7.i686
>>>>>> systemd-devel-0:219-62.el7_6.7.x86_64
>>>>>> systemd-journal-gateway-0:219-62.el7_6.7.x86_64
>>>>>> systemd-libs-0:219-62.el7_6.7.i686
>>>>>> systemd-libs-0:219-62.el7_6.7.x86_64
>>>>>> systemd-networkd-0:219-62.el7_6.7.x86_64
>>>>>> systemd-python-0:219-62.el7_6.7.x86_64
>>>>>> systemd-resolved-0:219-62.el7_6.7.i686
>>>>>> systemd-resolved-0:219-62.el7_6.7.x86_64
>>>>>> systemd-sysv-0:219-62.el7_6.7.x86_64
>>>>>> [dron@dron ovirt-system-tests]$
>>>>>>
>>>>>> so if there is a new centos 7.7 now, we should update the ost image
>>>>>> to 7.7
>>>>>>
>>>>>> Another solution to avoid such failures in the future is to add a yum
>>>>>> update to the test suite and a reboot to the vm, that case we will not 
>>>>>> fail
>>>>>> on new os version in initialize engine
>>>>>>
>>>>>> Adding Galit as she needs to create a new image
>>>>>>
>>>>>> Thanks,
>>>>>> Dafna
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 4, 2019 at 8:41 PM Nir Soffer  wrote:
>>>>>>
>>>>>>> I had 3 failures with OST master:
>>>>>>>
>>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5061/pipeline/
>>>>>>>
>>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5062/pipeline
>>>>>>>
>>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5063/pipeline
>>>>>>>
>>>>>>> First 2 builds are mostly logging fixes in vdsm and removing dead
>>>>>>> code, unrelated to the failures.
>>>>>>> Last build is from master, same patch passed OST today here:
>>>>>>>
>>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5059/pipeline/
>>>>>>>
>>>&g

[ovirt-devel] Re: OST fail to install hosts: ​requires: device-mapper = 7:1.02.149-10.el7_6.8 installed: ​device-mapper-1.02.149-10.el7_6.7.x86_64

[Dafna Ron]

you can fix the tests and add a rhel update to all engine and hosts
deployment

On Mon, Jul 8, 2019 at 1:09 PM Nir Soffer  wrote:

>
>
> On Mon, Jul 8, 2019, 10:28 Dafna Ron  wrote:
>
>> sorry, it has 7.6.7 packages instead of 7.6.6 packages and in order to
>> resolve this in ost we either need to add rhel update to all machines in
>> all suites (not just for engine in upgrade suites) or Galit has to do a new
>> image which she is doing now
>>
>
> This block merging in master, we don't merge without passing OST.
>
> How can we prevent this from happening in the next update (7.6.8)?
>
>
>>
>> On Mon, Jul 8, 2019 at 8:26 AM Dafna Ron  wrote:
>>
>>> you're right. its 7.6.7 instead of 7.6.6 :)
>>>
>>> On Mon, Jul 8, 2019 at 8:25 AM Sandro Bonazzola 
>>> wrote:
>>>
>>>>
>>>>
>>>> Il giorno ven 5 lug 2019 alle ore 12:35 Dafna Ron  ha
>>>> scritto:
>>>>
>>>>> The issue is that the images installed by ost are 7.6 which means the
>>>>> packages are 7.6 packages
>>>>>
>>>>> templates:
>>>>>   engine: el7.6-base-4
>>>>>   host: el7.6-base-4
>>>>>
>>>>> however, the packages coming from the repo are 7.7
>>>>>
>>>>
>>>> No, packages are not from 7.7. They are from 7.6 (el7_6)
>>>> Errata is here:
>>>> https://lists.centos.org/pipermail/centos-announce/2019-July/023356.html
>>>> They were released in RHEL 7.6 batch #5 on June 4th. Sad it took a
>>>> month to get into OST.
>>>>
>>>>
>>>>>
>>>>> Repository google-chrome is listed more than once in the configuration
>>>>> pcp-pmda-systemd-0:4.1.0-5.el7_6.x86_64
>>>>> systemd-0:219-62.el7_6.7.x86_64
>>>>> systemd-devel-0:219-62.el7_6.7.i686
>>>>> systemd-devel-0:219-62.el7_6.7.x86_64
>>>>> systemd-journal-gateway-0:219-62.el7_6.7.x86_64
>>>>> systemd-libs-0:219-62.el7_6.7.i686
>>>>> systemd-libs-0:219-62.el7_6.7.x86_64
>>>>> systemd-networkd-0:219-62.el7_6.7.x86_64
>>>>> systemd-python-0:219-62.el7_6.7.x86_64
>>>>> systemd-resolved-0:219-62.el7_6.7.i686
>>>>> systemd-resolved-0:219-62.el7_6.7.x86_64
>>>>> systemd-sysv-0:219-62.el7_6.7.x86_64
>>>>> [dron@dron ovirt-system-tests]$
>>>>>
>>>>> so if there is a new centos 7.7 now, we should update the ost image to
>>>>> 7.7
>>>>>
>>>>> Another solution to avoid such failures in the future is to add a yum
>>>>> update to the test suite and a reboot to the vm, that case we will not 
>>>>> fail
>>>>> on new os version in initialize engine
>>>>>
>>>>> Adding Galit as she needs to create a new image
>>>>>
>>>>> Thanks,
>>>>> Dafna
>>>>>
>>>>>
>>>>> On Thu, Jul 4, 2019 at 8:41 PM Nir Soffer  wrote:
>>>>>
>>>>>> I had 3 failures with OST master:
>>>>>>
>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5061/pipeline/
>>>>>>
>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5062/pipeline
>>>>>>
>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5063/pipeline
>>>>>>
>>>>>> First 2 builds are mostly logging fixes in vdsm and removing dead
>>>>>> code, unrelated to the failures.
>>>>>> Last build is from master, same patch passed OST today here:
>>>>>>
>>>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5059/pipeline/
>>>>>>
>>>>>> Looks like some required packages are missing.
>>>>>>
>>>>>> + yum -y install ovirt-host
>>>>>>
>>>>>> Error: Package: 7:device-mapper-event-1.02.149-10.el7_6.8.x86_64
>>>>>> (alocalsync)
>>>>>>
>>>>>>Requires: device-mapper = 7:1.02.149-10.el7_6.8
>>>>>>
>>>>>>Installed: 7:device-mapper-1.02.149-10.el7_6.7.x86_

[ovirt-devel] Re: OST fail to install hosts: ​requires: device-mapper = 7:1.02.149-10.el7_6.8 installed: ​device-mapper-1.02.149-10.el7_6.7.x86_64

[Dafna Ron]

sorry, it has 7.6.7 packages instead of 7.6.6 packages and in order to
resolve this in ost we either need to add rhel update to all machines in
all suites (not just for engine in upgrade suites) or Galit has to do a new
image which she is doing now


On Mon, Jul 8, 2019 at 8:26 AM Dafna Ron  wrote:

> you're right. its 7.6.7 instead of 7.6.6 :)
>
> On Mon, Jul 8, 2019 at 8:25 AM Sandro Bonazzola 
> wrote:
>
>>
>>
>> Il giorno ven 5 lug 2019 alle ore 12:35 Dafna Ron  ha
>> scritto:
>>
>>> The issue is that the images installed by ost are 7.6 which means the
>>> packages are 7.6 packages
>>>
>>> templates:
>>>   engine: el7.6-base-4
>>>   host: el7.6-base-4
>>>
>>> however, the packages coming from the repo are 7.7
>>>
>>
>> No, packages are not from 7.7. They are from 7.6 (el7_6)
>> Errata is here:
>> https://lists.centos.org/pipermail/centos-announce/2019-July/023356.html
>> They were released in RHEL 7.6 batch #5 on June 4th. Sad it took a month
>> to get into OST.
>>
>>
>>>
>>> Repository google-chrome is listed more than once in the configuration
>>> pcp-pmda-systemd-0:4.1.0-5.el7_6.x86_64
>>> systemd-0:219-62.el7_6.7.x86_64
>>> systemd-devel-0:219-62.el7_6.7.i686
>>> systemd-devel-0:219-62.el7_6.7.x86_64
>>> systemd-journal-gateway-0:219-62.el7_6.7.x86_64
>>> systemd-libs-0:219-62.el7_6.7.i686
>>> systemd-libs-0:219-62.el7_6.7.x86_64
>>> systemd-networkd-0:219-62.el7_6.7.x86_64
>>> systemd-python-0:219-62.el7_6.7.x86_64
>>> systemd-resolved-0:219-62.el7_6.7.i686
>>> systemd-resolved-0:219-62.el7_6.7.x86_64
>>> systemd-sysv-0:219-62.el7_6.7.x86_64
>>> [dron@dron ovirt-system-tests]$
>>>
>>> so if there is a new centos 7.7 now, we should update the ost image to
>>> 7.7
>>>
>>> Another solution to avoid such failures in the future is to add a yum
>>> update to the test suite and a reboot to the vm, that case we will not fail
>>> on new os version in initialize engine
>>>
>>> Adding Galit as she needs to create a new image
>>>
>>> Thanks,
>>> Dafna
>>>
>>>
>>> On Thu, Jul 4, 2019 at 8:41 PM Nir Soffer  wrote:
>>>
>>>> I had 3 failures with OST master:
>>>>
>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5061/pipeline/
>>>>
>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5062/pipeline
>>>>
>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5063/pipeline
>>>>
>>>> First 2 builds are mostly logging fixes in vdsm and removing dead code,
>>>> unrelated to the failures.
>>>> Last build is from master, same patch passed OST today here:
>>>>
>>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5059/pipeline/
>>>>
>>>> Looks like some required packages are missing.
>>>>
>>>> + yum -y install ovirt-host
>>>>
>>>> Error: Package: 7:device-mapper-event-1.02.149-10.el7_6.8.x86_64
>>>> (alocalsync)
>>>>
>>>>Requires: device-mapper = 7:1.02.149-10.el7_6.8
>>>>
>>>>Installed: 7:device-mapper-1.02.149-10.el7_6.7.x86_64
>>>> (installed)
>>>>
>>>>device-mapper = 7:1.02.149-10.el7_6.7
>>>>
>>>> Error: Package: systemd-python-219-62.el7_6.7.x86_64 (alocalsync)
>>>>
>>>>Requires: systemd-libs = 219-62.el7_6.7
>>>>
>>>>Installed: systemd-libs-219-62.el7_6.6.x86_64 (installed)
>>>>
>>>>systemd-libs = 219-62.el7_6.6
>>>>
>>>> Error: Package: systemd-python-219-62.el7_6.7.x86_64 (alocalsync)
>>>>
>>>>Requires: systemd = 219-62.el7_6.7
>>>>
>>>>Installed: systemd-219-62.el7_6.6.x86_64 (installed)
>>>>
>>>>systemd = 219-62.el7_6.6
>>>>
>>>> Error: Package: libgudev1-219-62.el7_6.7.x86_64 (alocalsync)
>>>>
>>>>Requires: systemd-libs = 219-62.el7_6.7
>>>>
>>>>Installed: systemd-libs-219-62.el7_6.6.x86_64 (installed)
>>>>
>>>>systemd-libs = 219-62.el7_6.6
>>>>
>>>>
>>>>
>>
>> --
>>
>> Sandro Bonazzola
>>
>> MANAGER, SOFTWARE ENGINEERING, EMEA R RHV
>>
>> Red Hat EMEA <https://www.redhat.com/>
>>
>> sbona...@redhat.com
>> <https://www.redhat.com/>
>>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GNPQDKQWZ4FQMPWUYFUO3V4QWUUXA4QS/

[ovirt-devel] Re: OST fail to install hosts: ​requires: device-mapper = 7:1.02.149-10.el7_6.8 installed: ​device-mapper-1.02.149-10.el7_6.7.x86_64

[Dafna Ron]

you're right. its 7.6.7 instead of 7.6.6 :)

On Mon, Jul 8, 2019 at 8:25 AM Sandro Bonazzola  wrote:

>
>
> Il giorno ven 5 lug 2019 alle ore 12:35 Dafna Ron  ha
> scritto:
>
>> The issue is that the images installed by ost are 7.6 which means the
>> packages are 7.6 packages
>>
>> templates:
>>   engine: el7.6-base-4
>>   host: el7.6-base-4
>>
>> however, the packages coming from the repo are 7.7
>>
>
> No, packages are not from 7.7. They are from 7.6 (el7_6)
> Errata is here:
> https://lists.centos.org/pipermail/centos-announce/2019-July/023356.html
> They were released in RHEL 7.6 batch #5 on June 4th. Sad it took a month
> to get into OST.
>
>
>>
>> Repository google-chrome is listed more than once in the configuration
>> pcp-pmda-systemd-0:4.1.0-5.el7_6.x86_64
>> systemd-0:219-62.el7_6.7.x86_64
>> systemd-devel-0:219-62.el7_6.7.i686
>> systemd-devel-0:219-62.el7_6.7.x86_64
>> systemd-journal-gateway-0:219-62.el7_6.7.x86_64
>> systemd-libs-0:219-62.el7_6.7.i686
>> systemd-libs-0:219-62.el7_6.7.x86_64
>> systemd-networkd-0:219-62.el7_6.7.x86_64
>> systemd-python-0:219-62.el7_6.7.x86_64
>> systemd-resolved-0:219-62.el7_6.7.i686
>> systemd-resolved-0:219-62.el7_6.7.x86_64
>> systemd-sysv-0:219-62.el7_6.7.x86_64
>> [dron@dron ovirt-system-tests]$
>>
>> so if there is a new centos 7.7 now, we should update the ost image to
>> 7.7
>>
>> Another solution to avoid such failures in the future is to add a yum
>> update to the test suite and a reboot to the vm, that case we will not fail
>> on new os version in initialize engine
>>
>> Adding Galit as she needs to create a new image
>>
>> Thanks,
>> Dafna
>>
>>
>> On Thu, Jul 4, 2019 at 8:41 PM Nir Soffer  wrote:
>>
>>> I had 3 failures with OST master:
>>>
>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5061/pipeline/
>>>
>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5062/pipeline
>>>
>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5063/pipeline
>>>
>>> First 2 builds are mostly logging fixes in vdsm and removing dead code,
>>> unrelated to the failures.
>>> Last build is from master, same patch passed OST today here:
>>>
>>> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5059/pipeline/
>>>
>>> Looks like some required packages are missing.
>>>
>>> + yum -y install ovirt-host
>>>
>>> Error: Package: 7:device-mapper-event-1.02.149-10.el7_6.8.x86_64
>>> (alocalsync)
>>>
>>>Requires: device-mapper = 7:1.02.149-10.el7_6.8
>>>
>>>Installed: 7:device-mapper-1.02.149-10.el7_6.7.x86_64
>>> (installed)
>>>
>>>device-mapper = 7:1.02.149-10.el7_6.7
>>>
>>> Error: Package: systemd-python-219-62.el7_6.7.x86_64 (alocalsync)
>>>
>>>Requires: systemd-libs = 219-62.el7_6.7
>>>
>>>Installed: systemd-libs-219-62.el7_6.6.x86_64 (installed)
>>>
>>>systemd-libs = 219-62.el7_6.6
>>>
>>> Error: Package: systemd-python-219-62.el7_6.7.x86_64 (alocalsync)
>>>
>>>Requires: systemd = 219-62.el7_6.7
>>>
>>>Installed: systemd-219-62.el7_6.6.x86_64 (installed)
>>>
>>>systemd = 219-62.el7_6.6
>>>
>>> Error: Package: libgudev1-219-62.el7_6.7.x86_64 (alocalsync)
>>>
>>>Requires: systemd-libs = 219-62.el7_6.7
>>>
>>>Installed: systemd-libs-219-62.el7_6.6.x86_64 (installed)
>>>
>>>systemd-libs = 219-62.el7_6.6
>>>
>>>
>>>
>
> --
>
> Sandro Bonazzola
>
> MANAGER, SOFTWARE ENGINEERING, EMEA R RHV
>
> Red Hat EMEA <https://www.redhat.com/>
>
> sbona...@redhat.com
> <https://www.redhat.com/>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P326IYEVHUVXEO3XLSYOM43T6STVKKWQ/

[ovirt-devel] [Ovirt] [CQ weekly status] [05-07-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last failure was on 02-07 for ovirt-provider-ovm due to failed test:
098_ovirt_provider_ovn.use_ovn_provide.
This was a code regression and was fixed by patch
https://gerrit.ovirt.org/#/c/97072/

*CQ-4.3*:  RED (#1)

We are failing because our ost image is centos 7.6 and we now have 7.7
packages.
Galit will need to create a new image to fix the failing tests.

*CQ-Master:*  RED (#1)

We are failing because our ost image is centos 7.6 and we now have 7.7
packages.
Galit will need to create a new image to fix the failing tests.

 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/65Y52U4DML4CPQLYEPJYWIIPBZQVIKQM/

[ovirt-devel] Re: OST fail to install hosts: ​requires: device-mapper = 7:1.02.149-10.el7_6.8 installed: ​device-mapper-1.02.149-10.el7_6.7.x86_64

[Dafna Ron]

The issue is that the images installed by ost are 7.6 which means the
packages are 7.6 packages

templates:
  engine: el7.6-base-4
  host: el7.6-base-4

however, the packages coming from the repo are 7.7

Repository google-chrome is listed more than once in the configuration
pcp-pmda-systemd-0:4.1.0-5.el7_6.x86_64
systemd-0:219-62.el7_6.7.x86_64
systemd-devel-0:219-62.el7_6.7.i686
systemd-devel-0:219-62.el7_6.7.x86_64
systemd-journal-gateway-0:219-62.el7_6.7.x86_64
systemd-libs-0:219-62.el7_6.7.i686
systemd-libs-0:219-62.el7_6.7.x86_64
systemd-networkd-0:219-62.el7_6.7.x86_64
systemd-python-0:219-62.el7_6.7.x86_64
systemd-resolved-0:219-62.el7_6.7.i686
systemd-resolved-0:219-62.el7_6.7.x86_64
systemd-sysv-0:219-62.el7_6.7.x86_64
[dron@dron ovirt-system-tests]$

so if there is a new centos 7.7 now, we should update the ost image to 7.7

Another solution to avoid such failures in the future is to add a yum
update to the test suite and a reboot to the vm, that case we will not fail
on new os version in initialize engine

Adding Galit as she needs to create a new image

Thanks,
Dafna


On Thu, Jul 4, 2019 at 8:41 PM Nir Soffer  wrote:

> I had 3 failures with OST master:
>
> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5061/pipeline/
>
> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5062/pipeline
>
> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5063/pipeline
>
> First 2 builds are mostly logging fixes in vdsm and removing dead code,
> unrelated to the failures.
> Last build is from master, same patch passed OST today here:
>
> https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_manual/detail/ovirt-system-tests_manual/5059/pipeline/
>
> Looks like some required packages are missing.
>
> + yum -y install ovirt-host
>
> Error: Package: 7:device-mapper-event-1.02.149-10.el7_6.8.x86_64
> (alocalsync)
>
>Requires: device-mapper = 7:1.02.149-10.el7_6.8
>
>Installed: 7:device-mapper-1.02.149-10.el7_6.7.x86_64
> (installed)
>
>device-mapper = 7:1.02.149-10.el7_6.7
>
> Error: Package: systemd-python-219-62.el7_6.7.x86_64 (alocalsync)
>
>Requires: systemd-libs = 219-62.el7_6.7
>
>Installed: systemd-libs-219-62.el7_6.6.x86_64 (installed)
>
>systemd-libs = 219-62.el7_6.6
>
> Error: Package: systemd-python-219-62.el7_6.7.x86_64 (alocalsync)
>
>Requires: systemd = 219-62.el7_6.7
>
>Installed: systemd-219-62.el7_6.6.x86_64 (installed)
>
>systemd = 219-62.el7_6.6
>
> Error: Package: libgudev1-219-62.el7_6.7.x86_64 (alocalsync)
>
>Requires: systemd-libs = 219-62.el7_6.7
>
>Installed: systemd-libs-219-62.el7_6.6.x86_64 (installed)
>
>systemd-libs = 219-62.el7_6.6
>
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/AI4DJMH4QFBJARAR6LSDRIKKRI2FD7PP/

[ovirt-devel] Re: mom install error on fc29

[Dafna Ron]

On Thu, Jun 20, 2019 at 5:35 PM Miguel Duarte de Mora Barroso <
mdbarr...@redhat.com> wrote:

> Hi,
>
> I'm seeing the following error when I try to install
> ovirt-provider-ovn-driver (which requires vdsm):
>
> [MIRROR] mom-0.5.12-0.0.master.fc29.noarch.rpm: Interrupted by header
> callback: Server reports Content-Length: 340 but expected size is:
> 133356
> 17:13:50  [FAILED] mom-0.5.12-0.0.master.fc29.noarch.rpm: No more
> mirrors to try - All mirrors were already tried without success
> 17:13:50  Error: Error downloading packages:
> 17:13:50Cannot download
> noarch/mom-0.5.12-0.0.master.fc29.noarch.rpm: All mirrors were tried
>
> Any clue ?.. it can be seen in [0].
>
> Thanks in advance,
> Miguel
>
> [0] -
> https://jenkins.ovirt.org/job/ovirt-provider-ovn_standard-check-patch/2309/console
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/OLLWGI4NKMZD2TDCYLMIK3KXXET4RP55/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/OTHOUEQ3S5ZFXMAE55NCE6TGQGPLRKY4/

[ovirt-devel] Re: Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

Thanks Shani.
Tal, any updates on merging this?

On Mon, Jun 24, 2019 at 2:18 PM Shani Leviim  wrote:

> Hi Dafna,
> A patch for backporting was done: https://gerrit.ovirt.org/#/c/101113/
> Waiting for merging it.
>
>
> *Regards,*
>
> *Shani Leviim*
>
>
> On Mon, Jun 24, 2019 at 11:17 AM Dafna Ron  wrote:
>
>> Hi Shani and Tal,
>>
>> Can you backport this to 4.3 as well? we had a failure on 4.3 for same
>> issue but not for master.
>>
>> Thanks,
>> Dafna
>>
>>
>>
>> On Thu, Jun 20, 2019 at 3:39 PM Dafna Ron  wrote:
>>
>>> Hi Shani,
>>>
>>> Thanks for the patch
>>> I will monitor and in case we see any further failures will let you
>>> know.
>>>
>>> Thanks again,
>>> Dafna
>>>
>>>
>>> On Sun, Jun 16, 2019 at 3:53 PM Shani Leviim  wrote:
>>>
>>>> I've worked on this patch: https://gerrit.ovirt.org/#/c/100852/
>>>> Dafna, can you please try it?
>>>>
>>>>
>>>> *Regards,*
>>>>
>>>> *Shani Leviim*
>>>>
>>>>
>>>> On Fri, Jun 14, 2019 at 11:07 AM Dafna Ron  wrote:
>>>>
>>>>> and another:
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14559/
>>>>>
>>>>> On Fri, Jun 14, 2019 at 9:05 AM Dafna Ron  wrote:
>>>>>
>>>>>> here you go:
>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14561/
>>>>>> I pressed the save build forever button. please press the don't keep
>>>>>> button once you get the info from the failure.
>>>>>>
>>>>>> Thanks,
>>>>>> Dafna
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 5, 2019 at 9:09 AM Galit Rosenthal 
>>>>>> wrote:
>>>>>>
>>>>>>> We didn't make any changes.
>>>>>>> I don't think retrie will help in this case
>>>>>>> we are trying to catch what cause this.
>>>>>>>
>>>>>>> If you see this again please let us know.
>>>>>>>
>>>>>>> We are still debugging it
>>>>>>>
>>>>>>> On Wed, Jun 5, 2019 at 10:56 AM Dafna Ron  wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> This is a random failure - so no, I have not.
>>>>>>>> However, I looked at several failures and they are all the same,
>>>>>>>> the action on engine/vdsm side succeeds and lago repots a failure but
>>>>>>>> prints a success from logs.
>>>>>>>> Did you add anything more to the tests to allow better debugging?
>>>>>>>> Did you add a re-try to the test?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Dafna
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 5, 2019 at 8:21 AM Galit Rosenthal 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Dafna
>>>>>>>>>
>>>>>>>>> If you see this failure again, please send us the link to the job.
>>>>>>>>>
>>>>>>>>> We are trying to reproduce it and find the root cause.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Galit
>>>>>>>>>
>>>>>>>>> On Tue, May 21, 2019 at 5:15 PM Dafna Ron  wrote:
>>>>>>>>>
>>>>>>>>>> sure
>>>>>>>>>>
>>>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, May 21, 2019 at 3:04 PM Shani Leviim 
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Dafna,
>>>>>>>>>>> Can you please direct me to the full test's log?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>

[ovirt-devel] [Ovirt] [CQ weekly status] [28-06-2019]

[Dafna Ron]

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last failure was on 28-06 for mom due to failed test:
002_bootstrap.add_master_storage_domain. this is a known issue which has a
fix which was probably not added to the upgrade suite. I re-triggered the
change to see if it passes


*CQ-4.3*:  GREEN (#2)

Last failure was on 27-06-2019 for project ovirt-web-ui due to test
get_host_device which seemed to be a race. I re-triggered the patch and it
passed.

*CQ-Master:*  GREEN (#1)

Last failure was on 28-06-3019 for project ovirt-node-ng-image on
build-artifacts for fc29
There is a ticket opened https://ovirt-jira.atlassian.net/browse/OVIRT-2747

 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DUNGHY4OYXMIKDTXYB36E7PVXWDW3V4E/

[ovirt-devel] Re: [Ovirt] [CQ weekly status] [21-06-2019]

[Dafna Ron]

On Mon, Jun 24, 2019 at 4:15 PM Eyal Edri  wrote:

>
>
> On Fri, Jun 21, 2019 at 11:14 AM Dafna Ron  wrote:
>
>> This mail is to provide the current status of CQ and allow people to
>> review status before and after the weekend.
>> Please refer to below colour map for further information on the meaning
>> of the colours.
>>
>> *CQ-4.2*:  GREEN (#1)
>>
>> Last failure was on 18-06 for project v2v-conversion-host due to failed
>> build-artifacts
>> which is already fixed by next merged patches.
>>
>> *CQ-4.3*:  RED (#1)
>>
>> 1. We have a failure on ovirt-engine-metrics due to a dependency to a new
>> ansible package that has not been synced yet to centos repo.
>> I am adding virt-sig-common repo until this is synced next week:
>> https://gerrit.ovirt.org/#/c/101023/
>>
>
> It looks like its constantly failing on ovirt-ovn-provider now -
> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/1287/
>
>
>>
>> *CQ-Master:*  RED (#1)
>>
>> 1. same failure on ovirt-engine-metrics as 4.3
>>
>
> Is it fixed or still failing?
>
looks like metrics passed today

>
>
>> 2. ovirt-hosted-engine-setup is failing due to package dependency change
>> from python-libguestfs to python2-libguestfs. mail sent to Ido to check the
>> issue.
>>
>>
>>  Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be
>> found here:
>>
>> [1]
>> http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/
>>
>> [2]
>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/
>>
>> [3]
>> http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/
>>
>> Happy week!
>> Dafna
>>
>>
>>
>> ---
>> COLOUR MAP
>>
>> Green = job has been passing successfully
>>
>> ** green for more than 3 days may suggest we need a review of our test
>> coverage
>>
>>
>>1.
>>
>>1-3 days   GREEN (#1)
>>2.
>>
>>4-7 days   GREEN (#2)
>>3.
>>
>>Over 7 days GREEN (#3)
>>
>>
>> Yellow = intermittent failures for different projects but no lasting or
>> current regressions
>>
>> ** intermittent would be a healthy project as we expect a number of
>> failures during the week
>>
>> ** I will not report any of the solved failures or regressions.
>>
>>
>>1.
>>
>>Solved job failuresYELLOW (#1)
>>2.
>>
>>Solved regressions  YELLOW (#2)
>>
>>
>> Red = job has been failing
>>
>> ** Active Failures. The colour will change based on the amount of time
>> the project/s has been broken. Only active regressions would be reported.
>>
>>
>>1.
>>
>>1-3 days  RED (#1)
>>2.
>>
>>4-7 days  RED (#2)
>>3.
>>
>>Over 7 days RED (#3)
>>
>>
>
> --
>
> Eyal edri
>
> He / Him / His
>
>
> MANAGER
>
> CONTINUOUS PRODUCTIZATION
>
> SYSTEM ENGINEERING
>
> Red Hat <https://www.redhat.com/>
> <https://red.ht/sig>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel)
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/42BD6SQVNDHJK5GRSP3QEHXZ7E7TX4ER/

[ovirt-devel] Re: Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

Hi Shani and Tal,

Can you backport this to 4.3 as well? we had a failure on 4.3 for same
issue but not for master.

Thanks,
Dafna



On Thu, Jun 20, 2019 at 3:39 PM Dafna Ron  wrote:

> Hi Shani,
>
> Thanks for the patch
> I will monitor and in case we see any further failures will let you know.
>
> Thanks again,
> Dafna
>
>
> On Sun, Jun 16, 2019 at 3:53 PM Shani Leviim  wrote:
>
>> I've worked on this patch: https://gerrit.ovirt.org/#/c/100852/
>> Dafna, can you please try it?
>>
>>
>> *Regards,*
>>
>> *Shani Leviim*
>>
>>
>> On Fri, Jun 14, 2019 at 11:07 AM Dafna Ron  wrote:
>>
>>> and another:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14559/
>>>
>>> On Fri, Jun 14, 2019 at 9:05 AM Dafna Ron  wrote:
>>>
>>>> here you go:
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14561/
>>>> I pressed the save build forever button. please press the don't keep
>>>> button once you get the info from the failure.
>>>>
>>>> Thanks,
>>>> Dafna
>>>>
>>>>
>>>> On Wed, Jun 5, 2019 at 9:09 AM Galit Rosenthal 
>>>> wrote:
>>>>
>>>>> We didn't make any changes.
>>>>> I don't think retrie will help in this case
>>>>> we are trying to catch what cause this.
>>>>>
>>>>> If you see this again please let us know.
>>>>>
>>>>> We are still debugging it
>>>>>
>>>>> On Wed, Jun 5, 2019 at 10:56 AM Dafna Ron  wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> This is a random failure - so no, I have not.
>>>>>> However, I looked at several failures and they are all the same, the
>>>>>> action on engine/vdsm side succeeds and lago repots a failure but prints 
>>>>>> a
>>>>>> success from logs.
>>>>>> Did you add anything more to the tests to allow better debugging?
>>>>>> Did you add a re-try to the test?
>>>>>>
>>>>>> Thanks,
>>>>>> Dafna
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 5, 2019 at 8:21 AM Galit Rosenthal 
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Dafna
>>>>>>>
>>>>>>> If you see this failure again, please send us the link to the job.
>>>>>>>
>>>>>>> We are trying to reproduce it and find the root cause.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Galit
>>>>>>>
>>>>>>> On Tue, May 21, 2019 at 5:15 PM Dafna Ron  wrote:
>>>>>>>
>>>>>>>> sure
>>>>>>>>
>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 21, 2019 at 3:04 PM Shani Leviim 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Dafna,
>>>>>>>>> Can you please direct me to the full test's log?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Regards,*
>>>>>>>>>
>>>>>>>>> *Shani Leviim*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 21, 2019 at 11:57 AM Tal Nisan 
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Sure, Shani can you please have a look?
>>>>>>>>>>
>>>>>>>>>> On Mon, May 20, 2019 at 8:30 PM Dafna Ron 
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Tal,
>>>>>>>>>>>
>>>>>>>>>>> I am seeing random failures for test
>>>>>>>>>>> 002_bootstrap.resize_and_refresh_storage_domain
>>>>>>>>>>> It looks like this is a timing issue since by the time we print
>>>>>>>>>>> out the error we actually see the resize succeeded. see example 
>>>>>>>>>>> below:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/testReport/junit/(root)/002_bootstrap/running_tests___basic_suite_el7_x86_64___resize_and_refresh_storage_domain/
>>>>>>>>>>>
>>>>>>>>>>> Can you please assign someone from the storage team to fix this
>>>>>>>>>>> test?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Dafna
>>>>>>>>>>>
>>>>>>>>>>> ___
>>>>>>>> Devel mailing list -- devel@ovirt.org
>>>>>>>> To unsubscribe send an email to devel-le...@ovirt.org
>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>>>>>> oVirt Code of Conduct:
>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>>>> List Archives:
>>>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P4UZJ4PWELVW2AGLVIFXKM5OTMHUHKGC/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> GALIT ROSENTHAL
>>>>>>>
>>>>>>> SOFTWARE ENGINEER
>>>>>>>
>>>>>>> Red Hat
>>>>>>>
>>>>>>> <https://www.redhat.com/>
>>>>>>>
>>>>>>> ga...@redhat.comT: 972-9-7692230
>>>>>>> <https://red.ht/sig>
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> GALIT ROSENTHAL
>>>>>
>>>>> SOFTWARE ENGINEER
>>>>>
>>>>> Red Hat
>>>>>
>>>>> <https://www.redhat.com/>
>>>>>
>>>>> ga...@redhat.comT: 972-9-7692230
>>>>> <https://red.ht/sig>
>>>>>
>>>>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/LGQ7S7NLQQ22VQ44T6OLEW2EKII667K5/

[ovirt-devel] [Ovirt] [CQ weekly status] [21-06-2019]

[Dafna Ron]

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last failure was on 18-06 for project v2v-conversion-host due to failed
build-artifacts
which is already fixed by next merged patches.

*CQ-4.3*:  RED (#1)

1. We have a failure on ovirt-engine-metrics due to a dependency to a new
ansible package that has not been synced yet to centos repo.
I am adding virt-sig-common repo until this is synced next week:
https://gerrit.ovirt.org/#/c/101023/

*CQ-Master:*  RED (#1)

1. same failure on ovirt-engine-metrics as 4.3
2. ovirt-hosted-engine-setup is failing due to package dependency change
from python-libguestfs to python2-libguestfs. mail sent to Ido to check the
issue.


 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/FFCX4RLOHSNC4DHOEUDOML2LUCOTMXQA/

[ovirt-devel] Re: Builds on s390x fail again

[Dafna Ron]

Hi Nir,

I can see that the normal build-artifacts job for vdsm has the s360x build.
also, please note that there is no engine build for any architectures
except x68_64 so you cannot run ost on it.

Thanks,
Dafna



On Wed, Jun 19, 2019 at 3:20 AM Nir Soffer  wrote:

> I cannot see any reason for the failure in the pipeline logs, and there
> are no
> build logs.
>
> https://jenkins.ovirt.org/job/standard-manual-runner/329/
>
> https://jenkins.ovirt.org/job/standard-manual-runner/329//artifact/build-artifacts.build-py27.fc29.s390x/mock_logs/script/stdout_stderr.log
>
> The el7 x86_64 repo was created, so I can still use this build to run OST,
> but we need to fix this
> quickly, or disable the s390x builds until we have a stable solution.
>
> Nir
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/LCLDTPSWIF36BUWQ7VSONCYJM3LC5NDJ/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MJWWSWJ6EXGGUOWK67OXPPTMWZD7773Y/

[ovirt-devel] Re: Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

Hi Shani,

Thanks for the patch
I will monitor and in case we see any further failures will let you know.

Thanks again,
Dafna


On Sun, Jun 16, 2019 at 3:53 PM Shani Leviim  wrote:

> I've worked on this patch: https://gerrit.ovirt.org/#/c/100852/
> Dafna, can you please try it?
>
>
> *Regards,*
>
> *Shani Leviim*
>
>
> On Fri, Jun 14, 2019 at 11:07 AM Dafna Ron  wrote:
>
>> and another:
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14559/
>>
>> On Fri, Jun 14, 2019 at 9:05 AM Dafna Ron  wrote:
>>
>>> here you go:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14561/
>>> I pressed the save build forever button. please press the don't keep
>>> button once you get the info from the failure.
>>>
>>> Thanks,
>>> Dafna
>>>
>>>
>>> On Wed, Jun 5, 2019 at 9:09 AM Galit Rosenthal 
>>> wrote:
>>>
>>>> We didn't make any changes.
>>>> I don't think retrie will help in this case
>>>> we are trying to catch what cause this.
>>>>
>>>> If you see this again please let us know.
>>>>
>>>> We are still debugging it
>>>>
>>>> On Wed, Jun 5, 2019 at 10:56 AM Dafna Ron  wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> This is a random failure - so no, I have not.
>>>>> However, I looked at several failures and they are all the same, the
>>>>> action on engine/vdsm side succeeds and lago repots a failure but prints a
>>>>> success from logs.
>>>>> Did you add anything more to the tests to allow better debugging?
>>>>> Did you add a re-try to the test?
>>>>>
>>>>> Thanks,
>>>>> Dafna
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 5, 2019 at 8:21 AM Galit Rosenthal 
>>>>> wrote:
>>>>>
>>>>>> Hi Dafna
>>>>>>
>>>>>> If you see this failure again, please send us the link to the job.
>>>>>>
>>>>>> We are trying to reproduce it and find the root cause.
>>>>>>
>>>>>> Regards,
>>>>>> Galit
>>>>>>
>>>>>> On Tue, May 21, 2019 at 5:15 PM Dafna Ron  wrote:
>>>>>>
>>>>>>> sure
>>>>>>>
>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/
>>>>>>>
>>>>>>>
>>>>>>> On Tue, May 21, 2019 at 3:04 PM Shani Leviim 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Dafna,
>>>>>>>> Can you please direct me to the full test's log?
>>>>>>>>
>>>>>>>>
>>>>>>>> *Regards,*
>>>>>>>>
>>>>>>>> *Shani Leviim*
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 21, 2019 at 11:57 AM Tal Nisan 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Sure, Shani can you please have a look?
>>>>>>>>>
>>>>>>>>> On Mon, May 20, 2019 at 8:30 PM Dafna Ron  wrote:
>>>>>>>>>
>>>>>>>>>> Hi Tal,
>>>>>>>>>>
>>>>>>>>>> I am seeing random failures for test
>>>>>>>>>> 002_bootstrap.resize_and_refresh_storage_domain
>>>>>>>>>> It looks like this is a timing issue since by the time we print
>>>>>>>>>> out the error we actually see the resize succeeded. see example 
>>>>>>>>>> below:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/testReport/junit/(root)/002_bootstrap/running_tests___basic_suite_el7_x86_64___resize_and_refresh_storage_domain/
>>>>>>>>>>
>>>>>>>>>> Can you please assign someone from the storage team to fix this
>>>>>>>>>> test?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Dafna
>>>>>>>>>>
>>>>>>>>>> ___
>>>>>>> Devel mailing list -- devel@ovirt.org
>>>>>>> To unsubscribe send an email to devel-le...@ovirt.org
>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>>>>> oVirt Code of Conduct:
>>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>>> List Archives:
>>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P4UZJ4PWELVW2AGLVIFXKM5OTMHUHKGC/
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> GALIT ROSENTHAL
>>>>>>
>>>>>> SOFTWARE ENGINEER
>>>>>>
>>>>>> Red Hat
>>>>>>
>>>>>> <https://www.redhat.com/>
>>>>>>
>>>>>> ga...@redhat.comT: 972-9-7692230
>>>>>> <https://red.ht/sig>
>>>>>>
>>>>>
>>>>
>>>> --
>>>>
>>>> GALIT ROSENTHAL
>>>>
>>>> SOFTWARE ENGINEER
>>>>
>>>> Red Hat
>>>>
>>>> <https://www.redhat.com/>
>>>>
>>>> ga...@redhat.comT: 972-9-7692230
>>>> <https://red.ht/sig>
>>>>
>>>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NSFMZ3SFQPQCL2T34ZYSU2CXSPSHBXXR/

[ovirt-devel] [Ovirt] [CQ weekly status] [14-06-2019]

[Dafna Ron]

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#3)

Last failure was on 05-06 for project ovirt-ansible-cluster-upgrade due to
failure in test 002_bootstrap.add_master_storage_domain.
The fix was created and merged

*CQ-4.3*:  GREEN (#1)

There were two main failures this week.
1. vdsm was failing due to the sos change which was causing metrics tests
to fail as it was using vdsm sos plugin and that is no longer available.
this was fix by mperina:
https://gerrit.ovirt.org/100716
https://gerrit.ovirt.org/100809

2. there was a general failure due to packages. two patches were merged to
fix the issue:
https://gerrit.ovirt.org/#/c/100764/
https://gerrit.ovirt.org/#/c/100691/

*CQ-Master:*  GREEN (#1)

Master was failing on the same issues as in 4.3 which are now fixed


 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/Q2GYU4HYEHY6RZVOUZXVFYQG7Z67SBBJ/

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

thanks Martin!
we have a passing build on 4.3 as well
thanks for all the help :)
Dafna


On Thu, Jun 13, 2019 at 2:34 PM Martin Perina  wrote:

>
>
> On Thu, Jun 13, 2019 at 2:32 PM Dafna Ron  wrote:
>
>> will monitor new change once merged :)
>>
>
>> On Thu, Jun 13, 2019 at 1:23 PM Martin Perina  wrote:
>>
>>>
>>>
>>> On Thu, Jun 13, 2019 at 2:02 PM Dafna Ron  wrote:
>>>
>>>> 4.3 is still failing.
>>>>
>>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/1172/
>>>> Martin, can you apply the change to 4.3 tests as well?
>>>>
>>>
>>> Arghhh.
>>>
>>> I looked if master and 4.3 shares the same file when I started to work
>>> on the fix and they were the same.
>>> But at the same day I posted orignal patch, master and 4.3 were split :-(
>>>
>>> So hopefully here's the final fix: https://gerrit.ovirt.org/100809
>>>
>>
> Just merged the patch, manual OST verification was successful
>
>>
>>>
>>>> On Thu, Jun 13, 2019 at 10:33 AM Dafna Ron  wrote:
>>>>
>>>>> We have a passing build on master
>>>>> waiting for 4.3
>>>>>
>>>>> On Thu, Jun 13, 2019 at 9:01 AM Dafna Ron  wrote:
>>>>>
>>>>>> Thanks Shirly and Martin.
>>>>>> I see the patch was merged so monitoring and will update
>>>>>>
>>>>>> On Wed, Jun 12, 2019 at 4:09 PM Martin Perina 
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 12, 2019 at 11:53 AM Dafna Ron  wrote:
>>>>>>>
>>>>>>>> latest tests are still failing on log collector
>>>>>>>>
>>>>>>>
>>>>>>> I had a typo in my patch, here is the fix:
>>>>>>> https://gerrit.ovirt.org/#/c/100784/
>>>>>>> Currently trying to verify using manual OST to make sure latest VDSM
>>>>>>> is used (and not latest tested as in check-patch executed from the fix) 
>>>>>>> ...
>>>>>>>
>>>>>>>
>>>>>>>> On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron  wrote:
>>>>>>>>
>>>>>>>>> thanks Martin.
>>>>>>>>> we are failing on missing packages so I have a patch to fix it.
>>>>>>>>> I will update once we have a vdsm build
>>>>>>>>>
>>>>>>>>> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina 
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina 
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron 
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Galit's patch should have solved it.
>>>>>>>>>>>> Marcin, are you still failing?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I've just rebased https://gerrit.ovirt.org/100716 on top of
>>>>>>>>>>> Galit's change so we should know within an hour or so ...
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> CI finished successfully, please review so we can merge ...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk <
>>>>>>>>>>>>> msobc...@redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>&

[ovirt-devel] Re: Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

and another:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14559/

On Fri, Jun 14, 2019 at 9:05 AM Dafna Ron  wrote:

> here you go:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14561/
> I pressed the save build forever button. please press the don't keep
> button once you get the info from the failure.
>
> Thanks,
> Dafna
>
>
> On Wed, Jun 5, 2019 at 9:09 AM Galit Rosenthal 
> wrote:
>
>> We didn't make any changes.
>> I don't think retrie will help in this case
>> we are trying to catch what cause this.
>>
>> If you see this again please let us know.
>>
>> We are still debugging it
>>
>> On Wed, Jun 5, 2019 at 10:56 AM Dafna Ron  wrote:
>>
>>> Hi,
>>>
>>> This is a random failure - so no, I have not.
>>> However, I looked at several failures and they are all the same, the
>>> action on engine/vdsm side succeeds and lago repots a failure but prints a
>>> success from logs.
>>> Did you add anything more to the tests to allow better debugging?
>>> Did you add a re-try to the test?
>>>
>>> Thanks,
>>> Dafna
>>>
>>>
>>>
>>> On Wed, Jun 5, 2019 at 8:21 AM Galit Rosenthal 
>>> wrote:
>>>
>>>> Hi Dafna
>>>>
>>>> If you see this failure again, please send us the link to the job.
>>>>
>>>> We are trying to reproduce it and find the root cause.
>>>>
>>>> Regards,
>>>> Galit
>>>>
>>>> On Tue, May 21, 2019 at 5:15 PM Dafna Ron  wrote:
>>>>
>>>>> sure
>>>>>
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/
>>>>>
>>>>>
>>>>> On Tue, May 21, 2019 at 3:04 PM Shani Leviim 
>>>>> wrote:
>>>>>
>>>>>> Hi Dafna,
>>>>>> Can you please direct me to the full test's log?
>>>>>>
>>>>>>
>>>>>> *Regards,*
>>>>>>
>>>>>> *Shani Leviim*
>>>>>>
>>>>>>
>>>>>> On Tue, May 21, 2019 at 11:57 AM Tal Nisan  wrote:
>>>>>>
>>>>>>> Sure, Shani can you please have a look?
>>>>>>>
>>>>>>> On Mon, May 20, 2019 at 8:30 PM Dafna Ron  wrote:
>>>>>>>
>>>>>>>> Hi Tal,
>>>>>>>>
>>>>>>>> I am seeing random failures for test
>>>>>>>> 002_bootstrap.resize_and_refresh_storage_domain
>>>>>>>> It looks like this is a timing issue since by the time we print out
>>>>>>>> the error we actually see the resize succeeded. see example below:
>>>>>>>>
>>>>>>>>
>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/testReport/junit/(root)/002_bootstrap/running_tests___basic_suite_el7_x86_64___resize_and_refresh_storage_domain/
>>>>>>>>
>>>>>>>> Can you please assign someone from the storage team to fix this
>>>>>>>> test?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Dafna
>>>>>>>>
>>>>>>>> ___
>>>>> Devel mailing list -- devel@ovirt.org
>>>>> To unsubscribe send an email to devel-le...@ovirt.org
>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>>> oVirt Code of Conduct:
>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P4UZJ4PWELVW2AGLVIFXKM5OTMHUHKGC/
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> GALIT ROSENTHAL
>>>>
>>>> SOFTWARE ENGINEER
>>>>
>>>> Red Hat
>>>>
>>>> <https://www.redhat.com/>
>>>>
>>>> ga...@redhat.comT: 972-9-7692230
>>>> <https://red.ht/sig>
>>>>
>>>
>>
>> --
>>
>> GALIT ROSENTHAL
>>
>> SOFTWARE ENGINEER
>>
>> Red Hat
>>
>> <https://www.redhat.com/>
>>
>> ga...@redhat.comT: 972-9-7692230
>> <https://red.ht/sig>
>>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/H2CWSHM5CZQJTTOFFQVSHJCIQHONUEI4/

[ovirt-devel] Re: Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

here you go:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14561/
I pressed the save build forever button. please press the don't keep button
once you get the info from the failure.

Thanks,
Dafna


On Wed, Jun 5, 2019 at 9:09 AM Galit Rosenthal  wrote:

> We didn't make any changes.
> I don't think retrie will help in this case
> we are trying to catch what cause this.
>
> If you see this again please let us know.
>
> We are still debugging it
>
> On Wed, Jun 5, 2019 at 10:56 AM Dafna Ron  wrote:
>
>> Hi,
>>
>> This is a random failure - so no, I have not.
>> However, I looked at several failures and they are all the same, the
>> action on engine/vdsm side succeeds and lago repots a failure but prints a
>> success from logs.
>> Did you add anything more to the tests to allow better debugging?
>> Did you add a re-try to the test?
>>
>> Thanks,
>> Dafna
>>
>>
>>
>> On Wed, Jun 5, 2019 at 8:21 AM Galit Rosenthal 
>> wrote:
>>
>>> Hi Dafna
>>>
>>> If you see this failure again, please send us the link to the job.
>>>
>>> We are trying to reproduce it and find the root cause.
>>>
>>> Regards,
>>> Galit
>>>
>>> On Tue, May 21, 2019 at 5:15 PM Dafna Ron  wrote:
>>>
>>>> sure
>>>>
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/
>>>>
>>>>
>>>> On Tue, May 21, 2019 at 3:04 PM Shani Leviim 
>>>> wrote:
>>>>
>>>>> Hi Dafna,
>>>>> Can you please direct me to the full test's log?
>>>>>
>>>>>
>>>>> *Regards,*
>>>>>
>>>>> *Shani Leviim*
>>>>>
>>>>>
>>>>> On Tue, May 21, 2019 at 11:57 AM Tal Nisan  wrote:
>>>>>
>>>>>> Sure, Shani can you please have a look?
>>>>>>
>>>>>> On Mon, May 20, 2019 at 8:30 PM Dafna Ron  wrote:
>>>>>>
>>>>>>> Hi Tal,
>>>>>>>
>>>>>>> I am seeing random failures for test
>>>>>>> 002_bootstrap.resize_and_refresh_storage_domain
>>>>>>> It looks like this is a timing issue since by the time we print out
>>>>>>> the error we actually see the resize succeeded. see example below:
>>>>>>>
>>>>>>>
>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/testReport/junit/(root)/002_bootstrap/running_tests___basic_suite_el7_x86_64___resize_and_refresh_storage_domain/
>>>>>>>
>>>>>>> Can you please assign someone from the storage team to fix this
>>>>>>> test?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dafna
>>>>>>>
>>>>>>> ___
>>>> Devel mailing list -- devel@ovirt.org
>>>> To unsubscribe send an email to devel-le...@ovirt.org
>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>> oVirt Code of Conduct:
>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>> List Archives:
>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P4UZJ4PWELVW2AGLVIFXKM5OTMHUHKGC/
>>>>
>>>
>>>
>>> --
>>>
>>> GALIT ROSENTHAL
>>>
>>> SOFTWARE ENGINEER
>>>
>>> Red Hat
>>>
>>> <https://www.redhat.com/>
>>>
>>> ga...@redhat.comT: 972-9-7692230
>>> <https://red.ht/sig>
>>>
>>
>
> --
>
> GALIT ROSENTHAL
>
> SOFTWARE ENGINEER
>
> Red Hat
>
> <https://www.redhat.com/>
>
> ga...@redhat.comT: 972-9-7692230
> <https://red.ht/sig>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/APZMFLVSCAK2SFMCSJWD3BIXQDY5EYPX/

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

will monitor new change once merged :)

On Thu, Jun 13, 2019 at 1:23 PM Martin Perina  wrote:

>
>
> On Thu, Jun 13, 2019 at 2:02 PM Dafna Ron  wrote:
>
>> 4.3 is still failing.
>>
>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/1172/
>> Martin, can you apply the change to 4.3 tests as well?
>>
>
> Arghhh.
>
> I looked if master and 4.3 shares the same file when I started to work on
> the fix and they were the same.
> But at the same day I posted orignal patch, master and 4.3 were split :-(
>
> So hopefully here's the final fix: https://gerrit.ovirt.org/100809
>
>
>> On Thu, Jun 13, 2019 at 10:33 AM Dafna Ron  wrote:
>>
>>> We have a passing build on master
>>> waiting for 4.3
>>>
>>> On Thu, Jun 13, 2019 at 9:01 AM Dafna Ron  wrote:
>>>
>>>> Thanks Shirly and Martin.
>>>> I see the patch was merged so monitoring and will update
>>>>
>>>> On Wed, Jun 12, 2019 at 4:09 PM Martin Perina 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 12, 2019 at 11:53 AM Dafna Ron  wrote:
>>>>>
>>>>>> latest tests are still failing on log collector
>>>>>>
>>>>>
>>>>> I had a typo in my patch, here is the fix:
>>>>> https://gerrit.ovirt.org/#/c/100784/
>>>>> Currently trying to verify using manual OST to make sure latest VDSM
>>>>> is used (and not latest tested as in check-patch executed from the fix) 
>>>>> ...
>>>>>
>>>>>
>>>>>> On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron  wrote:
>>>>>>
>>>>>>> thanks Martin.
>>>>>>> we are failing on missing packages so I have a patch to fix it.
>>>>>>> I will update once we have a vdsm build
>>>>>>>
>>>>>>> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina 
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron  wrote:
>>>>>>>>>
>>>>>>>>>> Galit's patch should have solved it.
>>>>>>>>>> Marcin, are you still failing?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I've just rebased https://gerrit.ovirt.org/100716 on top of
>>>>>>>>> Galit's change so we should know within an hour or so ...
>>>>>>>>>
>>>>>>>>
>>>>>>>> CI finished successfully, please review so we can merge ...
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri 
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk <
>>>>>>>>>>> msobc...@redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> basic suite fails for the patch with the following log:
>>>>>>>>>>>>
>>>>>>>>>>>>  
>>>>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-769>[2019-06-11T11:34:26.175Z]
>>>>>>>>>>>>- STDERR
>>>>>>>>>>>>  
>>>>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-770>[2019-06-11T11:34:26.175Z]
>>>>>>>>>>>>  + yum -y install ovirt-host
>>>>>>>>>>>>  
>>>>>&g

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

4.3 is still failing.
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/1172/
Martin, can you apply the change to 4.3 tests as well?

On Thu, Jun 13, 2019 at 10:33 AM Dafna Ron  wrote:

> We have a passing build on master
> waiting for 4.3
>
> On Thu, Jun 13, 2019 at 9:01 AM Dafna Ron  wrote:
>
>> Thanks Shirly and Martin.
>> I see the patch was merged so monitoring and will update
>>
>> On Wed, Jun 12, 2019 at 4:09 PM Martin Perina  wrote:
>>
>>>
>>>
>>> On Wed, Jun 12, 2019 at 11:53 AM Dafna Ron  wrote:
>>>
>>>> latest tests are still failing on log collector
>>>>
>>>
>>> I had a typo in my patch, here is the fix:
>>> https://gerrit.ovirt.org/#/c/100784/
>>> Currently trying to verify using manual OST to make sure latest VDSM is
>>> used (and not latest tested as in check-patch executed from the fix) ...
>>>
>>>
>>>> On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron  wrote:
>>>>
>>>>> thanks Martin.
>>>>> we are failing on missing packages so I have a patch to fix it.
>>>>> I will update once we have a vdsm build
>>>>>
>>>>> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina 
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina 
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron  wrote:
>>>>>>>
>>>>>>>> Galit's patch should have solved it.
>>>>>>>> Marcin, are you still failing?
>>>>>>>>
>>>>>>>
>>>>>>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's
>>>>>>> change so we should know within an hour or so ...
>>>>>>>
>>>>>>
>>>>>> CI finished successfully, please review so we can merge ...
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri  wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk <
>>>>>>>>> msobc...@redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> basic suite fails for the patch with the following log:
>>>>>>>>>>
>>>>>>>>>>  
>>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-769>[2019-06-11T11:34:26.175Z]
>>>>>>>>>>- STDERR
>>>>>>>>>>  
>>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-770>[2019-06-11T11:34:26.175Z]
>>>>>>>>>>  + yum -y install ovirt-host
>>>>>>>>>>  
>>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-771>[2019-06-11T11:34:26.175Z]
>>>>>>>>>>  Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync)
>>>>>>>>>>  
>>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-772>[2019-06-11T11:34:26.175Z]
>>>>>>>>>> Requires: bind-license = 32:9.9.4-74.el7_6.1
>>>>>>>>>>  
>>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-773>[2019-06-11T11:34:26.175Z]
>>>>>>>>>> Installed: 32:bind-license-9.9.4-73.el7_6.noarch 
>>>>>>>>>> (installed)
>>>>&g

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

We have a passing build on master
waiting for 4.3

On Thu, Jun 13, 2019 at 9:01 AM Dafna Ron  wrote:

> Thanks Shirly and Martin.
> I see the patch was merged so monitoring and will update
>
> On Wed, Jun 12, 2019 at 4:09 PM Martin Perina  wrote:
>
>>
>>
>> On Wed, Jun 12, 2019 at 11:53 AM Dafna Ron  wrote:
>>
>>> latest tests are still failing on log collector
>>>
>>
>> I had a typo in my patch, here is the fix:
>> https://gerrit.ovirt.org/#/c/100784/
>> Currently trying to verify using manual OST to make sure latest VDSM is
>> used (and not latest tested as in check-patch executed from the fix) ...
>>
>>
>>> On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron  wrote:
>>>
>>>> thanks Martin.
>>>> we are failing on missing packages so I have a patch to fix it.
>>>> I will update once we have a vdsm build
>>>>
>>>> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina 
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron  wrote:
>>>>>>
>>>>>>> Galit's patch should have solved it.
>>>>>>> Marcin, are you still failing?
>>>>>>>
>>>>>>
>>>>>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's
>>>>>> change so we should know within an hour or so ...
>>>>>>
>>>>>
>>>>> CI finished successfully, please review so we can merge ...
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri  wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> basic suite fails for the patch with the following log:
>>>>>>>>>
>>>>>>>>>  
>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-769>[2019-06-11T11:34:26.175Z]
>>>>>>>>>- STDERR
>>>>>>>>>  
>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-770>[2019-06-11T11:34:26.175Z]
>>>>>>>>>  + yum -y install ovirt-host
>>>>>>>>>  
>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-771>[2019-06-11T11:34:26.175Z]
>>>>>>>>>  Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync)
>>>>>>>>>  
>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-772>[2019-06-11T11:34:26.175Z]
>>>>>>>>> Requires: bind-license = 32:9.9.4-74.el7_6.1
>>>>>>>>>  
>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-773>[2019-06-11T11:34:26.175Z]
>>>>>>>>> Installed: 32:bind-license-9.9.4-73.el7_6.noarch 
>>>>>>>>> (installed)
>>>>>>>>>  
>>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-774>[2019-06-11T11:34:26.175Z]
>>>>>>>>> bind-license = 32:9.9.4-73.el7_6
>>>>>>>>>
>>>>>>>>>
>>>>>>>> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2
>>>>>>>> hours ago should address this issu

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

Thanks Shirly and Martin.
I see the patch was merged so monitoring and will update

On Wed, Jun 12, 2019 at 4:09 PM Martin Perina  wrote:

>
>
> On Wed, Jun 12, 2019 at 11:53 AM Dafna Ron  wrote:
>
>> latest tests are still failing on log collector
>>
>
> I had a typo in my patch, here is the fix:
> https://gerrit.ovirt.org/#/c/100784/
> Currently trying to verify using manual OST to make sure latest VDSM is
> used (and not latest tested as in check-patch executed from the fix) ...
>
>
>> On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron  wrote:
>>
>>> thanks Martin.
>>> we are failing on missing packages so I have a patch to fix it.
>>> I will update once we have a vdsm build
>>>
>>> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina 
>>> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron  wrote:
>>>>>
>>>>>> Galit's patch should have solved it.
>>>>>> Marcin, are you still failing?
>>>>>>
>>>>>
>>>>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's
>>>>> change so we should know within an hour or so ...
>>>>>
>>>>
>>>> CI finished successfully, please review so we can merge ...
>>>>
>>>>
>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri  wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> basic suite fails for the patch with the following log:
>>>>>>>>
>>>>>>>>  
>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-769>[2019-06-11T11:34:26.175Z]
>>>>>>>>- STDERR
>>>>>>>>  
>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-770>[2019-06-11T11:34:26.175Z]
>>>>>>>>  + yum -y install ovirt-host
>>>>>>>>  
>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-771>[2019-06-11T11:34:26.175Z]
>>>>>>>>  Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync)
>>>>>>>>  
>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-772>[2019-06-11T11:34:26.175Z]
>>>>>>>> Requires: bind-license = 32:9.9.4-74.el7_6.1
>>>>>>>>  
>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-773>[2019-06-11T11:34:26.175Z]
>>>>>>>> Installed: 32:bind-license-9.9.4-73.el7_6.noarch 
>>>>>>>> (installed)
>>>>>>>>  
>>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-774>[2019-06-11T11:34:26.175Z]
>>>>>>>> bind-license = 32:9.9.4-73.el7_6
>>>>>>>>
>>>>>>>>
>>>>>>> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2
>>>>>>> hours ago should address this issue.
>>>>>>> Maybe other reposync files should be updated as well?
>>>>>>>
>>>>>>>
>>>>>>>>  Seems unrelated and I'm having the same issue locally on my server.
>>>>>>>> Dafna, do you have some insight regarding this dependency error?
>>>>>>>>
>>>>>>>> Thanks, Marcin
>>>

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

latest tests are still failing on log collector

On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron  wrote:

> thanks Martin.
> we are failing on missing packages so I have a patch to fix it.
> I will update once we have a vdsm build
>
> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina  wrote:
>
>>
>>
>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina  wrote:
>>
>>>
>>>
>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron  wrote:
>>>
>>>> Galit's patch should have solved it.
>>>> Marcin, are you still failing?
>>>>
>>>
>>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's
>>> change so we should know within an hour or so ...
>>>
>>
>> CI finished successfully, please review so we can merge ...
>>
>>
>>>
>>>>
>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri  wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk 
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> basic suite fails for the patch with the following log:
>>>>>>
>>>>>>  
>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-769>[2019-06-11T11:34:26.175Z]
>>>>>>- STDERR
>>>>>>  
>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-770>[2019-06-11T11:34:26.175Z]
>>>>>>  + yum -y install ovirt-host
>>>>>>  
>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-771>[2019-06-11T11:34:26.175Z]
>>>>>>  Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync)
>>>>>>  
>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-772>[2019-06-11T11:34:26.175Z]
>>>>>> Requires: bind-license = 32:9.9.4-74.el7_6.1
>>>>>>  
>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-773>[2019-06-11T11:34:26.175Z]
>>>>>> Installed: 32:bind-license-9.9.4-73.el7_6.noarch (installed)
>>>>>>  
>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-774>[2019-06-11T11:34:26.175Z]
>>>>>> bind-license = 32:9.9.4-73.el7_6
>>>>>>
>>>>>>
>>>>> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2 hours
>>>>> ago should address this issue.
>>>>> Maybe other reposync files should be updated as well?
>>>>>
>>>>>
>>>>>>  Seems unrelated and I'm having the same issue locally on my server.
>>>>>> Dafna, do you have some insight regarding this dependency error?
>>>>>>
>>>>>> Thanks, Marcin
>>>>>> On 6/11/19 1:15 PM, Martin Perina wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2019 at 11:58 AM Milan Zamazal 
>>>>>> wrote:
>>>>>>
>>>>>>> Dafna Ron  writes:
>>>>>>>
>>>>>>> > Hi,
>>>>>>> >
>>>>>>> > Please note vdsm has been broken since Fri the 7th
>>>>>>> >
>>>>>>> > to summarize again,  vdsm has a patch to remove sos plugin which
>>>>>>> is what
>>>>>>> > metrics is using in its ost tests
>>>>>>> > due to that, vdsm is failing the metrics tests and in order to
>>>>>>> solve it we
>>>>>>> > need to make a choice:
>>>>>>> > 1. fix the metrics tests to not use sos
>>>>>>> > 2. disable the metrics tests
>>>>>>> > 3. re

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

thanks Martin.
we are failing on missing packages so I have a patch to fix it.
I will update once we have a vdsm build

On Tue, Jun 11, 2019 at 10:08 PM Martin Perina  wrote:

>
>
> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina  wrote:
>
>>
>>
>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron  wrote:
>>
>>> Galit's patch should have solved it.
>>> Marcin, are you still failing?
>>>
>>
>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's
>> change so we should know within an hour or so ...
>>
>
> CI finished successfully, please review so we can merge ...
>
>
>>
>>>
>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri  wrote:
>>>
>>>>
>>>>
>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk 
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> basic suite fails for the patch with the following log:
>>>>>
>>>>>  
>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-769>[2019-06-11T11:34:26.175Z]
>>>>>- STDERR
>>>>>  
>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-770>[2019-06-11T11:34:26.175Z]
>>>>>  + yum -y install ovirt-host
>>>>>  
>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-771>[2019-06-11T11:34:26.175Z]
>>>>>  Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync)
>>>>>  
>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-772>[2019-06-11T11:34:26.175Z]
>>>>> Requires: bind-license = 32:9.9.4-74.el7_6.1
>>>>>  
>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-773>[2019-06-11T11:34:26.175Z]
>>>>> Installed: 32:bind-license-9.9.4-73.el7_6.noarch (installed)
>>>>>  
>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-774>[2019-06-11T11:34:26.175Z]
>>>>>         bind-license = 32:9.9.4-73.el7_6
>>>>>
>>>>>
>>>> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2 hours
>>>> ago should address this issue.
>>>> Maybe other reposync files should be updated as well?
>>>>
>>>>
>>>>>  Seems unrelated and I'm having the same issue locally on my server.
>>>>> Dafna, do you have some insight regarding this dependency error?
>>>>>
>>>>> Thanks, Marcin
>>>>> On 6/11/19 1:15 PM, Martin Perina wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 11, 2019 at 11:58 AM Milan Zamazal 
>>>>> wrote:
>>>>>
>>>>>> Dafna Ron  writes:
>>>>>>
>>>>>> > Hi,
>>>>>> >
>>>>>> > Please note vdsm has been broken since Fri the 7th
>>>>>> >
>>>>>> > to summarize again,  vdsm has a patch to remove sos plugin which is
>>>>>> what
>>>>>> > metrics is using in its ost tests
>>>>>> > due to that, vdsm is failing the metrics tests and in order to
>>>>>> solve it we
>>>>>> > need to make a choice:
>>>>>> > 1. fix the metrics tests to not use sos
>>>>>> > 2. disable the metrics tests
>>>>>> > 3. revert the sos patch until a decision is made on ^^
>>>>>>
>>>>>> #3 is not an option, it would make Vdsm uninstallable on newer RHEL
>>>>>> versions.
>>>>>>
>>>>>
>>>>> I've posted a patch https://gerrit.ovirt.org/100716 which is trying
>>>>> to install vdsm sos plugin if it's not installed either by vdsm nor sos.
>>>>> Currenlt

[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

Galit's patch should have solved it.
Marcin, are you still failing?


On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri  wrote:

>
>
> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk 
> wrote:
>
>> Hi,
>>
>> basic suite fails for the patch with the following log:
>>
>>  
>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-769>[2019-06-11T11:34:26.175Z]
>>- STDERR
>>  
>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-770>[2019-06-11T11:34:26.175Z]
>>  + yum -y install ovirt-host
>>  
>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-771>[2019-06-11T11:34:26.175Z]
>>  Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync)
>>  
>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-772>[2019-06-11T11:34:26.175Z]
>> Requires: bind-license = 32:9.9.4-74.el7_6.1
>>  
>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-773>[2019-06-11T11:34:26.175Z]
>> Installed: 32:bind-license-9.9.4-73.el7_6.noarch (installed)
>>  
>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_standard-check-patch/detail/ovirt-system-tests_standard-check-patch/4762/pipeline/107#step-188-log-774>[2019-06-11T11:34:26.175Z]
>> bind-license = 32:9.9.4-73.el7_6
>>
>>
> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2 hours ago
> should address this issue.
> Maybe other reposync files should be updated as well?
>
>
>>  Seems unrelated and I'm having the same issue locally on my server.
>> Dafna, do you have some insight regarding this dependency error?
>>
>> Thanks, Marcin
>> On 6/11/19 1:15 PM, Martin Perina wrote:
>>
>>
>>
>> On Tue, Jun 11, 2019 at 11:58 AM Milan Zamazal 
>> wrote:
>>
>>> Dafna Ron  writes:
>>>
>>> > Hi,
>>> >
>>> > Please note vdsm has been broken since Fri the 7th
>>> >
>>> > to summarize again,  vdsm has a patch to remove sos plugin which is
>>> what
>>> > metrics is using in its ost tests
>>> > due to that, vdsm is failing the metrics tests and in order to solve
>>> it we
>>> > need to make a choice:
>>> > 1. fix the metrics tests to not use sos
>>> > 2. disable the metrics tests
>>> > 3. revert the sos patch until a decision is made on ^^
>>>
>>> #3 is not an option, it would make Vdsm uninstallable on newer RHEL
>>> versions.
>>>
>>
>> I've posted a patch https://gerrit.ovirt.org/100716 which is trying to
>> install vdsm sos plugin if it's not installed either by vdsm nor sos.
>> Currenlty waiting for CI, if run is successfull, I will extend the patch
>> also for 4.3 basic suite.
>>
>>
>>> > Thanks,
>>> > Dafna
>>> >
>>> >
>>> > -- Forwarded message -
>>> > From: Dafna Ron 
>>> > Date: Mon, Jun 10, 2019 at 1:30 PM
>>> > Subject: Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [
>>> 07-06-2019 ]
>>> > [ 003_00_metrics_bootstrap.metrics_and_log_collector ]
>>> > To: Martin Perina , Milan Zamazal <
>>> mzama...@redhat.com>,
>>> > Shirly Radco 
>>> > Cc: devel , infra 
>>> >
>>> >
>>> > Shirly? any update on this?
>>> >
>>> > On Fri, Jun 7, 2019 at 11:54 AM Dafna Ron  wrote:
>>> >
>>> >> Hi,
>>> >>
>>> >> We have a failure in vdsm project on master.
>>> >>
>>> >> The issue is change:
>>> >> https://gerrit.ovirt.org/#/c/100576/ - Remove SOS VDSM plugin
>>> >>
>>> >> which is failing on metrics as metrics is calling sos-logcollector.
>>> >>
>>> >> The patch cannot be changed as until centos 7.7 when sos-3.7-3, which
>>> >> contains vdsm plugin will come out.
>>> >> so until then, w

[ovirt-devel] Re: vdsm has been tagged (v4.30.19)

[Dafna Ron]

note that vdsm has been failing in CQ since Friday and have not passed CQ.


On Tue, Jun 11, 2019 at 10:44 AM Milan Zamazal  wrote:

>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5FSD24HBSB6N7CL3UIXIZSA5HUCVDOA7/

[ovirt-devel] [urgent] vdsm broken since Friday (failing CQ)

[Dafna Ron]

Hi,

Please note vdsm has been broken since Fri the 7th

to summarize again,  vdsm has a patch to remove sos plugin which is what
metrics is using in its ost tests
due to that, vdsm is failing the metrics tests and in order to solve it we
need to make a choice:
1. fix the metrics tests to not use sos
2. disable the metrics tests
3. revert the sos patch until a decision is made on ^^

Thanks,
Dafna


-- Forwarded message -
From: Dafna Ron 
Date: Mon, Jun 10, 2019 at 1:30 PM
Subject: Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 07-06-2019 ]
[ 003_00_metrics_bootstrap.metrics_and_log_collector ]
To: Martin Perina , Milan Zamazal ,
Shirly Radco 
Cc: devel , infra 


Shirly? any update on this?

On Fri, Jun 7, 2019 at 11:54 AM Dafna Ron  wrote:

> Hi,
>
> We have a failure in vdsm project on master.
>
> The issue is change:
> https://gerrit.ovirt.org/#/c/100576/ - Remove SOS VDSM plugin
>
> which is failing on metrics as metrics is calling sos-logcollector.
>
> The patch cannot be changed as until centos 7.7 when sos-3.7-3, which
> contains vdsm plugin will come out.
> so until then, we are left with no sos plugin, which is causing the
> metrics test to fail.
>
> Shirly, can you please take a look and see if we can change the test to
> not call sos-logcollector?
> Please note, that we are expecting 4.3 to fail on same issue very soon.
>
> failed job can be found here:
>
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14452/
>
>
> ERROR from test:
>
> lago.ssh: DEBUG: Command 8626fe70 on lago-basic-suite-master-engine  errors:
>  ERROR: Failed to get a sosreport from: lago-basic-suite-master-host-1; Could 
> not parse sosreport output
> ERROR: Failed to get a sosreport from: lago-basic-suite-master-host-0; Could 
> not parse sosreport output
>
> lago.utils: DEBUG: Error while running thread Thread-3
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in 
> _ret_via_queue
> queue.put({'return': func()})
>   File 
> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/003_00_metrics_bootstrap.py",
>  line 97, in run_log_collector
> 'log collector failed. Exit code is %s' % result.code
>   File "/usr/lib64/python2.7/unittest/case.py", line 462, in assertTrue
> raise self.failureException(msg)
> AssertionError: log collector failed. Exit code is 1
> - >> end captured logging << --
>
> Thanks,
> Dafna
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/E2UBWXYAE73XJ6I75N4ESXUFDIVMSIIS/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 07-06-2019 ] [ 003_00_metrics_bootstrap.metrics_and_log_collector ]

[Dafna Ron]

Shirly? any update on this?

On Fri, Jun 7, 2019 at 11:54 AM Dafna Ron  wrote:

> Hi,
>
> We have a failure in vdsm project on master.
>
> The issue is change:
> https://gerrit.ovirt.org/#/c/100576/ - Remove SOS VDSM plugin
>
> which is failing on metrics as metrics is calling sos-logcollector.
>
> The patch cannot be changed as until centos 7.7 when sos-3.7-3, which
> contains vdsm plugin will come out.
> so until then, we are left with no sos plugin, which is causing the
> metrics test to fail.
>
> Shirly, can you please take a look and see if we can change the test to
> not call sos-logcollector?
> Please note, that we are expecting 4.3 to fail on same issue very soon.
>
> failed job can be found here:
>
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14452/
>
>
> ERROR from test:
>
> lago.ssh: DEBUG: Command 8626fe70 on lago-basic-suite-master-engine  errors:
>  ERROR: Failed to get a sosreport from: lago-basic-suite-master-host-1; Could 
> not parse sosreport output
> ERROR: Failed to get a sosreport from: lago-basic-suite-master-host-0; Could 
> not parse sosreport output
>
> lago.utils: DEBUG: Error while running thread Thread-3
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in 
> _ret_via_queue
> queue.put({'return': func()})
>   File 
> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/003_00_metrics_bootstrap.py",
>  line 97, in run_log_collector
> 'log collector failed. Exit code is %s' % result.code
>   File "/usr/lib64/python2.7/unittest/case.py", line 462, in assertTrue
> raise self.failureException(msg)
> AssertionError: log collector failed. Exit code is 1
> - >> end captured logging << --
>
> Thanks,
> Dafna
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/OTRK36TZ25KWDPSMSTTE3SX2T7VUJVCE/

[ovirt-devel] [Ovirt] [CQ weekly status] [07-06-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last failure was on 05-06 for project ovirt-ansible-cluster-upgrade due to
failure in test 002_bootstrap.add_master_storage_domain.
The fix was created and merged 

*CQ-4.3*:  RED (#1)

vdsm is currently broken due to https://gerrit.ovirt.org/#/c/100576/ -
Remove SOS VDSM plugin
vdsm sos plugin will be available from centos 7.7 as part of sos-3.7-3.
however, the metrics tests are using sos-logcollector which s causing a
failure in ost.
We are waiting for Shirly to have a look at the tests and see how they can
be fixed.

*CQ-Master:*  RED (#1)

vdsm is currently broken due to https://gerrit.ovirt.org/#/c/100576/ -
Remove SOS VDSM plugin
vdsm sos plugin will be available from centos 7.7 as part of sos-3.7-3.
however, the metrics tests are using sos-logcollector which s causing a
failure in ost.
We are waiting for Shirly to have a look at the tests and see how they can
be fixed.

 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QCT52YUDJANNZMTIRE5I23R32M56YR7C/

[ovirt-devel] [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 07-06-2019 ] [ 003_00_metrics_bootstrap.metrics_and_log_collector ]

[Dafna Ron]

Hi,

We have a failure in vdsm project on master.

The issue is change:
https://gerrit.ovirt.org/#/c/100576/ - Remove SOS VDSM plugin

which is failing on metrics as metrics is calling sos-logcollector.

The patch cannot be changed as until centos 7.7 when sos-3.7-3, which
contains vdsm plugin will come out.
so until then, we are left with no sos plugin, which is causing the metrics
test to fail.

Shirly, can you please take a look and see if we can change the test to not
call sos-logcollector?
Please note, that we are expecting 4.3 to fail on same issue very soon.

failed job can be found here:

https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14452/


ERROR from test:

lago.ssh: DEBUG: Command 8626fe70 on lago-basic-suite-master-engine  errors:
 ERROR: Failed to get a sosreport from:
lago-basic-suite-master-host-1; Could not parse sosreport output
ERROR: Failed to get a sosreport from: lago-basic-suite-master-host-0;
Could not parse sosreport output

lago.utils: DEBUG: Error while running thread Thread-3
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in
_ret_via_queue
queue.put({'return': func()})
  File 
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/003_00_metrics_bootstrap.py",
line 97, in run_log_collector
'log collector failed. Exit code is %s' % result.code
  File "/usr/lib64/python2.7/unittest/case.py", line 462, in assertTrue
raise self.failureException(msg)
AssertionError: log collector failed. Exit code is 1
- >> end captured logging << --

Thanks,
Dafna
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/AGIX5H4EJQ66FULFJ6UL5OEDHZTLVMLO/

[ovirt-devel] Re: Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

Hi,

This is a random failure - so no, I have not.
However, I looked at several failures and they are all the same, the action
on engine/vdsm side succeeds and lago repots a failure but prints a success
from logs.
Did you add anything more to the tests to allow better debugging?
Did you add a re-try to the test?

Thanks,
Dafna



On Wed, Jun 5, 2019 at 8:21 AM Galit Rosenthal  wrote:

> Hi Dafna
>
> If you see this failure again, please send us the link to the job.
>
> We are trying to reproduce it and find the root cause.
>
> Regards,
> Galit
>
> On Tue, May 21, 2019 at 5:15 PM Dafna Ron  wrote:
>
>> sure
>>
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/
>>
>>
>> On Tue, May 21, 2019 at 3:04 PM Shani Leviim  wrote:
>>
>>> Hi Dafna,
>>> Can you please direct me to the full test's log?
>>>
>>>
>>> *Regards,*
>>>
>>> *Shani Leviim*
>>>
>>>
>>> On Tue, May 21, 2019 at 11:57 AM Tal Nisan  wrote:
>>>
>>>> Sure, Shani can you please have a look?
>>>>
>>>> On Mon, May 20, 2019 at 8:30 PM Dafna Ron  wrote:
>>>>
>>>>> Hi Tal,
>>>>>
>>>>> I am seeing random failures for test
>>>>> 002_bootstrap.resize_and_refresh_storage_domain
>>>>> It looks like this is a timing issue since by the time we print out
>>>>> the error we actually see the resize succeeded. see example below:
>>>>>
>>>>>
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/testReport/junit/(root)/002_bootstrap/running_tests___basic_suite_el7_x86_64___resize_and_refresh_storage_domain/
>>>>>
>>>>> Can you please assign someone from the storage team to fix this test?
>>>>>
>>>>> Thanks,
>>>>> Dafna
>>>>>
>>>>> ___
>> Devel mailing list -- devel@ovirt.org
>> To unsubscribe send an email to devel-le...@ovirt.org
>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P4UZJ4PWELVW2AGLVIFXKM5OTMHUHKGC/
>>
>
>
> --
>
> GALIT ROSENTHAL
>
> SOFTWARE ENGINEER
>
> Red Hat
>
> <https://www.redhat.com/>
>
> ga...@redhat.comT: 972-9-7692230
> <https://red.ht/sig>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PY5PCZWS3ED7PRC7EEHHDEMX23GW4PLE/

[ovirt-devel] Re: Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

sure
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/


On Tue, May 21, 2019 at 3:04 PM Shani Leviim  wrote:

> Hi Dafna,
> Can you please direct me to the full test's log?
>
>
> *Regards,*
>
> *Shani Leviim*
>
>
> On Tue, May 21, 2019 at 11:57 AM Tal Nisan  wrote:
>
>> Sure, Shani can you please have a look?
>>
>> On Mon, May 20, 2019 at 8:30 PM Dafna Ron  wrote:
>>
>>> Hi Tal,
>>>
>>> I am seeing random failures for test
>>> 002_bootstrap.resize_and_refresh_storage_domain
>>> It looks like this is a timing issue since by the time we print out the
>>> error we actually see the resize succeeded. see example below:
>>>
>>>
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/testReport/junit/(root)/002_bootstrap/running_tests___basic_suite_el7_x86_64___resize_and_refresh_storage_domain/
>>>
>>> Can you please assign someone from the storage team to fix this test?
>>>
>>> Thanks,
>>> Dafna
>>>
>>>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P4UZJ4PWELVW2AGLVIFXKM5OTMHUHKGC/

[ovirt-devel] vdsm failing on kernel package in CQ - Fix will be merged soon

[Dafna Ron]

Hi,

vdsm project is failing on 4.3 and master due to a kernel package
dependency [1]

Galit already has a patch and will merge once it passes CI [2]

[1]
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14195/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/lago-upgrade-from-release-suite-master-engine/_var_log/ovirt-engine/host-deploy/ovirt-host-mgmt-ansible-20190521064741-lago-upgrade-from-release-suite-master-host-0-cd7bb16e-b4e6-49fd-ad52-fbce54d240a0.log
[2] https://gerrit.ovirt.org/#/c/100182/


Thanks,
Dafna
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MPIQHWFARGHOIYKIU2WMKPB6GY6NPYVH/

[ovirt-devel] Storage test case needs to be fixed: 002_bootstrap.resize_and_refresh_storage_domain

[Dafna Ron]

Hi Tal,

I am seeing random failures for test
002_bootstrap.resize_and_refresh_storage_domain
It looks like this is a timing issue since by the time we print out the
error we actually see the resize succeeded. see example below:

http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14185/testReport/junit/(root)/002_bootstrap/running_tests___basic_suite_el7_x86_64___resize_and_refresh_storage_domain/

Can you please assign someone from the storage team to fix this test?

Thanks,
Dafna
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/L4RSLYR4M6RJ6VFXHDSB2TR6HW5RCRNU/

[ovirt-devel] [Ovirt] [CQ weekly status] [17-05-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last failure was on 15-5 for project ovirt-ansible-engine-setup due to
failure in test 002_bootstrap.add_master_storage_domain.
The fix for this test was submitted and merged in ost:
https://gerrit.ovirt.org/#/c/100089/

*CQ-4.3*:  GREEN (#1)

Last failure was on 16-05 for project ovirt-host due to a build-artifacts
for fc28.
the issue has been resolved.

*CQ-Master:*  RED (#1)

We have a current failure in project ovirt-engine which started last night.
The failure is in run vm test and caused by change:
https://gerrit.ovirt.org/#/c/99372/6 - core: Initial Ignition support over
custom script 
It already has a fix (thank you Roy for the fast response) and we are
awaiting a merge for the patch:
https://gerrit.ovirt.org/#/c/100133/ - core: handle a case where cloudinit
script is null

 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/354VGI5DB7VSX2GAYDHXWB5ROE3SXB4A/

[ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 17-05-19 ] [ 004_basic_sanity.run_vms ]

[Dafna Ron]

Hi,

We are failing to run vm in project ovirt-engine on master branch on both
basic and upgrade suites

CQ reported this patch as root cause:

https://gerrit.ovirt.org/#/c/99372/6 - core: Initial Ignition support over
custom script 

I can see errors in the log related to cloud-init which happen at the same
time we try to run the vm.
There are also NPE on GetVMLeaseInfo happening before.

Roy can you please take a look?

Full logs from first failure can be found here:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14148/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/


Thanks,
Dafna


Error from log:

2019-05-16 12:53:46,184-04 INFO
 [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-2)
[ad7888f0-4368-4792-b6a8-f7cd4b0ebbe6] START, CreateVDSCommand( CreateVD
SCommandParameters:{hostId='31dd3a99-5821-437b-8995-232cfcd67d84',
vmId='362080c4-4d08-4b40-b6e7-9c04bf854d68', vm='VM [vm0]'}), log id:
522026aa
2019-05-16 12:53:46,190-04 INFO
 [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(default task-2) [ad7888f0-4368-4792-b6a8-f7cd4b0ebbe6] START, CreateBrok
erVDSCommand(HostName = lago-basic-suite-master-host-1,
CreateVDSCommandParameters:{hostId='31dd3a99-5821-437b-8995-232cfcd67d84',
vmId='362080c4-4d08-4b40-b6e7-9c04bf854d68
', vm='VM [vm0]'}), log id: 6bbd62c6
2019-05-16 12:53:46,201-04 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default
task-2) [ad7888f0-4368-4792-b6a8-f7cd4b0ebbe6] Failed in 'CreateBrokerVDS'
method, for vds: 'lago-basic-suite-master-host-1'; host:
'lago-basic-suite-master-host-1': Failed to build cloud-init data:
2019-05-16 12:53:46,201-04 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default
task-2) [ad7888f0-4368-4792-b6a8-f7cd4b0ebbe6] Command
'CreateBrokerVDSCommand(HostName = lago-basic-suite-master-host-1,
CreateVDSCommandParameters:{hostId='31dd3a99-5821-437b-8995-232cfcd67d84',
vmId='362080c4-4d08-4b40-b6e7-9c04bf854d68', vm='VM [vm0]'})' execution
failed: Failed to build cloud-init data:
2019-05-16 12:53:46,201-04 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default
task-2) [ad7888f0-4368-4792-b6a8-f7cd4b0ebbe6] Exception:
java.lang.RuntimeException: Failed to build cloud-init data:
at
org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand.getPayload(CreateBrokerVDSCommand.java:177)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand.generateDomainXml(CreateBrokerVDSCommand.java:90)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand.createInfo(CreateBrokerVDSCommand.java:52)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand.executeVdsBrokerCommand(CreateBrokerVDSCommand.java:44)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
[vdsbroker.jar:]
at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31)
[dal.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:397)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
Source) [vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.CreateVDSCommand.executeVmCommand(CreateVDSCommand.java:37)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ManagingVmCommand.executeVDSCommand(ManagingVmCommand.java:17)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
[vdsbroker.jar:]
at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31)
[dal.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:397)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
Source) [vdsbroker.jar:]
at sun.reflect.GeneratedMethodAccessor253.invoke(Unknown Source)
[:1.8.0_212]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_212]
at

[ovirt-devel] Re: URGENT - ovirt-engine broken for 3 days Re: Subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 05-05-2019 ] [ upgrade_hosts ]

[Dafna Ron]

we have a passing ovirt-engine build today.
Thank you all for a fast response.
Dafna


On Thu, May 9, 2019 at 12:43 PM Sandro Bonazzola 
wrote:

>
>
> Il giorno gio 9 mag 2019 alle ore 12:59 Dafna Ron  ha
> scritto:
>
>> As IL are on independence day, anyone else can merge?
>> https://gerrit.ovirt.org/#/c/99845/
>>
>>
> I have merge rights but I need at least CI to pass. Waiting on jenkins.
>
>
>>
>> On Thu, May 9, 2019 at 11:30 AM Dafna Ron  wrote:
>>
>>> Thanks Andrej.
>>> I will follow the patch and update.
>>> Dafna
>>>
>>> On Thu, May 9, 2019 at 11:23 AM Andrej Krejcir 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Ok, I have posted the reverting patch:
>>>> https://gerrit.ovirt.org/#/c/99845/
>>>>
>>>> I'm still investigating what is the problem. Sorry for the delay, we
>>>> had a public holiday yesturday.
>>>>
>>>>
>>>> Andrej
>>>>
>>>> On Thu, 9 May 2019 at 11:20, Dafna Ron  wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have not heard back on this issue and ovirt-engine has been broken
>>>>> for the past 3 days.
>>>>>
>>>>> As this does not seem a simple debug and fix I suggest reverting the
>>>>> patch and investigating later.
>>>>>
>>>>> thanks,
>>>>> Dafna
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 8, 2019 at 9:42 AM Dafna Ron  wrote:
>>>>>
>>>>>> Any news?
>>>>>>
>>>>>> Thanks,
>>>>>> Dafna
>>>>>>
>>>>>>
>>>>>> On Tue, May 7, 2019 at 4:57 PM Dafna Ron  wrote:
>>>>>>
>>>>>>> thanks for the quick reply and investigation.
>>>>>>> Please update me if I can help any further and if you find the cause
>>>>>>> and have a patch let me know.
>>>>>>> Note that ovirt-engine project is broken and if we cannot find the
>>>>>>> cause relatively fast we should consider reverting the patch to allow a 
>>>>>>> new
>>>>>>> package to be built in CQ with other changes that were submitted.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dafna
>>>>>>>
>>>>>>>
>>>>>>> On Tue, May 7, 2019 at 4:42 PM Andrej Krejcir 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> After running a few OSTs manually, it seems that the patch is the
>>>>>>>> cause. Investigating...
>>>>>>>>
>>>>>>>> On Tue, 7 May 2019 at 14:58, Andrej Krejcir 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> The issue is probably not caused by the patch.
>>>>>>>>>
>>>>>>>>> This log line means that the VM does not exist in the DB:
>>>>>>>>>
>>>>>>>>> 2019-05-07 06:02:04,215-04 WARN
>>>>>>>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>>>>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] 
>>>>>>>>> Validation
>>>>>>>>> of action 'MigrateMultipleVms' failed for user admin@internal-authz.
>>>>>>>>> Reasons: ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>>>>>>>>
>>>>>>>>> I will investigate more, why the VM is missing.
>>>>>>>>>
>>>>>>>>> On Tue, 7 May 2019 at 14:07, Dafna Ron  wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> We are failing test upgrade_hosts on
>>>>>>>>>> upgrade-from-release-suite-master.
>>>>>>>>>> From the logs I can see that we are calling migrate vm when we
>>>>>>>>>> have only one host and the vm seem to have been shut down before the
>>>>>>>>>> maintenance call is issued.
>>>>>>>>>>
>>>>>>>>>> Can you please look into this?
>>>>>>>

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 10-05-2019 ] [ 004_basic_sanity.vm_run ]

[Dafna Ron]

On Fri, May 10, 2019 at 12:04 PM Milan Zamazal  wrote:

> Marcin Sobczyk  writes:
>
> > Hi,
> >
> > the mentioned patch only touches our yum plugin - I don't think it's
> > related to the failure.
> >
> > VDSM fails on 'VM.create' call - Milan, could you please take a quick
> > look at it?
>
> I merged a patch to master yesterday that disables legacy pre-XML Engine
> (< 4.2) support.  Engines and clusters < 4.2 are no longer supported in
> master.
>
> I can see the VM is run with a legacy configuration rather than with
> `xml' parameter.  I wonder why -- is perhaps the cluster level < 4.2?
> If so then the cluster level must be updated before the VM is run.
>

This is on master branch. but it is happening on upgrade suite.
however, the suite is upgrading from 4.3 to 4.4:

pre-reposync-config.repo -> ../common/yum-repos/ovirt-4.3-cq.repo
reposync-config.repo -> ../common/yum-repos/ovirt-master-cq.repo



> > Regards, Marcin
> >
> > On 5/10/19 11:36 AM, Dafna Ron wrote:
> >> Hi,
> >>
> >> We are failing upgrade-from-release-suite.el7.x86_64 /
> >> 004_basic_sanity.vm_run
> >>
> >> The issue is an unexpected exception in vdsm.
> >>
> >> root cause based on CQ is this patch:
> >> https://gerrit.ovirt.org/#/c/99854/ - yum: Allow downloading only
> >> 'vdsm' package
> >>
> >> Logs can be found here:
> >>
> >>
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14079/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/
> >>
> >> Marcin, can you please take a look?
> >>
> >> error:
> >>
> >> vdsm:
> >>
> >> 2019-05-10 05:06:38,329-0400 ERROR (jsonrpc/0) [api] FINISH create
> >> error=Unexpected exception (api:131)
> >> Traceback (most recent call last):
> >>   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line
> >> 124, in method
> >> ret = func(*args, **kwargs)
> >>   File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 245, in
> create
> >> raise exception.UnexpectedError()
> >> UnexpectedError: Unexpected exception
> >> 2019-05-10 05:06:38,330-0400 INFO  (jsonrpc/0) [api.virt] FINISH
> >> create return={'status': {'message': 'Unexpected exception', 'code':
> >> 16}} from=:::192.168.201.2,39486,
> >> flow_id=83e4f0c4-a39a-45aa-891f-4022765e1a87,
> >> vmId=397013a0-b7d4-4d38-86c3-e944cebd75a7 (api:54)
> >> 2019-05-10 05:06:38,331-0400 INFO  (jsonrpc/0)
> >> [jsonrpc.JsonRpcServer] RPC call VM.create failed (error 16) in 0.00
> >> seconds (__init__:312)
> >> 2019-05-10 05:06:38,629-0400 INFO  (jsonrpc/7) [vdsm.api] FINISH
> >> getStorageDomainInfo return={'info': {'uuid':
> >> '24498263-8985-46a9-9161-f65ee776cb7f', 'vgMetadataDevice':
> >> '360014051f333158820a4cc6ab3be5b55', 'vguuid':
> >> 'rXTLNc-JwEz-0dtL-Z2EC-ASqn-gzkg-jDJa1b', 'metadataDevice':
> >> '360014051f333158820a4cc6ab3be5b55', 'state': 'OK', 'version': '4',
> >> 'role': 'Master', 'type': 'ISCSI', 'class': 'Data', 'pool':
> >> ['3e330122-c587-4078-b9a4-13dbb697c5cc'], 'name': 'iscsi'}}
> >> from=:::192.168.201.2,39486, flow_id=3f76d32a,
> >> task_id=084e7ee5-3c43-4905-833f-9a522043a862 (api:54)
> >>
> >> engine:
> >>
> >> 2019-05-10 05:06:38,328-04 ERROR
> >> [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-1)
> >> [83e4f0c4-a39a-45aa-891f-4022765e1a87] VDS::create Failed creating
> >> vm 'v
> >> m0' in vds = '566eda0f-3ef0-4791-a618-1e649af4b0be' error =
> >> 'org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> >> VDSGenericException: VDSErrorException: Failed to C
> >> reateBrokerVDS, error = Unexpected exception, code = 16'
> >> 2019-05-10 05:06:38,328-04 INFO
> >> [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-1)
> >> [83e4f0c4-a39a-45aa-891f-4022765e1a87] FINISH, CreateVDSCommand,
> >> return:
> >>  Down, log id: 3cb7c62d
> >> 2019-05-10 05:06:38,328-04 WARN
> >> [org.ovirt.engine.core.bll.RunVmCommand] (default task-1)
> >> [83e4f0c4-a39a-45aa-891f-4022765e1a87] Failed to run VM 'vm0':
> >> EngineException: or
> >> g.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> >> VDSGenericException: VDSErrorException: Failed to CreateBrokerVDS,
> >> error = Unexpected exception, code = 16 (Failed
> >>  with error unexpected

[ovirt-devel] [ OST Failure Report ] [ oVirt Master (vdsm) ] [ 10-05-2019 ] [ 004_basic_sanity.vm_run ]

[Dafna Ron]

Hi,

We are failing  upgrade-from-release-suite.el7.x86_64 /
004_basic_sanity.vm_run

The issue is an unexpected exception in vdsm.

root cause based on CQ is this patch:
https://gerrit.ovirt.org/#/c/99854/ - yum: Allow downloading only 'vdsm'
package

Logs can be found here:

http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14079/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/

Marcin, can you please take a look?

error:

vdsm:

2019-05-10 05:06:38,329-0400 ERROR (jsonrpc/0) [api] FINISH create
error=Unexpected exception (api:131)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in
method
ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 245, in create
raise exception.UnexpectedError()
UnexpectedError: Unexpected exception
2019-05-10 05:06:38,330-0400 INFO  (jsonrpc/0) [api.virt] FINISH create
return={'status': {'message': 'Unexpected exception', 'code': 16}}
from=:::192.168.201.2,39486,
flow_id=83e4f0c4-a39a-45aa-891f-4022765e1a87,
vmId=397013a0-b7d4-4d38-86c3-e944cebd75a7 (api:54)
2019-05-10 05:06:38,331-0400 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
call VM.create failed (error 16) in 0.00 seconds (__init__:312)
2019-05-10 05:06:38,629-0400 INFO  (jsonrpc/7) [vdsm.api] FINISH
getStorageDomainInfo return={'info': {'uuid':
'24498263-8985-46a9-9161-f65ee776cb7f', 'vgMetadataDevice':
'360014051f333158820a4cc6ab3be5b55', 'vguuid':
'rXTLNc-JwEz-0dtL-Z2EC-ASqn-gzkg-jDJa1b', 'metadataDevice':
'360014051f333158820a4cc6ab3be5b55', 'state': 'OK', 'version': '4', 'role':
'Master', 'type': 'ISCSI', 'class': 'Data', 'pool':
['3e330122-c587-4078-b9a4-13dbb697c5cc'], 'name': 'iscsi'}}
from=:::192.168.201.2,39486, flow_id=3f76d32a,
task_id=084e7ee5-3c43-4905-833f-9a522043a862 (api:54)

engine:

2019-05-10 05:06:38,328-04 ERROR
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-1)
[83e4f0c4-a39a-45aa-891f-4022765e1a87] VDS::create Failed creating vm 'v
m0' in vds = '566eda0f-3ef0-4791-a618-1e649af4b0be' error =
'org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to C
reateBrokerVDS, error = Unexpected exception, code = 16'
2019-05-10 05:06:38,328-04 INFO
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-1)
[83e4f0c4-a39a-45aa-891f-4022765e1a87] FINISH, CreateVDSCommand, return:
 Down, log id: 3cb7c62d
2019-05-10 05:06:38,328-04 WARN  [org.ovirt.engine.core.bll.RunVmCommand]
(default task-1) [83e4f0c4-a39a-45aa-891f-4022765e1a87] Failed to run VM
'vm0': EngineException: or
g.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
VDSGenericException: VDSErrorException: Failed to CreateBrokerVDS, error =
Unexpected exception, code = 16 (Failed
 with error unexpected and code 16)
2019-05-10 05:06:38,328-04 INFO
[org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1)
[83e4f0c4-a39a-45aa-891f-4022765e1a87] Lock freed to object
'EngineLock:{exclu
siveLocks='[397013a0-b7d4-4d38-86c3-e944cebd75a7=VM]', sharedLocks=''}'
2019-05-10 05:06:38,329-04 INFO  [org.ovirt.engine.core.bll.RunVmCommand]
(default task-1) [83e4f0c4-a39a-45aa-891f-4022765e1a87] Trying to rerun VM
'vm0'
2019-05-10 05:06:38,373-04 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-1) [83e4f0c4-a39a-45aa-891f-4022765e1a87] EVENT_ID: USE
R_INITIATED_RUN_VM_FAILED(151), Failed to run VM vm0 on Host
lago-upgrade-from-release-suite-master-host-0.
2019-05-10 05:06:38,381-04 INFO
[org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1)
[83e4f0c4-a39a-45aa-891f-4022765e1a87] Lock Acquired to object
'EngineLock:{ex
clusiveLocks='[397013a0-b7d4-4d38-86c3-e944cebd75a7=VM]', sharedLocks=''}'
2019-05-10 05:06:38,393-04 INFO
[org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default
task-1) [83e4f0c4-a39a-45aa-891f-4022765e1a87] START, IsVmDuringIn
itiatingVDSCommand(
IsVmDuringInitiatingVDSCommandParameters:{vmId='397013a0-b7d4-4d38-86c3-e944cebd75a7'}),
log id: 96be328
2019-05-10 05:06:38,393-04 INFO
[org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default
task-1) [83e4f0c4-a39a-45aa-891f-4022765e1a87] FINISH, IsVmDuringI
nitiatingVDSCommand, return: false, log id: 96be328
2019-05-10 05:06:38,403-04 WARN
[org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1)
[83e4f0c4-a39a-45aa-891f-4022765e1a87] Validation of action 'RunVmOnce'
failed
 for user admin@internal-authz. Reasons:
VAR__ACTION__RUN,VAR__TYPE__VM,VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_NO_HOSTS
2019-05-10 05:06:38,404-04 INFO
[org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1)
[83e4f0c4-a39a-45aa-891f-4022765e1a87] Lock freed to object
'EngineLock:{exclu
siveLocks='[397013a0-b7d4-4d38-86c3-e944cebd75a7=VM]', sharedLocks=''}'
2019-05-10 05:06:38,413-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]

[ovirt-devel] Re: URGENT - ovirt-engine broken for 3 days Re: Subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 05-05-2019 ] [ upgrade_hosts ]

[Dafna Ron]

As IL are on independence day, anyone else can merge?
https://gerrit.ovirt.org/#/c/99845/


On Thu, May 9, 2019 at 11:30 AM Dafna Ron  wrote:

> Thanks Andrej.
> I will follow the patch and update.
> Dafna
>
> On Thu, May 9, 2019 at 11:23 AM Andrej Krejcir 
> wrote:
>
>> Hi,
>>
>> Ok, I have posted the reverting patch:
>> https://gerrit.ovirt.org/#/c/99845/
>>
>> I'm still investigating what is the problem. Sorry for the delay, we had
>> a public holiday yesturday.
>>
>>
>> Andrej
>>
>> On Thu, 9 May 2019 at 11:20, Dafna Ron  wrote:
>>
>>> Hi,
>>>
>>> I have not heard back on this issue and ovirt-engine has been broken for
>>> the past 3 days.
>>>
>>> As this does not seem a simple debug and fix I suggest reverting the
>>> patch and investigating later.
>>>
>>> thanks,
>>> Dafna
>>>
>>>
>>>
>>> On Wed, May 8, 2019 at 9:42 AM Dafna Ron  wrote:
>>>
>>>> Any news?
>>>>
>>>> Thanks,
>>>> Dafna
>>>>
>>>>
>>>> On Tue, May 7, 2019 at 4:57 PM Dafna Ron  wrote:
>>>>
>>>>> thanks for the quick reply and investigation.
>>>>> Please update me if I can help any further and if you find the cause
>>>>> and have a patch let me know.
>>>>> Note that ovirt-engine project is broken and if we cannot find the
>>>>> cause relatively fast we should consider reverting the patch to allow a 
>>>>> new
>>>>> package to be built in CQ with other changes that were submitted.
>>>>>
>>>>> Thanks,
>>>>> Dafna
>>>>>
>>>>>
>>>>> On Tue, May 7, 2019 at 4:42 PM Andrej Krejcir 
>>>>> wrote:
>>>>>
>>>>>> After running a few OSTs manually, it seems that the patch is the
>>>>>> cause. Investigating...
>>>>>>
>>>>>> On Tue, 7 May 2019 at 14:58, Andrej Krejcir 
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> The issue is probably not caused by the patch.
>>>>>>>
>>>>>>> This log line means that the VM does not exist in the DB:
>>>>>>>
>>>>>>> 2019-05-07 06:02:04,215-04 WARN
>>>>>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] 
>>>>>>> Validation
>>>>>>> of action 'MigrateMultipleVms' failed for user admin@internal-authz.
>>>>>>> Reasons: ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>>>>>>
>>>>>>> I will investigate more, why the VM is missing.
>>>>>>>
>>>>>>> On Tue, 7 May 2019 at 14:07, Dafna Ron  wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> We are failing test upgrade_hosts on
>>>>>>>> upgrade-from-release-suite-master.
>>>>>>>> From the logs I can see that we are calling migrate vm when we have
>>>>>>>> only one host and the vm seem to have been shut down before the 
>>>>>>>> maintenance
>>>>>>>> call is issued.
>>>>>>>>
>>>>>>>> Can you please look into this?
>>>>>>>>
>>>>>>>> suspected patch reported as root cause by CQ is:
>>>>>>>>
>>>>>>>> https://gerrit.ovirt.org/#/c/98920/ - core: Add MigrateMultipleVms
>>>>>>>> command and use it for host maintenance
>>>>>>>>
>>>>>>>>
>>>>>>>> logs are found here:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14021/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/
>>>>>>>>
>>>>>>>>
>>>>>>>> I can see the issue is vm migration when putting host in
>>>>>>>> maintenance:
>>>>>>>>
>>>>>>>>
>>>>>>>> 2019-05-07 06:02:04,170-04 INFO

[ovirt-devel] Re: URGENT - ovirt-engine broken for 3 days Re: Subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 05-05-2019 ] [ upgrade_hosts ]

[Dafna Ron]

Thanks Andrej.
I will follow the patch and update.
Dafna

On Thu, May 9, 2019 at 11:23 AM Andrej Krejcir  wrote:

> Hi,
>
> Ok, I have posted the reverting patch: https://gerrit.ovirt.org/#/c/99845/
>
> I'm still investigating what is the problem. Sorry for the delay, we had a
> public holiday yesturday.
>
>
> Andrej
>
> On Thu, 9 May 2019 at 11:20, Dafna Ron  wrote:
>
>> Hi,
>>
>> I have not heard back on this issue and ovirt-engine has been broken for
>> the past 3 days.
>>
>> As this does not seem a simple debug and fix I suggest reverting the
>> patch and investigating later.
>>
>> thanks,
>> Dafna
>>
>>
>>
>> On Wed, May 8, 2019 at 9:42 AM Dafna Ron  wrote:
>>
>>> Any news?
>>>
>>> Thanks,
>>> Dafna
>>>
>>>
>>> On Tue, May 7, 2019 at 4:57 PM Dafna Ron  wrote:
>>>
>>>> thanks for the quick reply and investigation.
>>>> Please update me if I can help any further and if you find the cause
>>>> and have a patch let me know.
>>>> Note that ovirt-engine project is broken and if we cannot find the
>>>> cause relatively fast we should consider reverting the patch to allow a new
>>>> package to be built in CQ with other changes that were submitted.
>>>>
>>>> Thanks,
>>>> Dafna
>>>>
>>>>
>>>> On Tue, May 7, 2019 at 4:42 PM Andrej Krejcir 
>>>> wrote:
>>>>
>>>>> After running a few OSTs manually, it seems that the patch is the
>>>>> cause. Investigating...
>>>>>
>>>>> On Tue, 7 May 2019 at 14:58, Andrej Krejcir 
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> The issue is probably not caused by the patch.
>>>>>>
>>>>>> This log line means that the VM does not exist in the DB:
>>>>>>
>>>>>> 2019-05-07 06:02:04,215-04 WARN
>>>>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] 
>>>>>> Validation
>>>>>> of action 'MigrateMultipleVms' failed for user admin@internal-authz.
>>>>>> Reasons: ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>>>>>
>>>>>> I will investigate more, why the VM is missing.
>>>>>>
>>>>>> On Tue, 7 May 2019 at 14:07, Dafna Ron  wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> We are failing test upgrade_hosts on
>>>>>>> upgrade-from-release-suite-master.
>>>>>>> From the logs I can see that we are calling migrate vm when we have
>>>>>>> only one host and the vm seem to have been shut down before the 
>>>>>>> maintenance
>>>>>>> call is issued.
>>>>>>>
>>>>>>> Can you please look into this?
>>>>>>>
>>>>>>> suspected patch reported as root cause by CQ is:
>>>>>>>
>>>>>>> https://gerrit.ovirt.org/#/c/98920/ - core: Add MigrateMultipleVms
>>>>>>> command and use it for host maintenance
>>>>>>>
>>>>>>>
>>>>>>> logs are found here:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14021/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/
>>>>>>>
>>>>>>>
>>>>>>> I can see the issue is vm migration when putting host in
>>>>>>> maintenance:
>>>>>>>
>>>>>>>
>>>>>>> 2019-05-07 06:02:04,170-04 INFO
>>>>>>> [org.ovirt.engine.core.bll.MaintenanceVdsCommand]
>>>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2)
>>>>>>> [05592db2-f859-487b-b779-4b32eec5bab
>>>>>>> 3] Running command: MaintenanceVdsCommand internal: true. Entities
>>>>>>> affected : ID: 38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDS
>>>>>>> 2019-05-07 06:02:04,215-04 WARN
>>>>>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>

[ovirt-devel] URGENT - ovirt-engine broken for 3 days Re: Subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 05-05-2019 ] [ upgrade_hosts ]

[Dafna Ron]

Hi,

I have not heard back on this issue and ovirt-engine has been broken for
the past 3 days.

As this does not seem a simple debug and fix I suggest reverting the patch
and investigating later.

thanks,
Dafna



On Wed, May 8, 2019 at 9:42 AM Dafna Ron  wrote:

> Any news?
>
> Thanks,
> Dafna
>
>
> On Tue, May 7, 2019 at 4:57 PM Dafna Ron  wrote:
>
>> thanks for the quick reply and investigation.
>> Please update me if I can help any further and if you find the cause and
>> have a patch let me know.
>> Note that ovirt-engine project is broken and if we cannot find the cause
>> relatively fast we should consider reverting the patch to allow a new
>> package to be built in CQ with other changes that were submitted.
>>
>> Thanks,
>> Dafna
>>
>>
>> On Tue, May 7, 2019 at 4:42 PM Andrej Krejcir 
>> wrote:
>>
>>> After running a few OSTs manually, it seems that the patch is the cause.
>>> Investigating...
>>>
>>> On Tue, 7 May 2019 at 14:58, Andrej Krejcir  wrote:
>>>
>>>> Hi,
>>>>
>>>> The issue is probably not caused by the patch.
>>>>
>>>> This log line means that the VM does not exist in the DB:
>>>>
>>>> 2019-05-07 06:02:04,215-04 WARN
>>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Validation
>>>> of action 'MigrateMultipleVms' failed for user admin@internal-authz.
>>>> Reasons: ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>>>
>>>> I will investigate more, why the VM is missing.
>>>>
>>>> On Tue, 7 May 2019 at 14:07, Dafna Ron  wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We are failing test upgrade_hosts on
>>>>> upgrade-from-release-suite-master.
>>>>> From the logs I can see that we are calling migrate vm when we have
>>>>> only one host and the vm seem to have been shut down before the 
>>>>> maintenance
>>>>> call is issued.
>>>>>
>>>>> Can you please look into this?
>>>>>
>>>>> suspected patch reported as root cause by CQ is:
>>>>>
>>>>> https://gerrit.ovirt.org/#/c/98920/ - core: Add MigrateMultipleVms
>>>>> command and use it for host maintenance
>>>>>
>>>>>
>>>>> logs are found here:
>>>>>
>>>>>
>>>>>
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14021/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/
>>>>>
>>>>>
>>>>> I can see the issue is vm migration when putting host in maintenance:
>>>>>
>>>>>
>>>>> 2019-05-07 06:02:04,170-04 INFO
>>>>> [org.ovirt.engine.core.bll.MaintenanceVdsCommand]
>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2)
>>>>> [05592db2-f859-487b-b779-4b32eec5bab
>>>>> 3] Running command: MaintenanceVdsCommand internal: true. Entities
>>>>> affected : ID: 38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDS
>>>>> 2019-05-07 06:02:04,215-04 WARN
>>>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] 
>>>>> Validation
>>>>> of action
>>>>> 'MigrateMultipleVms' failed for user admin@internal-authz. Reasons:
>>>>> ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>>>> 2019-05-07 06:02:04,221-04 ERROR
>>>>> [org.ovirt.engine.core.bll.MaintenanceVdsCommand]
>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Failed to
>>>>> migrate one or
>>>>> more VMs.
>>>>> 2019-05-07 06:02:04,227-04 ERROR
>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] EVEN
>>>>> T_ID: VDS_MAINTENANCE_FAILED(17), Failed to switch Host
>>>>> lago-upgrade-from-release-suite-master-host-0 to Maintenance mode.
>>>>> 2019-05-07 06:02:04,239-04 INFO
>>>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Lock
>>>>> Acquired to object 'Eng
>>>>> ineLock:{exclusiveLocks='

[ovirt-devel] Re: Subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 05-05-2019 ] [ upgrade_hosts ]

[Dafna Ron]

Any news?

Thanks,
Dafna


On Tue, May 7, 2019 at 4:57 PM Dafna Ron  wrote:

> thanks for the quick reply and investigation.
> Please update me if I can help any further and if you find the cause and
> have a patch let me know.
> Note that ovirt-engine project is broken and if we cannot find the cause
> relatively fast we should consider reverting the patch to allow a new
> package to be built in CQ with other changes that were submitted.
>
> Thanks,
> Dafna
>
>
> On Tue, May 7, 2019 at 4:42 PM Andrej Krejcir  wrote:
>
>> After running a few OSTs manually, it seems that the patch is the cause.
>> Investigating...
>>
>> On Tue, 7 May 2019 at 14:58, Andrej Krejcir  wrote:
>>
>>> Hi,
>>>
>>> The issue is probably not caused by the patch.
>>>
>>> This log line means that the VM does not exist in the DB:
>>>
>>> 2019-05-07 06:02:04,215-04 WARN
>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Validation
>>> of action 'MigrateMultipleVms' failed for user admin@internal-authz.
>>> Reasons: ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>>
>>> I will investigate more, why the VM is missing.
>>>
>>> On Tue, 7 May 2019 at 14:07, Dafna Ron  wrote:
>>>
>>>> Hi,
>>>>
>>>> We are failing test upgrade_hosts on upgrade-from-release-suite-master.
>>>> From the logs I can see that we are calling migrate vm when we have
>>>> only one host and the vm seem to have been shut down before the maintenance
>>>> call is issued.
>>>>
>>>> Can you please look into this?
>>>>
>>>> suspected patch reported as root cause by CQ is:
>>>>
>>>> https://gerrit.ovirt.org/#/c/98920/ - core: Add MigrateMultipleVms
>>>> command and use it for host maintenance
>>>>
>>>>
>>>> logs are found here:
>>>>
>>>>
>>>>
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14021/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/
>>>>
>>>>
>>>> I can see the issue is vm migration when putting host in maintenance:
>>>>
>>>>
>>>> 2019-05-07 06:02:04,170-04 INFO
>>>> [org.ovirt.engine.core.bll.MaintenanceVdsCommand]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2)
>>>> [05592db2-f859-487b-b779-4b32eec5bab
>>>> 3] Running command: MaintenanceVdsCommand internal: true. Entities
>>>> affected : ID: 38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDS
>>>> 2019-05-07 06:02:04,215-04 WARN
>>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Validation
>>>> of action
>>>> 'MigrateMultipleVms' failed for user admin@internal-authz. Reasons:
>>>> ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>>> 2019-05-07 06:02:04,221-04 ERROR
>>>> [org.ovirt.engine.core.bll.MaintenanceVdsCommand]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Failed to
>>>> migrate one or
>>>> more VMs.
>>>> 2019-05-07 06:02:04,227-04 ERROR
>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] EVEN
>>>> T_ID: VDS_MAINTENANCE_FAILED(17), Failed to switch Host
>>>> lago-upgrade-from-release-suite-master-host-0 to Maintenance mode.
>>>> 2019-05-07 06:02:04,239-04 INFO
>>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Lock
>>>> Acquired to object 'Eng
>>>> ineLock:{exclusiveLocks='[38e1379b-c3b6-4a2e-91df-d1f346e414a9=VDS]',
>>>> sharedLocks=''}'
>>>> 2019-05-07 06:02:04,242-04 INFO
>>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Running
>>>> command: ActivateVds
>>>> Command internal: true. Entities affected : ID:
>>>> 38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDSAction group MANIPULATE_HOST
>>>> with role type ADMIN
>>>> 2019-05-07 06:02:04,243-04 INFO
>>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Before
>>

[ovirt-devel] Re: Subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 05-05-2019 ] [ upgrade_hosts ]

[Dafna Ron]

thanks for the quick reply and investigation.
Please update me if I can help any further and if you find the cause and
have a patch let me know.
Note that ovirt-engine project is broken and if we cannot find the cause
relatively fast we should consider reverting the patch to allow a new
package to be built in CQ with other changes that were submitted.

Thanks,
Dafna


On Tue, May 7, 2019 at 4:42 PM Andrej Krejcir  wrote:

> After running a few OSTs manually, it seems that the patch is the cause.
> Investigating...
>
> On Tue, 7 May 2019 at 14:58, Andrej Krejcir  wrote:
>
>> Hi,
>>
>> The issue is probably not caused by the patch.
>>
>> This log line means that the VM does not exist in the DB:
>>
>> 2019-05-07 06:02:04,215-04 WARN
>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Validation
>> of action 'MigrateMultipleVms' failed for user admin@internal-authz.
>> Reasons: ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>
>> I will investigate more, why the VM is missing.
>>
>> On Tue, 7 May 2019 at 14:07, Dafna Ron  wrote:
>>
>>> Hi,
>>>
>>> We are failing test upgrade_hosts on upgrade-from-release-suite-master.
>>> From the logs I can see that we are calling migrate vm when we have only
>>> one host and the vm seem to have been shut down before the maintenance call
>>> is issued.
>>>
>>> Can you please look into this?
>>>
>>> suspected patch reported as root cause by CQ is:
>>>
>>> https://gerrit.ovirt.org/#/c/98920/ - core: Add MigrateMultipleVms
>>> command and use it for host maintenance
>>>
>>>
>>> logs are found here:
>>>
>>>
>>>
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14021/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/
>>>
>>>
>>> I can see the issue is vm migration when putting host in maintenance:
>>>
>>>
>>> 2019-05-07 06:02:04,170-04 INFO
>>> [org.ovirt.engine.core.bll.MaintenanceVdsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2)
>>> [05592db2-f859-487b-b779-4b32eec5bab
>>> 3] Running command: MaintenanceVdsCommand internal: true. Entities
>>> affected : ID: 38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDS
>>> 2019-05-07 06:02:04,215-04 WARN
>>> [org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Validation
>>> of action
>>> 'MigrateMultipleVms' failed for user admin@internal-authz. Reasons:
>>> ACTION_TYPE_FAILED_VMS_NOT_FOUND
>>> 2019-05-07 06:02:04,221-04 ERROR
>>> [org.ovirt.engine.core.bll.MaintenanceVdsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Failed to
>>> migrate one or
>>> more VMs.
>>> 2019-05-07 06:02:04,227-04 ERROR
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] EVEN
>>> T_ID: VDS_MAINTENANCE_FAILED(17), Failed to switch Host
>>> lago-upgrade-from-release-suite-master-host-0 to Maintenance mode.
>>> 2019-05-07 06:02:04,239-04 INFO
>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Lock
>>> Acquired to object 'Eng
>>> ineLock:{exclusiveLocks='[38e1379b-c3b6-4a2e-91df-d1f346e414a9=VDS]',
>>> sharedLocks=''}'
>>> 2019-05-07 06:02:04,242-04 INFO
>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Running
>>> command: ActivateVds
>>> Command internal: true. Entities affected : ID:
>>> 38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDSAction group MANIPULATE_HOST
>>> with role type ADMIN
>>> 2019-05-07 06:02:04,243-04 INFO
>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Before
>>> acquiring lock in ord
>>> er to prevent monitoring for host
>>> 'lago-upgrade-from-release-suite-master-host-0' from data-center 'test-dc'
>>> 2019-05-07 06:02:04,243-04 INFO
>>> [org.ovirt.engine.core.bll.ActivateVdsCommand]
>>> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Lock
>>> acquired, from now a mo
>>> nitoring of host will be skipped for host
>>> 'lago-upgrade-f

[ovirt-devel] Subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 05-05-2019 ] [ upgrade_hosts ]

[Dafna Ron]

Hi,

We are failing test upgrade_hosts on upgrade-from-release-suite-master.
>From the logs I can see that we are calling migrate vm when we have only
one host and the vm seem to have been shut down before the maintenance call
is issued.

Can you please look into this?

suspected patch reported as root cause by CQ is:

https://gerrit.ovirt.org/#/c/98920/ - core: Add MigrateMultipleVms command
and use it for host maintenance


logs are found here:


http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14021/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-004_basic_sanity.py/


I can see the issue is vm migration when putting host in maintenance:


2019-05-07 06:02:04,170-04 INFO
[org.ovirt.engine.core.bll.MaintenanceVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2)
[05592db2-f859-487b-b779-4b32eec5bab
3] Running command: MaintenanceVdsCommand internal: true. Entities affected
: ID: 38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDS
2019-05-07 06:02:04,215-04 WARN
[org.ovirt.engine.core.bll.MigrateMultipleVmsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Validation
of action
'MigrateMultipleVms' failed for user admin@internal-authz. Reasons:
ACTION_TYPE_FAILED_VMS_NOT_FOUND
2019-05-07 06:02:04,221-04 ERROR
[org.ovirt.engine.core.bll.MaintenanceVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] Failed to
migrate one or
more VMs.
2019-05-07 06:02:04,227-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [33485140] EVEN
T_ID: VDS_MAINTENANCE_FAILED(17), Failed to switch Host
lago-upgrade-from-release-suite-master-host-0 to Maintenance mode.
2019-05-07 06:02:04,239-04 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Lock
Acquired to object 'Eng
ineLock:{exclusiveLocks='[38e1379b-c3b6-4a2e-91df-d1f346e414a9=VDS]',
sharedLocks=''}'
2019-05-07 06:02:04,242-04 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Running
command: ActivateVds
Command internal: true. Entities affected : ID:
38e1379b-c3b6-4a2e-91df-d1f346e414a9 Type: VDSAction group MANIPULATE_HOST
with role type ADMIN
2019-05-07 06:02:04,243-04 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Before
acquiring lock in ord
er to prevent monitoring for host
'lago-upgrade-from-release-suite-master-host-0' from data-center 'test-dc'
2019-05-07 06:02:04,243-04 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Lock
acquired, from now a mo
nitoring of host will be skipped for host
'lago-upgrade-from-release-suite-master-host-0' from data-center 'test-dc'
2019-05-07 06:02:04,252-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] START,
SetVdsStatu
sVDSCommand(HostName = lago-upgrade-from-release-suite-master-host-0,
SetVdsStatusVDSCommandParameters:{hostId='38e1379b-c3b6-4a2e-91df-d1f346e414a9',
status='Unassigned', n
onOperationalReason='NONE', stopSpmFailureLogged='false',
maintenanceReason='null'}), log id: 2c8aa211
2019-05-07 06:02:04,256-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] FINISH,
SetVdsStat
usVDSCommand, return: , log id: 2c8aa211
2019-05-07 06:02:04,261-04 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Activate
host finished. Lock
released. Monitoring can run now for host
'lago-upgrade-from-release-suite-master-host-0' from data-center 'test-dc'
2019-05-07 06:02:04,265-04 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] EVEN
T_ID: VDS_ACTIVATE(16), Activation of host
lago-upgrade-from-release-suite-master-host-0 initiated by
admin@internal-authz.
2019-05-07 06:02:04,266-04 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(EE-ManagedThreadFactory-commandCoordinator-Thread-2) [70840477] Lock freed
to object 'Engine
Lock:{exclusiveLocks='[38e1379b-c3b6-4a2e-91df-d1f346e414a9=VDS]',
sharedLocks=''}'
2019-05-07 06:02:04,484-04 ERROR
[org.ovirt.engine.core.bll.hostdeploy.HostUpgradeCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-96)
[05592db2-f859-487b-b779-4b32
eec5bab3] Host 'lago-upgrade-from-release-suite-master-host-0' failed to
move to maintenance mode. Upgrade process is terminated.

I can see there was only one vm running:


drwxrwxr-x. 2 dron dron 1024 May 7 11:49 qemu
[dron@dron post-004_basic_sanity.py]$ ls -l
lago-upgrade-from-release-suite-master-host-0/_var_log/libvirt/qemu/
total 6
-rw-rw-r--. 1 dron dron 4466 May 7 10:12 vm-with-iface.log

[ovirt-devel] Re: OST broken - looks like https://gerrit.ovirt.org/c/99717 is the reason

[Dafna Ron]

if you mean CQ then it did not pass CI - I sent you a mail to look at your
patch as it was failing CQ

On Thu, May 2, 2019 at 7:29 PM Dan Kenigsberg  wrote:

>
>
> On Thu, 2 May 2019, 18:53 Nir Soffer,  wrote:
>
>> Investigating unrelated OST failue with:
>> https://gerrit.ovirt.org/c/99293
>>
>> I found that This patch fail in OST:
>> https://gerrit.ovirt.org/c/99717
>>
>> Failed build:
>> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4630/
>>
>> 2019-05-02 15:46:07,258::ssh.py::ssh::96::lago.ssh::DEBUG::Command 650bdf66 
>> on lago-basic-suite-master-host-0  errors:
>>  + yum -y install ovirt-host
>> Error: Package: vdsm-common-4.40.0-202.git9757d7c.el7.noarch (alocalsync)
>>Requires: python2-enum34
>>
>>
>> Dan, can you take a look at this?
>>
>
> Wat?! I don't understand how it passed vdsm CI.
>
> worst case I'll revert.
>
>
>> Nir
>>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/F6IUW6SFQE3A2VDGSFFKTUXC5AYN4XSL/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.2 (vdsm-jsonrpc-java) ] [ 12-04-2019 ] [ 002_bootstrap.add_dc ]

[Dafna Ron]

This is also appearing on 4.3 on test:  002_bootstrap.download_engine_certs
http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/654/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.3/post-002_bootstrap.py

Master patch failed on host deploy but I think it would fail on the same if
I re-run.


On Sun, Apr 14, 2019 at 9:22 AM Dafna Ron  wrote:

> Hi,
>
> We had a failure on both basic and upgrade suites.
> The issue seems to be in initialize engine with the below error
>
> Root cause change: https://gerrit.ovirt.org/#/c/99278/ - Purge old events
> and EventPublisher should handle exceptions
>
> Logs can be found here:
>
> http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/4217/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.2/
>
> error:
>
> 2019-04-12 03:36:25,571-04 ERROR [org.ovirt.engine.core.bll.Backend]
> (ServerService Thread Pool -- 44) [] Error during initialization:
> org.jboss.weld.exceptions.WeldException: WELD-49: Unable to invoke
> private void
> org.ovirt.engine.core.vdsbroker.monitoring.VmMigrationProgressMonitoring.subscribe()
> on
> org.ovirt.engine.core.vdsbroker.monitoring.VmMigrationProgressMonitoring@c22d2ea
> at
> org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:85)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:66)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:122)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:162)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.contexts.AbstractContext.get(AbstractContext.java:96)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:100)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.bean.ContextualInstanceStrategy$ApplicationScopedContextualInstanceStrategy.get(ContextualInstanceStrategy.java:140)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:700)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.bean.builtin.InstanceImpl.getBeanInstance(InstanceImpl.java:252)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.weld.bean.builtin.InstanceImpl.get(InstanceImpl.java:114)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
> org.ovirt.engine.core.bll.ServiceLoader.load(ServiceLoader.java:34)
> [bll.jar:]
> at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:312)
> [bll.jar:]
> at org.ovirt.engine.core.bll.Backend.create(Backend.java:195)
> [bll.jar:]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [rt.jar:1.8.0_201]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [rt.jar:1.8.0_201]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [rt.jar:1.8.0_201]
> at java.lang.reflect.Method.invoke(Method.java:498)
> [rt.jar:1.8.0_201]
> at
> org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96)
> at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at
> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at
> org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:78)
> at
> org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:125)
> at
> org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:111)
> at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at
> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at
> org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
> [weld-ejb-3.0.5.Final.jar:3.0.5.Final]
> at
> org.jboss.as.weld.e

[ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (vdsm-jsonrpc-java) ] [ 12-04-2019 ] [ 002_bootstrap.add_dc ]

[Dafna Ron]

Hi,

We had a failure on both basic and upgrade suites.
The issue seems to be in initialize engine with the below error

Root cause change: https://gerrit.ovirt.org/#/c/99278/ - Purge old events
and EventPublisher should handle exceptions

Logs can be found here:
http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/4217/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.2/

error:

2019-04-12 03:36:25,571-04 ERROR [org.ovirt.engine.core.bll.Backend]
(ServerService Thread Pool -- 44) [] Error during initialization:
org.jboss.weld.exceptions.WeldException: WELD-49: Unable to invoke
private void
org.ovirt.engine.core.vdsbroker.monitoring.VmMigrationProgressMonitoring.subscribe()
on
org.ovirt.engine.core.vdsbroker.monitoring.VmMigrationProgressMonitoring@c22d2ea
at
org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:85)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:66)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:122)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:162)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.contexts.AbstractContext.get(AbstractContext.java:96)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:100)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.ContextualInstanceStrategy$ApplicationScopedContextualInstanceStrategy.get(ContextualInstanceStrategy.java:140)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:700)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.builtin.InstanceImpl.getBeanInstance(InstanceImpl.java:252)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.builtin.InstanceImpl.get(InstanceImpl.java:114)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.ovirt.engine.core.bll.ServiceLoader.load(ServiceLoader.java:34)
[bll.jar:]
at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:312)
[bll.jar:]
at org.ovirt.engine.core.bll.Backend.create(Backend.java:195)
[bll.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[rt.jar:1.8.0_201]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[rt.jar:1.8.0_201]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_201]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_201]
at
org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at
org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:78)
at
org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:125)
at
org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:111)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at
org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
[weld-ejb-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:53)
[wildfly-weld-14.0.1.Final.jar:14.0.1.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ee.component.AroundConstructInterceptorFactory$1.processInvocation(AroundConstructInterceptorFactory.java:28)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.weld.injection.WeldInterceptorInjectionInterceptor.processInvocation(WeldInterceptorInjectionInterceptor.java:56)

[ovirt-devel] Re: GetVdsHooksByIdQuery comes back with NullPointerException

[Dafna Ron]

we merged Marcin's patch to move the tests to a different location so
hopefully it will help.
if not, I will report back on any new findings.

Thanks Martin and Marcin!

On Thu, Apr 11, 2019 at 4:39 PM Martin Perina  wrote:

> Hi Dafna,
>
> I've just merged https://gerrit.ovirt.org/99339 which adds missing debug
> logs for BLL part of engine, which contains core functionality. So
> hopefully now we should have finally all debug info to be able to
> investigate a fix all race/random issues we have in the last weeks.
>
> Regarding the GetVdsHooksByIdQuery I think Marcin started to investigate
> and the issue was that for some unknown reason hooks are not stored within
> engine database, which seems like a race between hooks are queried from
> client and properly fetched from host withing HostMonitoring. So next time
> it happens we should hopefully have enough information to be able to find
> out reason.
>
> Thanks
> M.
>
> On Thu, Apr 11, 2019 at 2:13 PM Dafna Ron  wrote:
>
>> Hi,
>>
>> This is a code issue 100%, I am not sure which team sends the query but
>> this needs to be addressed. I think it would also help debug/fix the issue
>> with the random failure of get_host_hooks which was already reported and
>> communicated on in the list.
>>
>> Can someone from infra/network please look at the query and see what is
>> the cause of the NPE?
>>
>> Thanks,
>> Dafna
>>
>>
>>
>>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/RDCNI4SU7S4UYJCTL4BK7PLXWT7HKW3R/

[ovirt-devel] GetVdsHooksByIdQuery comes back with NullPointerException

[Dafna Ron]

Hi,

This is a code issue 100%, I am not sure which team sends the query but
this needs to be addressed. I think it would also help debug/fix the issue
with the random failure of get_host_hooks which was already reported and
communicated on in the list.

Can someone from infra/network please look at the query and see what is the
cause of the NPE?

Thanks,
Dafna
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/BQPISLLBQLVMBQRTOIQ7HJHS4TIPJLD4/

[ovirt-devel] [Ovirt] [CQ weekly status] [05-04-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last CQ job failure on 4.2 was 05-04-2019 on project ovirt-provider-ovn due
to a code regression which was caused by [1]
The issue has been found and a patch created [2] but due to the hotplug_cpu
failure, it took a long time to verify the fix before merge.
[1] https://gerrit.ovirt.org/#/c/98723/ - Stateless dhcpv6 does not support
fixed IPs
[2] https://gerrit.ovirt.org/#/c/99193/ - Fix bug when fixed IPs are not
provisioned

*CQ-4.3*:  RED (#3)

We have been having constant failures on tests:
hotplug_cpu
add_master_storage_domain
The failures on these two are very frequent and I need to re-trigger tests
on multiple failures daily.
There is a mail thread on this and several people have joined in to help
debug the issue but I think that this issue needs more help from some of
the other teams as its been going on for the last two weeks at least.
This is also disrupting fixing actual regressions as the testing keep
failing on these two unrelated issues.


*CQ-Master:*   RED (#3)

Master and 4.3 have the same issues.

 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DL4LFXYJAY6IPKK4QRMYSS3BTME5YQLW/

[ovirt-devel] [Ovirt] [CQ weekly status] [29-03-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last CQ job failure on 4.2 was 25-03-2019 on project
ovirt-ansible-hosted-engine-setup due missing polkit package.

*CQ-4.3*:  RED (#1)

failures in 4.3 and master this week were caused by two issues:
1. we have had random failures on 4 different tests which we are working on
debugging with Marcin, Martin and several more people. this has been
disruptive this week but did not cause any delays as I was re-triggering
failed projects to insure they have their packages buit in tested.
Currently we suspect api issue or changes to poastgres are causing the
failures
2. we had libcgroup-tools package missing which cause failures for all
projects in initialize engine. I merged a patch to fix the issue and
provide the package.

*CQ-Master:*   RED (#1)

We have had the same issues as in 4.3 this week.

 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/F7LITL444FA2LZJBN3HNLQFAL2LGTKER/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

Thank you Marcin :) I am also adding Didi to see if he knows of any
changes.
Anton, who from Engine/api side can help to take it from here?


On Fri, Mar 29, 2019 at 9:37 AM Marcin Sobczyk  wrote:

> Hi,
>
> so the thing boils down to this:
>
>
> https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/471/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.3/post-002_bootstrap.py/lago-basic-suite-4-3-engine/_etc_httpd/logs/ssl_access_log/*view*/
> This is 'httpd' log file from the most recent job that failed
> 'add_master_storage_domain'.
> At the end of the log file, we can see:
>
> 192.168.202.1 - - [28/Mar/2019:11:05:38 -0400] "GET 
> /ovirt-engine/api/hosts?search=datacenter%3Dtest-dc+AND+status%3Dinstalling+or+status%3Dinitializing
>  HTTP/1.1" 200 11845
>
> 192.168.202.1 - - [28/Mar/2019:11:05:41 -0400] "GET 
> /ovirt-engine/api/hosts?search=datacenter%3Dtest-dc+AND+status%3Dinstalling+or+status%3Dinitializing
>  HTTP/1.1" 200 11847
>
> 192.168.202.1 - - [28/Mar/2019:11:05:44 -0400] "GET 
> /ovirt-engine/api/hosts?search=datacenter%3Dtest-dc+AND+status%3Dinstalling+or+status%3Dinitializing
>  HTTP/1.1" 200 11847
>
> 192.168.202.1 - - [28/Mar/2019:11:05:47 -0400] "GET 
> /ovirt-engine/api/hosts?search=datacenter%3Dtest-dc+AND+status%3Dinstalling+or+status%3Dinitializing
>  HTTP/1.1" 200 5959
>
> 192.168.202.1 - - [28/Mar/2019:11:05:47 -0400] "GET 
> /ovirt-engine/api/hosts?search=datacenter%3Dtest-dc+AND+status%3Dup HTTP/1.1" 
> 200 7826
>
> 192.168.202.1 - - [28/Mar/2019:11:05:47 -0400] "GET /ovirt-engine/api 
> HTTP/1.1" 200 5112
>
> 192.168.202.1 - - [28/Mar/2019:11:05:48 -0400] "GET 
> /ovirt-engine/api/hosts?search=datacenter%3Dtest-dc+AND+status%3Dup HTTP/1.1" 
> 200 65
>
> 192.168.202.2 - - [28/Mar/2019:11:05:50 -0400] "POST 
> /ovirt-engine/sso/oauth/token-info HTTP/1.1" 200 223
>
> The first batch of 'status=installing or status=initializing' requests and
> the first 'status=up' request is 'verify_add_hosts'.
> The second 'status=up' request is the one that comes from '_hosts_in_dc'
> call from 'add_master_storage_domain'.
> They are executed with a 1 second time difference, yet the results are not
> the same - 7826 bytes vs 65 bytes.
> I went through every line of 'engine.log' from that 1 sec interval and
> couldn't find anything, that would indicate the host is going down.
> I've asked Ravi for help yesterday and I've found out, that the results of
> these requests are queries to our postgres db and they are somehow cached.
> I see 2 options here - either something really changed the db contents
> during that 1 second or there's a bug in our API/cache that returns the
> wrong info.
> I think we can eliminate the first option by adding collection of postgres
> db to our OST suites and looking at it's history after a failure.
> I don't really have a solution for the second possibility - any help from
> someone who knows engine's code better would be appreciated.
>
>
> On 3/26/19 5:37 PM, Dafna Ron wrote:
>
> We never remove any packages from tested :)
> I am actually suspecting a problem that is caused by something that is
> shared by all projects like the api.
> we currently have 3 different tests that are randomly failing because of
> engine/host communication and I am starting to think they are all
> connected.
>
>
> On Tue, Mar 26, 2019 at 2:03 PM Eyal Edri  wrote:
>
>>
>>
>> On Tue, Mar 26, 2019 at 3:56 PM Sandro Bonazzola 
>> wrote:
>>
>>>
>>>
>>> Il giorno mar 26 mar 2019 alle ore 14:42 Marcin Sobczyk <
>>> msobc...@redhat.com> ha scritto:
>>>
>>>> Ok, maybe I've found something. I was comparing two sets of log files -
>>>> one for basic-suite-4.3 run that succeeded and one that did not.
>>>>
>>>> The deployment procedure for host-1 is not complete in the unsuccessful
>>>> case. The last log entry is:
>>>>
>>>> 2019-03-22 17:54:49,973-04 INFO  
>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>>>> (VdsDeploy) [6f2e097b] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing 
>>>> Host lago-basic-suite-4-3-host-1. Starting vdsm.
>>>>
>>>> In the successful case we can see:
>>>>
>>>> 2019-03-25 06:16:31,789-04 INFO  
>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>>>> (VdsDeploy) [3add0cac] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing 
>>>> Host lago-basic-suite-4-3-host-1. Starting vdsm.
>>>> 2019-03-25 06:16:38,257-04 DEBUG 
>>>

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

We never remove any packages from tested :)
I am actually suspecting a problem that is caused by something that is
shared by all projects like the api.
we currently have 3 different tests that are randomly failing because of
engine/host communication and I am starting to think they are all
connected.


On Tue, Mar 26, 2019 at 2:03 PM Eyal Edri  wrote:

>
>
> On Tue, Mar 26, 2019 at 3:56 PM Sandro Bonazzola 
> wrote:
>
>>
>>
>> Il giorno mar 26 mar 2019 alle ore 14:42 Marcin Sobczyk <
>> msobc...@redhat.com> ha scritto:
>>
>>> Ok, maybe I've found something. I was comparing two sets of log files -
>>> one for basic-suite-4.3 run that succeeded and one that did not.
>>>
>>> The deployment procedure for host-1 is not complete in the unsuccessful
>>> case. The last log entry is:
>>>
>>> 2019-03-22 17:54:49,973-04 INFO  
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>>> (VdsDeploy) [6f2e097b] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing 
>>> Host lago-basic-suite-4-3-host-1. Starting vdsm.
>>>
>>> In the successful case we can see:
>>>
>>> 2019-03-25 06:16:31,789-04 INFO  
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>>> (VdsDeploy) [3add0cac] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing 
>>> Host lago-basic-suite-4-3-host-1. Starting vdsm.
>>> 2019-03-25 06:16:38,257-04 DEBUG 
>>> [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3add0cac] Got: 
>>> **%EventEnd STAGE closeup METHOD 
>>> otopi.plugins.ovirt_host_deploy.vdsm.packages.Plugin._start 
>>> (odeploycons.packages.vdsm.started)
>>> 2019-03-25 06:16:38,257-04 DEBUG 
>>> [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3add0cac] Got: 
>>> **%EventStart STAGE closeup METHOD 
>>> otopi.plugins.ovirt_host_deploy.vmconsole.packages.Plugin._start (None)
>>> 2019-03-25 06:16:38,257-04 DEBUG 
>>> [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3add0cac] Got: 
>>> ***L:INFO Starting ovirt-vmconsole-host-sshd
>>> 2019-03-25 06:16:38,257-04 DEBUG 
>>> [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3add0cac] 
>>> nextEvent: Log INFO Starting ovirt-vmconsole-host-sshd
>>> 2019-03-25 06:16:38,261-04 INFO  
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>>> (VdsDeploy) [3add0cac] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing 
>>> Host lago-basic-suite-4-3-host-1. Starting ovirt-vmconsole-host-sshd.
>>> 2019-03-25 06:16:39,479-04 DEBUG 
>>> [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3add0cac] Got: 
>>> **%EventEnd STAGE closeup METHOD 
>>> otopi.plugins.ovirt_host_deploy.vmconsole.packages.Plugin._start (None)
>>> ...
>>>
>>>
>>> It looks like ovirt-host-deploy plugin did not respond?
>>>
>>> The timing of:
>>>
>>> https://gerrit.ovirt.org/#/c/98762/
>>>
>>> kinda aligns with my patch which was merged for vdsm-4.3 and was a
>>> suspect:
>>>
>>> https://gerrit.ovirt.org/#/c/98748/
>>>
>>> Sandro, are you aware of any issues on ovirt-host-deploy plugin that
>>> might have caused these?
>>>
>>
>> I don't really see how a patch touching the automation configuratin for
>> STDCI v2 can be the cause of this but please go ahead and blacklist last
>> build.
>> It should differ from previous one only by the timestamp of the rpm :-)
>>
>
> If you want to verify a specific version of an RPM works , you can use the
> manual job for it.
>
>
>>
>>
>>> Dafna, can we temporarily blacklist the latest version of
>>> ovirt-host-deploy to see if it fixes CQ?
>>>
>>> Marcin
>>> On 3/25/19 2:45 PM, Dafna Ron wrote:
>>>
>>> yes. I suspect this is the problem.
>>> In the host I can see in the yum log the times the vdsm packages are
>>> installed:
>>> [dron@dron post-002_bootstrap.py]$ less
>>> lago-basic-suite-4-3-host-1/_var_log/yum.log |grep vdsm
>>> Mar 25 07:48:49 Installed: vdsm-api-4.30.11-23.git276b602.el7.noarch
>>> Mar 25 07:48:52 Installed:
>>> vdsm-yajsonrpc-4.30.11-23.git276b602.el7.noarch
>>> Mar 25 07:49:00 Installed: vdsm-common-4.30.11-23.git276b602.el7.noarch
>>> Mar 25 07:49:00 Installed:
>>> vdsm-hook-vhostmd-4.30.11-23.git276b602.el7.noarch
>>> Mar 25 07:49:38 Installed: vdsm-network-4.30.11-23.git276b602.el7.x86_64
>>> Mar 25 07:4

[ovirt-devel] Re: [NOTICE] Failing randomly on get_host_hooks with a NullPointerException

[Dafna Ron]

So another failure today.
http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/405/

any updates?

On Fri, Mar 22, 2019 at 3:08 PM Dan Kenigsberg  wrote:

> On Fri, Mar 22, 2019 at 1:57 PM Sandro Bonazzola 
> wrote:
>
>>
>>
>> Il giorno ven 22 mar 2019 alle ore 12:42 Dan Kenigsberg <
>> dan...@redhat.com> ha scritto:
>>
>>>
>>>
>>> On Fri, 22 Mar 2019, 12:21 Sandro Bonazzola, 
>>> wrote:
>>>


 Il giorno ven 22 mar 2019 alle ore 11:14 Dan Kenigsberg <
 dan...@redhat.com> ha scritto:

>
>
> On Fri, 22 Mar 2019, 12:00 Sandro Bonazzola, 
> wrote:
>
>>
>>
>> Il giorno ven 22 mar 2019 alle ore 10:52 Dan Kenigsberg <
>> dan...@redhat.com> ha scritto:
>>
>>> Yes, I'm repeating myself.
>>> SKIPPING TESTS IS BAD
>>>
>>
>> I agree. And having the suite failing on a broken test skipping all
>> the following tests is even worse.
>> This is why I would prefer the rest of the product being tested while
>> someone take ownership of the broken test and fix it.
>>
>
> This is a good reason to rewrite OST with pytest, which continues on
> failure.
>

 Patches are welcome :-)

>>>
>>> This is not an empty gesture. The network suite came into being because
>>> of this issue (and others)
>>>
>>>

> And a good reason to ping mperina on IRC to debug this. And a good
> reason not to merge new code.
>
> It doesn't convince me that we should ignore the failure without due
> debugging.
>

 Debugging in indeed needed but not on production system blocking the
 rest of the CI. Maintainer of the test can debug it on own test 
 environment.

>>>
>>> The product of this system are bugs. We found one. If you skip it, we
>>> all risk it being forgotten. Skipping should be rare, and happen only after
>>> the owner is found and admits that he is too busy/lazy to fix it now, and
>>> files a bug to fix it later.
>>>
>>
>> We didn't found a bug in the product we are testing, we found a bug in
>> the test that still need to be identified.
>> According to Dafna: "we are randomly failing on get_host_hooks test for
>> at least 3 weeks. its not a specific branch or project and there are no
>> commonalities that I can see,"
>> If it was a bug in the product I would have totally agreed with you, it
>> couldn't have been ignored. I'm not saying to ignore this as well.
>> Being a bug in the test itself
>>
>
> I have no idea if this is the case. NullPointerException smells like
> something coming deep from Engine's data model
>
>
>> I would rather prefer take a non reliable test off for further
>> investigation on a development environment and ensure the rest of the tests
>> are being executed in production environment finding bugs on the product if
>> there are.
>>
>
> Skip is still on the table as an option. The infra team may request us to
> use it. But we should first put the pressure on them to fix it properly.
> mperina and msobczyk are now aware of the issue; they should decide if
> they fix it now or asynchronously.
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MN3TTDUDKEB3L7LFOOI66EGAIHTPACF2/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt master (ovirt-engine) ] [ 22-03-2019 ] [004_basic_sanity.hotplug_cpu ]

[Dafna Ron]

This is still failing randomly


On Tue, Mar 26, 2019 at 8:15 AM Dominik Holler  wrote:

> On Mon, 25 Mar 2019 17:30:53 -0400
> Ryan Barry  wrote:
>
> > It may be virt, but I'm looking...
> >
> > I'm very suspicious of this happening immediately after hotplugging a
> NIC,
> > especially since the bug attached to https://gerrit.ovirt.org/#/c/98765/
> > talks about dropping packets. Dominik, did anything else change here?
> >
>
> No, nothing I am aware of.
>
> Is there already a pattern in the failed runs detected, or does it fail
> randomly?
>
> > On Mon, Mar 25, 2019 at 12:42 PM Anton Marchukov 
> > wrote:
> >
> > > Which team is it? Is it Virt? Just checking who should open a bug in
> > > libvirt as suggested.
> > >
> > > > On 22 Mar 2019, at 20:52, Nir Soffer  wrote:
> > > >
> > > > On Fri, Mar 22, 2019 at 7:12 PM Dafna Ron  wrote:
> > > > Hi,
> > > >
> > > > We are failing ovirt-engine master on test
> 004_basic_sanity.hotplug_cpu
> > > > looking at the logs, we can see that the for some reason, libvirt
> > > reports a vm as none responsive which fails the test.
> > > >
> > > > CQ first failure was for patch:
> > > > https://gerrit.ovirt.org/#/c/98553/ - core: Add display="on" for
> mdevs,
> > > use nodisplay to override
> > > > But I do not think this is the cause of failure.
> > > >
> > > > Adding Marcin, Milan and Dan as well as I think it may be netwrok
> > > related.
> > > >
> > > > You can see the libvirt log here:
> > > >
> > >
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/lago-basic-suite-master-host-1/_var_log/libvirt.log
> > > >
> > > > you can see the full logs here:
> > > >
> > > >
> > >
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/
> > > >
> > > > Evgheni and I confirmed this is not an infra issue and the problem is
> > > ssh connection to the internal vm
> > > >
> > > > Thanks,
> > > > Dafna
> > > >
> > > >
> > > > error:
> > > > 2019-03-22 15:08:22.658+: 22068: warning :
> qemuDomainObjTaint:7521 :
> > > Domain id=3 name='vm0' uuid=a9443d02-e054-40bb-8ea3-ae346e2d02a7 is
> > > tainted: hook-script
> > > >
> > > > Why our vm is tainted?
> > > >
> > > > 2019-03-22 15:08:22.693+: 22068: error :
> > > virProcessRunInMountNamespace:1159 : internal error: child reported:
> unable
> > > to set security context 'system_u:object_r:virt_content_t:s0' on
> > >
> '/rhev/data-center/mnt/blockSD/91d97292-9ac3-4d77-a152-c7ea3250b065/images/e60dae48-ecc7-4171-8bfe-42bfc2190ffd/40243c76-a384-4497-8a2d-792a5e10d510':
> > > No such file or directory
> > > >
> > > > This should not happen, libvirt is not adding labels to files in
> > > /rhev/data-center. It is using using its own mount
> > > > namespace and adding there the devices used by the VM. Since libvirt
> > > create the devices in its namespace
> > > > it should not complain about missing paths in /rhev/data-center.
> > > >
> > > > I think we should file a libvirt bug for this.
> > > >
> > > > 2019-03-22 15:08:28.168+: 22070: error :
> > > qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU
> guest
> > > agent is not connected
> > > > 2019-03-22 15:08:58.193+: 22070: error :
> > > qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU
> guest
> > > agent is not connected
> > > > 2019-03-22 15:13:58.179+: 22071: error :
> > > qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU
> guest
> > > agent is not connected
> > > >
> > > > Do we have guest agent in the test VMs?
> > > >
> > > > Nir
> > >
> > > --
> > > Anton Marchukov
> > > Associate Manager - RHV DevOps - Red Hat
> > >
> > >
> > >
> > >
> > >
> > > ___
> > > Infra mailing list -- in...@ovirt.org
> > > To unsubscribe send an email to infra-le...@ovirt.org
> > > Priva

[ovirt-devel] Re: Lago's local sync repo not available in OST

[Dafna Ron]

from what I can see, the host is unreachable at the time this is running.
The actual job failed due to a reported issue (which is the hotplug_cpu) in
libvirt:

2019-03-25 14:28:38.324+: 8010: info : libvirt version: 4.5.0, package:
10.el7_6.4 (CentOS BuildSystem ,
2019-01-29-17:31:22, x86-01.bsys.centos.org)
2019-03-25 14:28:38.324+: 8010: info : hostname:
lago-basic-suite-master-host-0
2019-03-25 14:28:38.324+: 8010: error : virNetSocketReadWire:1806 : End
of file while reading data: Input/output error
2019-03-25 14:42:54.353+: 22186: info : libvirt version: 4.5.0,
package: 10.el7_6.4 (CentOS BuildSystem ,
2019-01-29-17:31:22, x86-01.bsys.centos.org)
2019-03-25 14:42:54.353+: 22186: info : hostname:
lago-basic-suite-master-host-0
2019-03-25 14:42:54.353+: 22186: warning : qemuDomainObjTaint:7521 :
Domain id=1 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is
tainted: host-cpu
2019-03-25 14:43:51.078+: 22188: error : qemuDomainAgentAvailable:9133
: Guest agent is not responding: QEMU guest agent is not connected
2019-03-25 14:45:12.597+: 22186: warning : qemuDomainObjTaint:7521 :
Domain id=2 name='vm0' uuid=ed91d5d5-d342-4537-9807-a1fe4423ddcd is
tainted: hook-script
2019-03-25 14:45:12.692+: 22186: error :
virProcessRunInMountNamespace:1159 : internal error: child reported: unable
to set security context 'system_u:object_r:virt_content_t:s0' on
'/rhev/data-center/mnt/blockSD/cf0b3c9b-b447-48a8-a74a-ca8ffa775965/images/4319f48b-2d60-4295-886b-ecaf9f13d5d8/b1da85a2-3668-4ba2-be71-11e38e6ad772':
No such file or directory
2019-03-25 14:45:33.130+: 22188: error : qemuDomainAgentAvailable:9133
: Guest agent is not responding: QEMU guest agent is not connected
2019-03-25 14:46:35.836+: 22183: error : qemuMonitorIO:718 : internal
error: End of file from qemu monitor
2019-03-25 14:46:49.492+: 22184: warning : qemuDomainObjTaint:7521 :
Domain id=3 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is
tainted: host-cpu
2019-03-25 14:48:51.084+: 22187: error : qemuDomainAgentAvailable:9133
: Guest agent is not responding: QEMU guest agent is not connected
(END)



On Mon, Mar 25, 2019 at 4:04 PM Sandro Bonazzola 
wrote:

> Hi,
> just seen:
> 2019-03-25 10:41:36,062-04 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (EE-ManagedThreadFactory-hostUpdatesChecker-Thread-2) [] EVENT_ID:
> HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates
> on host lago-basic-suite-master-host-1 with message 'Failed to run
> check-update of host 'lago-basic-suite-master-host-1'. Error: fatal:
> [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "msg": "
> http://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14]
> curl#7 - \"Failed connect to 192.168.201.1:8585; Connection
> refused\"\nTrying other mirror.\nhttp://
> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 -
> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying
> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml:
> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection
> refused\"\nTrying other mirror.\nhttp://
> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 -
> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying
> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml:
> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection
> refused\"\nTrying other mirror.\nhttp://
> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 -
> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying
> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml:
> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection
> refused\"\nTrying other mirror.\nhttp://
> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 -
> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying
> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml:
> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection
> refused\"\nTrying other mirror.\nhttp://
> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 -
> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying
> other mirror.\n\n\n One of the configured repositories failed (Latest oVirt
> nightly),\n and yum doesn't have enough cached data to continue. At this
> point the only\n safe thing yum can do is fail. There are a few ways to
> work \"fix\" this:\n\n 1. Contact the upstream for the repository and
> get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for
> the repository, to point to a working\nupstream. This is most often
> useful if you are using a newer\ndistribution release than is
> supported by the repository (and the\npackages 

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt master (ovirt-engine) ] [ 22-03-2019 ] [004_basic_sanity.hotplug_cpu ]

[Dafna Ron]

we have more failures with this error.

I checked latest failure and guest-agent is installed.
logs can be found here for latest failed job:

http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/371/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.3/post-004_basic_sanity.py

2019-03-25 14:57:05,176::INFO::repoman.common.stores.RPM::Adding
package 
/var/lib/lago/ovirt-4.3-snapshot-static-el7/noarch/ovirt-guest-agent-common-1.0.16-1.el7.noarch.rpm
to repo Non persistent RPMStore


On Fri, Mar 22, 2019 at 7:52 PM Nir Soffer  wrote:

> On Fri, Mar 22, 2019 at 7:12 PM Dafna Ron  wrote:
>
>> Hi,
>>
>> We are failing ovirt-engine master on test 004_basic_sanity.hotplug_cpu
>> looking at the logs, we can see that the for some reason, libvirt reports
>> a vm as none responsive which fails the test.
>>
>> CQ first failure was for patch:
>> https://gerrit.ovirt.org/#/c/98553/ - core: Add display="on" for mdevs,
>> use nodisplay to override
>> But I do not think this is the cause of failure.
>>
>> Adding Marcin, Milan and Dan as well as I think it may be netwrok
>> related.
>>
>> You can see the libvirt log here:
>>
>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/lago-basic-suite-master-host-1/_var_log/libvirt.log
>>
>> you can see the full logs here:
>>
>>
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/
>>
>> Evgheni and I confirmed this is not an infra issue and the problem is ssh
>> connection to the internal vm
>>
>> Thanks,
>> Dafna
>>
>>
>> error:
>>
>> 2019-03-22 15:08:22.658+: 22068: warning : qemuDomainObjTaint:7521 : 
>> Domain id=3 name='vm0' uuid=a9443d02-e054-40bb-8ea3-ae346e2d02a7 is tainted: 
>> hook-script
>>
>> Why our vm is tainted?
>
> 2019-03-22 15:08:22.693+: 22068: error : 
> virProcessRunInMountNamespace:1159 : internal error: child reported: unable 
> to set security context 'system_u:object_r:virt_content_t:s0' on 
> '/rhev/data-center/mnt/blockSD/91d97292-9ac3-4d77-a152-c7ea3250b065/images/e60dae48-ecc7-4171-8bfe-42bfc2190ffd/40243c76-a384-4497-8a2d-792a5e10d510':
>  No such file or directory
>>
>> This should not happen, libvirt is not adding labels to files in
> /rhev/data-center. It is using using its own mount
> namespace and adding there the devices used by the VM. Since libvirt
> create the devices in its namespace
> it should not complain about missing paths in /rhev/data-center.
>
> I think we should file a libvirt bug for this.
>
>
>> 2019-03-22 15:08:28.168+: 22070: error : qemuDomainAgentAvailable:9133 : 
>> Guest agent is not responding: QEMU guest agent is not connected
>> 2019-03-22 15:08:58.193+: 22070: error : qemuDomainAgentAvailable:9133 : 
>> Guest agent is not responding: QEMU guest agent is not connected
>> 2019-03-22 15:13:58.179+: 22071: error : qemuDomainAgentAvailable:9133 : 
>> Guest agent is not responding: QEMU guest agent is not connected
>>
>> Do we have guest agent in the test VMs?
>
> Nir
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MKVMENZYN4YMSE5CTDQLC2PKUR4RS3CJ/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

yes. I suspect this is the problem.
In the host I can see in the yum log the times the vdsm packages are
installed:
[dron@dron post-002_bootstrap.py]$ less
lago-basic-suite-4-3-host-1/_var_log/yum.log |grep vdsm
Mar 25 07:48:49 Installed: vdsm-api-4.30.11-23.git276b602.el7.noarch
Mar 25 07:48:52 Installed: vdsm-yajsonrpc-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:00 Installed: vdsm-common-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:00 Installed:
vdsm-hook-vhostmd-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:38 Installed: vdsm-network-4.30.11-23.git276b602.el7.x86_64
Mar 25 07:49:38 Installed: vdsm-python-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:39 Installed: vdsm-client-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:39 Installed: vdsm-jsonrpc-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:39 Installed: vdsm-http-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:42 Installed:
vdsm-hook-openstacknet-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:57 Installed:
vdsm-hook-vmfex-dev-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:58 Installed: vdsm-4.30.11-23.git276b602.el7.x86_64
Mar 25 07:49:58 Installed:
vdsm-hook-ethtool-options-4.30.11-23.git276b602.el7.noarch
Mar 25 07:49:58 Installed: vdsm-hook-fcoe-4.30.11-23.git276b602.el7.noarch

and in the engine you can see that the engine already failed installation:

2019-03-25 07:48:33,920-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1)
[6baa2a84-b322-42fd-855d-b25272a19a23] Running command: AddVdsCommand
internal: false. Entities affected :  ID:
319fcdf5-5941-46b1-a75c-d335bf500f82 Type: ClusterAction group
CREATE_HOST with role type ADMIN


On Mon, Mar 25, 2019 at 1:35 PM Dafna Ron  wrote:

> I am also adding didi
> should this not be logged in host-deploy?
>
>
> On Mon, Mar 25, 2019 at 1:33 PM Marcin Sobczyk 
> wrote:
>
>> Indeed in my failed job, I can see, that 'vdsm-python' is installed
>> *after* engine's attempt to run 'vdsm-tool':
>>
>> lago-basic-suite-4-3-host-1/_var_log/yum.log
>> 403:Mar 22 17:54:12 Installed: vdsm-python-4.30.11-23.git276b602.el7.noarch
>>
>> lago-basic-suite-4-3-engine/_var_log/ovirt-engine/engine.log
>> 1580:2019-03-22 17:53:03,793-04 DEBUG 
>> [org.ovirt.engine.core.uutils.ssh.SSHClient] (default task-1) 
>> [d86d9b56-2564-44a9-a8d4-01a4d7f5a529] Executing: '/usr/bin/vdsm-tool 
>> vdsm-id'
>> 1581:2019-03-22 17:53:04,043-04 WARN  
>> [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1) 
>> [d86d9b56-2564-44a9-a8d4-01a4d7f5a529] Failed to initiate vdsm-id request on 
>> host: Command returned failure code 127 during SSH session 
>> 'root@lago-basic-suite-4-3-host-1'
>>
>>
>> On 3/25/19 2:22 PM, Dafna Ron wrote:
>>
>> and its only failing on one host as well.
>>
>>
>> On Mon, Mar 25, 2019 at 1:17 PM Marcin Sobczyk 
>> wrote:
>>
>>> Hmmm, that would suggest, that 'vdsm-tool' command is not found on the
>>> host. There was no work done around 'vdsm-tool's 'vdsm-id' recently.
>>> On 3/25/19 2:08 PM, Dafna Ron wrote:
>>>
>>> Actually, I can see in engine that it is trying to start vdsm
>>> installation on the host:
>>>
>>> 2019-03-25 07:48:33,123-04 DEBUG 
>>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) 
>>> [] Entered SsoRestApiAuthFilter
>>> 2019-03-25 07:48:33,123-04 DEBUG 
>>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) 
>>> [] SsoRestApiAuthFilter authenticating with sso
>>> 2019-03-25 07:48:33,123-04 DEBUG 
>>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) 
>>> [] SsoRestApiAuthFilter authenticating using BEARER header
>>> 2019-03-25 07:48:33,123-04 DEBUG 
>>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) 
>>> [] SsoRestApiAuthFilter successfully authenticated using BEARER header
>>> 2019-03-25 07:48:33,123-04 DEBUG 
>>> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default 
>>> task-1) [] Entered SsoRestApiNegotiationFilter
>>> 2019-03-25 07:48:33,124-04 DEBUG 
>>> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default 
>>> task-1) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth
>>> 2019-03-25 07:48:33,143-04 DEBUG 
>>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compiled stored 
>>> procedure. Call string is [{call getclusterforuserbyclustername(?, ?, ?)}]
>>> 2019-03-25 07:48:33,143-04 DEBUG 
>>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimp

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

I am also adding didi
should this not be logged in host-deploy?


On Mon, Mar 25, 2019 at 1:33 PM Marcin Sobczyk  wrote:

> Indeed in my failed job, I can see, that 'vdsm-python' is installed
> *after* engine's attempt to run 'vdsm-tool':
>
> lago-basic-suite-4-3-host-1/_var_log/yum.log
> 403:Mar 22 17:54:12 Installed: vdsm-python-4.30.11-23.git276b602.el7.noarch
>
> lago-basic-suite-4-3-engine/_var_log/ovirt-engine/engine.log
> 1580:2019-03-22 17:53:03,793-04 DEBUG 
> [org.ovirt.engine.core.uutils.ssh.SSHClient] (default task-1) 
> [d86d9b56-2564-44a9-a8d4-01a4d7f5a529] Executing: '/usr/bin/vdsm-tool vdsm-id'
> 1581:2019-03-22 17:53:04,043-04 WARN  
> [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1) 
> [d86d9b56-2564-44a9-a8d4-01a4d7f5a529] Failed to initiate vdsm-id request on 
> host: Command returned failure code 127 during SSH session 
> 'root@lago-basic-suite-4-3-host-1'
>
>
> On 3/25/19 2:22 PM, Dafna Ron wrote:
>
> and its only failing on one host as well.
>
>
> On Mon, Mar 25, 2019 at 1:17 PM Marcin Sobczyk 
> wrote:
>
>> Hmmm, that would suggest, that 'vdsm-tool' command is not found on the
>> host. There was no work done around 'vdsm-tool's 'vdsm-id' recently.
>> On 3/25/19 2:08 PM, Dafna Ron wrote:
>>
>> Actually, I can see in engine that it is trying to start vdsm
>> installation on the host:
>>
>> 2019-03-25 07:48:33,123-04 DEBUG 
>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
>> Entered SsoRestApiAuthFilter
>> 2019-03-25 07:48:33,123-04 DEBUG 
>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
>> SsoRestApiAuthFilter authenticating with sso
>> 2019-03-25 07:48:33,123-04 DEBUG 
>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
>> SsoRestApiAuthFilter authenticating using BEARER header
>> 2019-03-25 07:48:33,123-04 DEBUG 
>> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
>> SsoRestApiAuthFilter successfully authenticated using BEARER header
>> 2019-03-25 07:48:33,123-04 DEBUG 
>> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default 
>> task-1) [] Entered SsoRestApiNegotiationFilter
>> 2019-03-25 07:48:33,124-04 DEBUG 
>> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default 
>> task-1) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth
>> 2019-03-25 07:48:33,143-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compiled stored 
>> procedure. Call string is [{call getclusterforuserbyclustername(?, ?, ?)}]
>> 2019-03-25 07:48:33,143-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] SqlCall for 
>> procedure [GetClusterForUserByClusterName] compiled
>> 2019-03-25 07:48:33,145-04 DEBUG 
>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
>> (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] method: runQuery, 
>> params: [GetClusterByName, NameQueryParameters:{refresh='false', 
>> filtered='false'}], timeElapsed: 7ms
>> 2019-03-25 07:48:33,161-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [] Compiled stored procedure. Call string is [{call 
>> getvmstaticbyname(?)}]
>> 2019-03-25 07:48:33,161-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [] SqlCall for procedure [GetVmStaticByName] compiled
>> 2019-03-25 07:48:33,206-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compiled stored 
>> procedure. Call string is [{call getvdsbyname(?)}]
>> 2019-03-25 07:48:33,206-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] SqlCall for 
>> procedure [GetVdsByName] compiled
>> 2019-03-25 07:48:33,223-04 DEBUG 
>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
>> (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] method: getByName, 
>> params: [lago-basic-suite-4-3-host-1], timeElapsed: 20ms
>> 2019-03-25 07:48:33,225-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compi

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

and its only failing on one host as well.


On Mon, Mar 25, 2019 at 1:17 PM Marcin Sobczyk  wrote:

> Hmmm, that would suggest, that 'vdsm-tool' command is not found on the
> host. There was no work done around 'vdsm-tool's 'vdsm-id' recently.
> On 3/25/19 2:08 PM, Dafna Ron wrote:
>
> Actually, I can see in engine that it is trying to start vdsm installation
> on the host:
>
> 2019-03-25 07:48:33,123-04 DEBUG 
> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
> Entered SsoRestApiAuthFilter
> 2019-03-25 07:48:33,123-04 DEBUG 
> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
> SsoRestApiAuthFilter authenticating with sso
> 2019-03-25 07:48:33,123-04 DEBUG 
> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
> SsoRestApiAuthFilter authenticating using BEARER header
> 2019-03-25 07:48:33,123-04 DEBUG 
> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-1) [] 
> SsoRestApiAuthFilter successfully authenticated using BEARER header
> 2019-03-25 07:48:33,123-04 DEBUG 
> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default 
> task-1) [] Entered SsoRestApiNegotiationFilter
> 2019-03-25 07:48:33,124-04 DEBUG 
> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default 
> task-1) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth
> 2019-03-25 07:48:33,143-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compiled stored 
> procedure. Call string is [{call getclusterforuserbyclustername(?, ?, ?)}]
> 2019-03-25 07:48:33,143-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] SqlCall for 
> procedure [GetClusterForUserByClusterName] compiled
> 2019-03-25 07:48:33,145-04 DEBUG 
> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
> (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] method: runQuery, 
> params: [GetClusterByName, NameQueryParameters:{refresh='false', 
> filtered='false'}], timeElapsed: 7ms
> 2019-03-25 07:48:33,161-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [] Compiled stored procedure. Call string is [{call 
> getvmstaticbyname(?)}]
> 2019-03-25 07:48:33,161-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [] SqlCall for procedure [GetVmStaticByName] compiled
> 2019-03-25 07:48:33,206-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compiled stored 
> procedure. Call string is [{call getvdsbyname(?)}]
> 2019-03-25 07:48:33,206-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] SqlCall for 
> procedure [GetVdsByName] compiled
> 2019-03-25 07:48:33,223-04 DEBUG 
> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
> (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] method: getByName, 
> params: [lago-basic-suite-4-3-host-1], timeElapsed: 20ms
> 2019-03-25 07:48:33,225-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compiled stored 
> procedure. Call string is [{call getvdsbyhostname(?)}]
> 2019-03-25 07:48:33,225-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] SqlCall for 
> procedure [GetVdsByHostName] compiled
> 2019-03-25 07:48:33,252-04 DEBUG 
> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
> (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] method: 
> getAllForHostname, params: [lago-basic-suite-4-3-host-1], timeElapsed: 28ms
> 2019-03-25 07:48:33,254-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Compiled stored 
> procedure. Call string is [{call getstorage_poolsbyclusterid(?)}]
> 2019-03-25 07:48:33,254-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] SqlCall for 
> procedure [Getstorage_poolsByClusterId] compiled
> 2019-03-25 07:48:33,262-04 DEBUG [org.ovirt.engine.core.uutils.ssh.SSHClient] 
> (default task-1) [6baa2a84-b322-42fd-855d-b25272a19a23] Connecting 
>

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

: '/usr/bin/vdsm-tool
vdsm-id'
2019-03-25 07:48:33,902-04 WARN
[org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1)
[6baa2a84-b322-42fd-855d-b25272a19a23] Failed to initiate vdsm-id
request on host: Command returned failure code 127 during SSH session
'root@lago-basic-suite-4-3-host-1'
2019-03-25 07:48:33,920-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1)
[6baa2a84-b322-42fd-855d-b25272a19a23] Running command: AddVdsCommand
internal: false. Entities affected :  ID:
319fcdf5-5941-46b1-a75c-d335bf500f82 Type: ClusterAction group
CREATE_HOST with role type ADMIN




On Mon, Mar 25, 2019 at 12:53 PM Dafna Ron  wrote:

> we have another failing patch with the same test:
> http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/360/
>
> its obviously not related to the patch but there is something that is
> causing these failures randomly.
> From what I can see in the current failed job, you are correct and host-1
> and engine does not even try to deploy host-1.
>
> i can see host-1 is getting error 127 in lago for ntpd and I can see
> network manager errors in the host messages log
>
> In the host messages log I can see several messages that I suspect may
> cause issues in communication between engine and host:
>
> Mar 25 07:50:09 lago-basic-suite-4-3-host-1 sasldblistusers2:
> _sasldb_getkeyhandle has failed
> Mar 25 07:50:10 lago-basic-suite-4-3-host-1 saslpasswd2: error deleting
> entry from sasldb: BDB0073 DB_NOTFOUND: No matching key/data pair found
> Mar 25 07:50:10 lago-basic-suite-4-3-host-1 saslpasswd2: error deleting
> entry from sasldb: BDB0073 DB_NOTFOUND: No matching key/data pair found
> Mar 25 07:50:10 lago-basic-suite-4-3-host-1 saslpasswd2: error deleting
> entry from sasldb: BDB0073 DB_NOTFOUND: No matching key/data pair found
>
>
> ces/12)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bonding:
> bondscan-UMJa2S is being created...
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bonding:
> bondscan-UMJa2S is being deleted...
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 NetworkManager[2658]: 
> [1553514613.7774] manager: (bondscan-UMJa2S): new Bond device
> (/org/freedesktop/NetworkManager/Devices/13)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: (unregistered
> net_device) (unregistering): Released all slaves
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bonding:
> bondscan-liwvMR is being created...
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option fail_over_mac: invalid value (3)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option ad_select: invalid value (3)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option lacp_rate: mode dependency failed, not supported in mode
> balance-rr(0)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option arp_validate: invalid value (7)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option xmit_hash_policy: invalid value (5)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 NetworkManager[2658]: 
> [1553514613.8002] manager: (bondscan-liwvMR): new Bond device
> (/org/freedesktop/NetworkManager/Devices/14)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option primary_reselect: invalid value (3)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option arp_all_targets: invalid value (2)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bonding:
> bondscan-liwvMR is being deleted...
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: (unregistered
> net_device) (unregistering): Released all slaves
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bonding:
> bondscan-liwvMR is being created...
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option fail_over_mac: invalid value (3)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option ad_select: invalid value (3)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option lacp_rate: mode dependency failed, not supported in mode
> active-backup(1)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option arp_validate: invalid value (7)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 NetworkManager[2658]: 
> [1553514613.8429] manager: (bondscan-liwvMR): new Bond device
> (/org/freedesktop/NetworkManager/Devices/15)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option xmit_hash_policy: invalid value (5)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option primary_reselect: invalid value (3)
> Mar 25 07:50:13 lago-basic-suite-4-3-host-1 kernel: bondscan-liwvMR:
> option arp_all_targe

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

sic-suite-4-3-host-1 returned with 127
> 2019-03-25 10:14:46,384::ssh.py::ssh::96::lago.ssh::DEBUG::Command d0c49b54 
> on lago-basic-suite-4-3-host-1  errors:
>  bash: ntpdate: command not found
>
> On host-0 everything is ok:
>
> 2019-03-25 10:14:46,917::ssh.py::ssh::58::lago.ssh::DEBUG::Running d11b2a64 
> on lago-basic-suite-4-3-host-0: ntpdate -4 lago-basic-suite-4-3-engine
> 2019-03-25 10:14:53,088::ssh.py::ssh::81::lago.ssh::DEBUG::Command d11b2a64 
> on lago-basic-suite-4-3-host-0 returned with 0
> 2019-03-25 10:14:53,088::ssh.py::ssh::89::lago.ssh::DEBUG::Command d11b2a64 
> on lago-basic-suite-4-3-host-0 output:
>  25 Mar 06:14:53 ntpdate[6646]: adjust time server 192.168.202.2 offset 
> 0.017150 sec
>
> On 3/25/19 10:13 AM, Eyal Edri wrote:
>
> Still fails, now on a different component. ( ovirt-web-ui-extentions )
>
> https://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/339/
>
> On Fri, Mar 22, 2019 at 3:59 PM Dan Kenigsberg  wrote:
>
>>
>>
>> On Fri, Mar 22, 2019 at 3:21 PM Marcin Sobczyk 
>> wrote:
>>
>>> Dafna,
>>>
>>> in 'verify_add_hosts' we specifically wait for single host to be up with
>>> a timeout:
>>>
>>>  144 up_hosts = hosts_service.list(search='datacenter={} AND 
>>> status=up'.format(DC_NAME))
>>>  145 if len(up_hosts):
>>>  146 return True
>>>
>>> The log files say, that it took ~50 secs for one of the hosts to be up
>>> (seems reasonable) and no timeout is being reported.
>>> Just after running 'verify_add_hosts', we run
>>> 'add_master_storage_domain', which calls '_hosts_in_dc' function.
>>> That function does the exact same check, but it fails:
>>>
>>>  113 hosts = hosts_service.list(search='datacenter={} AND 
>>> status=up'.format(dc_name))
>>>  114 if hosts:
>>>  115 if random_host:
>>>  116 return random.choice(hosts)
>>>
>>> I don't think it is relevant to our current failure; but I consider
>> random_host=True as a bad practice. As if we do not have enough moving
>> parts, we are adding intentional randomness. Reproducibility is far more
>> important than coverage - particularly for a shared system test like OST.
>>
>>>  117 else:
>>>  118 return sorted(hosts, key=lambda host: host.name)
>>>  119 raise RuntimeError('Could not find hosts that are up in DC %s' % 
>>> dc_name)
>>>
>>>
>>> I'm also not able to reproduce this issue locally on my server. The
>>> investigation continues...
>>>
>>
>> I think that it would be fair to take the filtering by host state out of
>> Engine and into the test, where we can easily log the current status of
>> each host. Then we'd have better understanding on the next failure.
>>
>> On 3/22/19 1:17 PM, Marcin Sobczyk wrote:
>>>
>>> Hi,
>>>
>>> sure, I'm on it - it's weird though, I did ran 4.3 basic suite for this
>>> patch manually and everything was ok.
>>> On 3/22/19 1:05 PM, Dafna Ron wrote:
>>>
>>> Hi,
>>>
>>> We are failing branch 4.3 for test:
>>> 002_bootstrap.add_master_storage_domain
>>>
>>> It seems that in one of the hosts, the vdsm is not starting
>>> there is nothing in vdsm.log or in supervdsm.log
>>>
>>> CQ identified this patch as the suspected root cause:
>>>
>>> https://gerrit.ovirt.org/#/c/98748/ - vdsm: client: Add support for
>>> flow id
>>>
>>> Milan, Marcin, can you please have a look?
>>>
>>> full logs:
>>>
>>>
>>> http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/326/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.3/post-002_bootstrap.py/
>>>
>>> the only error I can see is about host not being up (makes sense as vdsm
>>> is not running)
>>>
>>> Stacktrace
>>>
>>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>>> testMethod()
>>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
>>> self.test(*self.arg)
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, 
>>> in wrapped_test
>>> test()
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in 
>>> wrapper
>>> return func(get_test_prefix(), *args, **kwargs)
>>>   File 
>>> "/home/jenkins/wor

[ovirt-devel] [Ovirt] [CQ weekly status] [22-03-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last CQ job failure on 4.2 was 21-03-2019 on project
ovirt-hosted-engine-setup due to access to repo centos-cr-el7 in mirrors.
this can be solved by projects setting their repo list to work with CI
local mirrors.

*CQ-4.3*:  RED (#1)

vdsm is failing on 4.3 due to failure in add_master_storage_domain. this
issue is currently investigated by Marcin Sobczyk

*CQ-Master:*   RED (#1)

we are failing on project ovirt-engine for test hotplug_cpu. this was
debugged by Evgheni and myself to determine that the issue is indeed in
ovirt and not in infra.
The test is failing as the internal vm is not responding and we cannot ssh
to vm0.

*There is a random failure for test get_host_hooks. *
It fails on all branches and I can see a NPE in the engine log.
The test owner was Yaniv Kaul and Marcin Sobczyk is now looking into the
failure.
I created a skiptest patch but I am hoping that this would be resolved soon
so we do not have to use it.


 Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found
here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.3_change-queue-tester/

[3]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/X6SS2FBC3MR4TNWLJVGRRFGX4HO5MFOK/

[ovirt-devel] [ OST Failure Report ] [ oVirt master (ovirt-engine) ] [ 22-03-2019 ] [004_basic_sanity.hotplug_cpu ]

[Dafna Ron]

Hi,

We are failing ovirt-engine master on test 004_basic_sanity.hotplug_cpu
looking at the logs, we can see that the for some reason, libvirt reports a
vm as none responsive which fails the test.

CQ first failure was for patch:
https://gerrit.ovirt.org/#/c/98553/ - core: Add display="on" for mdevs, use
nodisplay to override
But I do not think this is the cause of failure.

Adding Marcin, Milan and Dan as well as I think it may be netwrok related.

You can see the libvirt log here:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/lago-basic-suite-master-host-1/_var_log/libvirt.log

you can see the full logs here:

http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-004_basic_sanity.py/

Evgheni and I confirmed this is not an infra issue and the problem is ssh
connection to the internal vm

Thanks,
Dafna


error:

2019-03-22 15:08:22.658+: 22068: warning : qemuDomainObjTaint:7521
: Domain id=3 name='vm0' uuid=a9443d02-e054-40bb-8ea3-ae346e2d02a7 is
tainted: hook-script
2019-03-22 15:08:22.693+: 22068: error :
virProcessRunInMountNamespace:1159 : internal error: child reported:
unable to set security context 'system_u:object_r:virt_content_t:s0'
on 
'/rhev/data-center/mnt/blockSD/91d97292-9ac3-4d77-a152-c7ea3250b065/images/e60dae48-ecc7-4171-8bfe-42bfc2190ffd/40243c76-a384-4497-8a2d-792a5e10d510':
No such file or directory
2019-03-22 15:08:28.168+: 22070: error :
qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU
guest agent is not connected
2019-03-22 15:08:58.193+: 22070: error :
qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU
guest agent is not connected
2019-03-22 15:13:58.179+: 22071: error :
qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU
guest agent is not connected
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/Z77IVTBQQMZT2SGFNCK72E46O5DEEYKE/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

I wonder if there were any changes that merged around the same time that
could have caused it?

On Fri, Mar 22, 2019 at 12:17 PM Marcin Sobczyk  wrote:

> Hi,
>
> sure, I'm on it - it's weird though, I did ran 4.3 basic suite for this
> patch manually and everything was ok.
> On 3/22/19 1:05 PM, Dafna Ron wrote:
>
> Hi,
>
> We are failing branch 4.3 for test: 002_bootstrap.add_master_storage_domain
>
> It seems that in one of the hosts, the vdsm is not starting
> there is nothing in vdsm.log or in supervdsm.log
>
> CQ identified this patch as the suspected root cause:
>
> https://gerrit.ovirt.org/#/c/98748/ - vdsm: client: Add support for flow
> id
>
> Milan, Marcin, can you please have a look?
>
> full logs:
>
>
> http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/326/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.3/post-002_bootstrap.py/
>
> the only error I can see is about host not being up (makes sense as vdsm
> is not running)
>
> Stacktrace
>
>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
> testMethod()
>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
> self.test(*self.arg)
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in 
> wrapped_test
> test()
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in 
> wrapper
> return func(get_test_prefix(), *args, **kwargs)
>   File 
> "/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
>  line 417, in add_master_storage_domain
> add_iscsi_storage_domain(prefix)
>   File 
> "/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
>  line 561, in add_iscsi_storage_domain
> host=_random_host_from_dc(api, DC_NAME),
>   File 
> "/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
>  line 122, in _random_host_from_dc
> return _hosts_in_dc(api, dc_name, True)
>   File 
> "/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
>  line 119, in _hosts_in_dc
> raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name)
> 'Could not find hosts that are up in DC test-dc\n >> 
> begin captured logging << \nlago.ssh: DEBUG: start 
> task:937bdea7-a2a3-47ad-9383-36647ea37ddf:Get ssh client for 
> lago-basic-suite-4-3-engine:\nlago.ssh: DEBUG: end 
> task:937bdea7-a2a3-47ad-9383-36647ea37ddf:Get ssh client for 
> lago-basic-suite-4-3-engine:\nlago.ssh: DEBUG: Running c07b5ee2 on 
> lago-basic-suite-4-3-engine: cat /root/multipath.txt\nlago.ssh: DEBUG: 
> Command c07b5ee2 on lago-basic-suite-4-3-engine returned with 0\nlago.ssh: 
> DEBUG: Command c07b5ee2 on lago-basic-suite-4-3-engine output:\n 
> 3600140516f88cafa71243648ea218995\n360014053e28f60001764fed9978ec4b3\n360014059edc70114a6484891dcf1\n36001405d93d8585a50d43a4ad0bd8d19\n36001405e31361631de14bcf87d43e55a\n\n---
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DEOFT54GR3YI43T5SSL33NQ6OULRS7SV/

[ovirt-devel] [ OST Failure Report ] [ oVirt 4.3 (vdsm) ] [ 22-03-2019 ] [ 002_bootstrap.add_master_storage_domain ]

[Dafna Ron]

Hi,

We are failing branch 4.3 for test: 002_bootstrap.add_master_storage_domain

It seems that in one of the hosts, the vdsm is not starting
there is nothing in vdsm.log or in supervdsm.log

CQ identified this patch as the suspected root cause:

https://gerrit.ovirt.org/#/c/98748/ - vdsm: client: Add support for flow id

Milan, Marcin, can you please have a look?

full logs:

http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/326/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-4.3/post-002_bootstrap.py/

the only error I can see is about host not being up (makes sense as vdsm is
not running)

Stacktrace

  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
142, in wrapped_test
test()
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
60, in wrapper
return func(get_test_prefix(), *args, **kwargs)
  File 
"/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
line 417, in add_master_storage_domain
add_iscsi_storage_domain(prefix)
  File 
"/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
line 561, in add_iscsi_storage_domain
host=_random_host_from_dc(api, DC_NAME),
  File 
"/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
line 122, in _random_host_from_dc
return _hosts_in_dc(api, dc_name, True)
  File 
"/home/jenkins/workspace/ovirt-4.3_change-queue-tester/ovirt-system-tests/basic-suite-4.3/test-scenarios/002_bootstrap.py",
line 119, in _hosts_in_dc
raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name)
'Could not find hosts that are up in DC test-dc\n
>> begin captured logging << \nlago.ssh: DEBUG:
start task:937bdea7-a2a3-47ad-9383-36647ea37ddf:Get ssh client for
lago-basic-suite-4-3-engine:\nlago.ssh: DEBUG: end
task:937bdea7-a2a3-47ad-9383-36647ea37ddf:Get ssh client for
lago-basic-suite-4-3-engine:\nlago.ssh: DEBUG: Running c07b5ee2 on
lago-basic-suite-4-3-engine: cat /root/multipath.txt\nlago.ssh: DEBUG:
Command c07b5ee2 on lago-basic-suite-4-3-engine returned with
0\nlago.ssh: DEBUG: Command c07b5ee2 on lago-basic-suite-4-3-engine
output:\n 
3600140516f88cafa71243648ea218995\n360014053e28f60001764fed9978ec4b3\n360014059edc70114a6484891dcf1\n36001405d93d8585a50d43a4ad0bd8d19\n36001405e31361631de14bcf87d43e55a\n\n---
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ERZDWL44JD52IRCS4MSHZNI2UJT2JPOA/

[ovirt-devel] Re: [NOTICE] Failing randomly on get_host_hooks with a NullPointerException

[Dafna Ron]

Hi Dan,

I could not agree more that skipping a test is bad.
This has been reported several times and all managers and developers are on
the mailing list so if they choose to ignore it and not take ownership, and
the test is effecting all other projects, we will have no choice.
Martin is on the mail and can take ownership at any point.

build is:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13508/

Thanks,
Dafna


On Fri, Mar 22, 2019 at 9:52 AM Dan Kenigsberg  wrote:

> Yes, I'm repeating myself.
> SKIPPING TESTS IS BAD
>
> We have a test suite in order to fix bugs, not in order to kill itself.
>
> Host hooks are Infra. Infra is mperina, rnori and msobczik.
> Please consult with them before you shut our collective eyes.
>
> Please point them to a failing job, and record the failing traceback.
>
> On Fri, 22 Mar 2019, 11:27 Dafna Ron,  wrote:
>
>> patch submitted: https://gerrit.ovirt.org/#/c/98773/
>>
>> Thanks,
>> Dafna
>>
>>
>> On Fri, Mar 22, 2019 at 9:04 AM Sandro Bonazzola 
>> wrote:
>>
>>>
>>>
>>> Il giorno ven 22 mar 2019 alle ore 09:34 Dafna Ron  ha
>>> scritto:
>>>
>>>> Hi,
>>>>
>>>> we are randomly failing on get_host_hooks test for at least 3 weeks.
>>>> its not a specific branch or project and there are no commonalities
>>>> that I can see, aside from not being able to communicate with the host.
>>>>
>>>> this week its started happening at least once a day (this morning, 2
>>>> out of 3 failures were due to that test).
>>>>
>>>> This test has been added by Yaniv Kaul over a year ago and he is no
>>>> longer working on ovirt I think someone else should take ownership of this
>>>> test and fix it.
>>>> Please let me know if you are intending to investigate and either fix
>>>> the failure or fix the test if not I will add a skip to the test,
>>>>
>>>
>>> Please add a skip to the test and if someone will step in maintaining
>>> this test it will be re-enabled.
>>>
>>>
>>>
>>>>
>>>> Thanks,
>>>> Dafna
>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>> SANDRO BONAZZOLA
>>>
>>> MANAGER, SOFTWARE ENGINEERING, EMEA R RHV
>>>
>>> Red Hat EMEA <https://www.redhat.com/>
>>>
>>> sbona...@redhat.com
>>> <https://red.ht/sig>
>>>
>>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YH5DHXGHH2XZDND43NMRIKQKFTAXD6T4/

[ovirt-devel] Re: [NOTICE] Failing randomly on get_host_hooks with a NullPointerException

[Dafna Ron]

patch submitted: https://gerrit.ovirt.org/#/c/98773/

Thanks,
Dafna


On Fri, Mar 22, 2019 at 9:04 AM Sandro Bonazzola 
wrote:

>
>
> Il giorno ven 22 mar 2019 alle ore 09:34 Dafna Ron  ha
> scritto:
>
>> Hi,
>>
>> we are randomly failing on get_host_hooks test for at least 3 weeks.
>> its not a specific branch or project and there are no commonalities that
>> I can see, aside from not being able to communicate with the host.
>>
>> this week its started happening at least once a day (this morning, 2 out
>> of 3 failures were due to that test).
>>
>> This test has been added by Yaniv Kaul over a year ago and he is no
>> longer working on ovirt I think someone else should take ownership of this
>> test and fix it.
>> Please let me know if you are intending to investigate and either fix the
>> failure or fix the test if not I will add a skip to the test,
>>
>
> Please add a skip to the test and if someone will step in maintaining this
> test it will be re-enabled.
>
>
>
>>
>> Thanks,
>> Dafna
>>
>>
>>
>
> --
>
> SANDRO BONAZZOLA
>
> MANAGER, SOFTWARE ENGINEERING, EMEA R RHV
>
> Red Hat EMEA <https://www.redhat.com/>
>
> sbona...@redhat.com
> <https://red.ht/sig>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/X53SD6CUBGV6CNFQYG4Q5EFNVC27Q37V/

[ovirt-devel] [NOTICE] Failing randomly on get_host_hooks with a NullPointerException

[Dafna Ron]

Hi,

we are randomly failing on get_host_hooks test for at least 3 weeks.
its not a specific branch or project and there are no commonalities that I
can see, aside from not being able to communicate with the host.

this week its started happening at least once a day (this morning, 2 out of
3 failures were due to that test).

This test has been added by Yaniv Kaul over a year ago and he is no longer
working on ovirt I think someone else should take ownership of this test
and fix it.
Please let me know if you are intending to investigate and either fix the
failure or fix the test if not I will add a skip to the test,

Thanks,
Dafna
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/WX4KD2X4FHTMCUDX7SEGZBUUF6RAU2GY/

[ovirt-devel] [Ovirt] [CQ weekly status] [15-03-2019]

[Dafna Ron]

Hi,

This mail is to provide the current status of CQ and allow people to review
status before and after the weekend.
Please refer to below colour map for further information on the meaning of
the colours.

*CQ-4.2*:  GREEN (#1)

Last CQ job failure on 4.2 was 14-03-2019 on project v2v-conversion-host
due to missing failed build artifacts which was fixed the same day.

*CQ-4.3*:  RED (#1)

Last job to failed today. it had 13 changes and CQ is still bisecting the
result.

*CQ-Master:*   RED (#1)

Last failure was today due to a build-artifacts failure. a message was sent
to patch owner.

 Current running jobs for 4.2 [1] and master [2] can be found here:

[1]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-queue-tester/

[2]
http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/

Happy week!
Dafna


---
COLOUR MAP

Green = job has been passing successfully

** green for more than 3 days may suggest we need a review of our test
coverage


   1.

   1-3 days   GREEN (#1)
   2.

   4-7 days   GREEN (#2)
   3.

   Over 7 days GREEN (#3)


Yellow = intermittent failures for different projects but no lasting or
current regressions

** intermittent would be a healthy project as we expect a number of
failures during the week

** I will not report any of the solved failures or regressions.


   1.

   Solved job failuresYELLOW (#1)
   2.

   Solved regressions  YELLOW (#2)


Red = job has been failing

** Active Failures. The colour will change based on the amount of time the
project/s has been broken. Only active regressions would be reported.


   1.

   1-3 days  RED (#1)
   2.

   4-7 days  RED (#2)
   3.

   Over 7 days RED (#3)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/SH57CDHCMRYL5PGQFP3E4KYATT7TFKB3/