On Fri, Dec 20, 2019 at 11:00 AM Martin Perina <[email protected]> wrote:
>
>
>
> On Thu, Dec 19, 2019 at 11:49 AM Yedidyah Bar David <[email protected]> wrote:
>>
>> On Thu, Dec 19, 2019 at 3:07 AM oVirt Jenkins <[email protected]> wrote:
>> >
>> > Change 105811,4 (ovirt-engine) is probably the reason behind recent system 
>> > test
>> > failures in the "ovirt-master" change queue and needs to be fixed.
>> >
>> > This change had been removed from the testing queue. Artifacts build from 
>> > this
>> > change will not be released until it is fixed.
>> >
>> > For further details about the change see:
>> > https://gerrit.ovirt.org/#/c/105811/4
>> >
>> > For failed test results see:
>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17803/
>>
>> lago log [1] has:
>>
>> 2019-12-19 01:03:18,289::ssh.py::ssh::58::lago.ssh::DEBUG::Running
>> 57dde5d6 on lago-basic-suite-master-engine: /usr/bin/systemctl stop
>> ovirt-engine
>> 2019-12-19 01:03:20,379::ssh.py::ssh::81::lago.ssh::DEBUG::Command
>> 57dde5d6 on lago-basic-suite-master-engine returned with 0
>> 2019-12-19 01:03:20,381::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
>> task:b30be6a0-81f5-4611-b638-2cc46f8dd28f:Get ssh client for
>> lago-basic-suite-master-engine:
>> 2019-12-19 01:03:20,512::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
>> task:b30be6a0-81f5-4611-b638-2cc46f8dd28f:Get ssh client for
>> lago-basic-suite-master-engine:
>> 2019-12-19 01:03:20,777::ssh.py::ssh::58::lago.ssh::DEBUG::Running
>> 5959a5f8 on lago-basic-suite-master-engine: /usr/bin/systemctl status
>> --lines=0 ovirt-engine
>> 2019-12-19 01:03:20,828::ssh.py::ssh::81::lago.ssh::DEBUG::Command
>> 5959a5f8 on lago-basic-suite-master-engine returned with 3
>> 2019-12-19 01:03:20,828::ssh.py::ssh::89::lago.ssh::DEBUG::Command
>> 5959a5f8 on lago-basic-suite-master-engine output:
>>  â—  ovirt-engine.service - oVirt Engine
>>    Loaded: loaded (/usr/lib/systemd/system/ovirt-engine.service;
>> enabled; vendor preset: disabled)
>>    Active: inactive (dead) since Wed 2019-12-18 20:03:20 EST; 455ms ago
>>   Process: 24166
>> ExecStart=/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.py
>> --redirect-output --systemd=notify $EXTRA_ARGS start (code=exited,
>> status=0/SUCCESS)
>>  Main PID: 24166 (code=exited, status=0/SUCCESS)
>>
>> 2019-12-19 01:03:20,829::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
>> task:6b4dab7c-64f3-43e5-ae85-2fe6246d9376:Get ssh client for
>> lago-basic-suite-master-engine:
>> 2019-12-19 01:03:20,998::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
>> task:6b4dab7c-64f3-43e5-ae85-2fe6246d9376:Get ssh client for
>> lago-basic-suite-master-engine:
>> 2019-12-19 01:03:21,214::ssh.py::ssh::58::lago.ssh::DEBUG::Running
>> 599c5b96 on lago-basic-suite-master-engine: /usr/bin/systemctl start
>> ovirt-engine
>> 2019-12-19 01:04:13,735::ssh.py::ssh::81::lago.ssh::DEBUG::Command
>> 599c5b96 on lago-basic-suite-master-engine returned with 0
>> 2019-12-19 01:04:13,737::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
>> task:5b775d53-c53c-4d15-9cae-2806c44079b7:Get ssh client for
>> lago-basic-suite-master-engine:
>> 2019-12-19 01:04:13,971::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
>> task:5b775d53-c53c-4d15-9cae-2806c44079b7:Get ssh client for
>> lago-basic-suite-master-engine:
>> 2019-12-19 01:04:14,488::ssh.py::ssh::58::lago.ssh::DEBUG::Running
>> 795d5098 on lago-basic-suite-master-engine: /usr/bin/systemctl status
>> --lines=0 ovirt-engine
>> 2019-12-19 01:04:15,039::ssh.py::ssh::81::lago.ssh::DEBUG::Command
>> 795d5098 on lago-basic-suite-master-engine returned with 0
>> 2019-12-19 01:04:15,040::ssh.py::ssh::89::lago.ssh::DEBUG::Command
>> 795d5098 on lago-basic-suite-master-engine output:
>>  â—  ovirt-engine.service - oVirt Engine
>>    Loaded: loaded (/usr/lib/systemd/system/ovirt-engine.service;
>> enabled; vendor preset: disabled)
>>    Active: active (running) since Wed 2019-12-18 20:04:13 EST; 1s ago
>>  Main PID: 5718 (ovirt-engine.py)
>>    CGroup: /system.slice/ovirt-engine.service
>>            ├─5718 /usr/bin/python2
>> /usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.py
>> --redirect-output --systemd=notify start
>>            └─6073 ovirt-engine --add-modules java.se -server
>> -XX:+TieredCompilation -Xms1024M -Xmx1024M -Xss1M
>> -Djava.awt.headless=true -Dsun.rmi.dgc.client.gcInterval=3600000
>> -Dsun.rmi.dgc.server.gcInterval=3600000
>> -Djsse.enableSNIExtension=false -Dresteasy.preferJacksonOverJsonB=true
>> -Djackson.deserialization.whitelist.packages=org,com,java,javax
>> -XX:+HeapDumpOnOutOfMemoryError
>> -XX:HeapDumpPath=/var/log/ovirt-engine/dump
>> -Djava.util.logging.manager=org.jboss.logmanager
>> -Dlogging.configuration=file:///var/lib/ovirt-engine/jboss_runtime/config/ovirt-engine-logging.properties
>> -Dorg.jboss.resolver.warning=true
>> -Djboss.modules.system.pkgs=org.jboss.byteman
>> -Djboss.server.default.config=ovirt-engine
>> -Djboss.home.dir=/usr/share/ovirt-engine-wildfly
>> -Djboss.server.base.dir=/usr/share/ovirt-engine
>> -Djboss.server.data.dir=/var/lib/ovirt-engine
>> -Djboss.server.log.dir=/var/log/ovirt-engine
>> -Djboss.server.config.dir=/var/lib/ovirt-engine/jboss_runtime/config
>> -Djboss.server.temp.dir=/var/lib/ovirt-engine/jboss_runtime/tmp
>> -Djboss.controller.temp.dir=/var/lib/ovirt-engine/jboss_runtime/tmp
>> -jar /usr/share/ovirt-engine-wildfly/jboss-modules.jar -mp
>> /usr/share/ovirt-engine-wildfly-overlay/modules:/usr/share/ovirt-engine/modules/common:/usr/share/ovirt-engine-extension-aaa-jdbc/modules:/usr/share/ovirt-engine-extension-aaa-ldap/modules:/usr/share/ovirt-engine-wildfly/modules
>> -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -c
>> ovirt-engine.xml
>>
>> 2019-12-19 01:04:15,040::log_utils.py::end_log_task::670::nose::INFO::
>>  # add_ldap_provider:  [32mSuccess [0m (in 0:01:08)
>> 2019-12-19 01:05:12,552::log_utils.py::start_log_task::655::nose::INFO::
>>  # Failure: RuntimeError (test api call failed):  [0m [0m
>>
>> So probably the engine didn't manage to start yet and connecting to
>> its api failed [2]. Perhaps wait, after starting it, until it's really
>> up (checking the api or health)?
>
>
> Maybe, but above change should not affect anyhow engine startup. And also 
> newer CQ tests were green:
>
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17808/
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17810/
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17811/
>
> So it seems to me as some single occasion env failure ...

The fact that starting the engine and then immediately trying to use it
does not always work, is definitely not a single occasion:

https://bugzilla.redhat.com/show_bug.cgi?id=1075013

>>
>>
>> [1] 
>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17803/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-099_aaa-ldap.py/lago_logs/lago.log/*view*/
>> [2] 
>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17803/testReport/junit/nose.failure/Failure/runTest/
>>
>> > _______________________________________________
>> > Infra mailing list -- [email protected]
>> > To unsubscribe send an email to [email protected]
>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> > oVirt Code of Conduct: 
>> > https://www.ovirt.org/community/about/community-guidelines/
>> > List Archives: 
>> > https://lists.ovirt.org/archives/list/[email protected]/message/P3BLXLG63LC7PNFGYW6UI6H3R7A6XNV3/
>>
>>
>>
>> --
>> Didi
>>
>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.



-- 
Didi
_______________________________________________
Devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/[email protected]/message/I4CTFF4RQQO5Y4UH2LOOQEVOY6BFCVB3/

Reply via email to