Re: ost host addition failure

2016-12-27 Thread Yaniv Kaul 
On Dec 27, 2016 3:01 PM, "Gil Shinar"  wrote:

After the following fix had been merged, we still had an issue with vm_run
but it had been fixed as well.
Master experimental is now working properly.


Excellent news!
1. Can we publish it?
2. 4.1 branch?
Y.


Thanks Dan
Gil

On Tue, Dec 27, 2016 at 10:24 AM, Dan Kenigsberg  wrote:

> On Tue, Dec 27, 2016 at 9:59 AM, Eyal Edri  wrote:
> >
> >
> > On Tue, Dec 27, 2016 at 9:56 AM, Eyal Edri  wrote:
> >>
> >> Any updates?
> >> The tests are still failing on vdsmd won't start from Sunday... master
> >> repos havn't been refreshed for a few days due to this.
> >>
> >> from host deploy log: [1]
> >> basic-suite-master-engine/_var_log_ovirt-engine/host-deploy/
> ovirt-host-deploy-20161227012930-192.168.201.4-14af2bf0.log
> >> the job links [2]
> >>
> >>
> >>
> >>
> >>
> >> [1]
> >> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma
> ster/lastCompletedBuild/artifact/exported-artifacts/basic_
> suite_master.sh-el7/exported-artifacts/test_logs/basic-
> suite-master/post-002_bootstrap.py/lago-
> >
> >
> > Now with the full link:
> > http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma
> ster/lastCompletedBuild/artifact/exported-artifacts/basic_
> suite_master.sh-el7/exported-artifacts/test_logs/basic-
> suite-master/post-002_bootstrap.py/lago-basic-suite-master-
> engine/_var_log_ovirt-engine/host-deploy/ovirt-host-deploy-
> 20161227012930-192.168.201.4-14af2bf0.log
> >
> >>
> >>
> >>
> >>
> >> 016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
> >> plugin.execute:921 execute-output: ('/bin/systemctl', 'start',
> >> 'vdsmd.service') stdout:
> >>
> >> 2016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
> >> plugin.execute:926 execute-output: ('/bin/systemctl', 'start',
> >> 'vdsmd.service') stderr:
> >> A dependency job for vdsmd.service failed. See 'journalctl -xe' for
> >> details.
> >>
> >> 2016-12-27 01:29:29 DEBUG otopi.context context._executeMethod:142
> method
> >> exception
> >> Traceback (most recent call last):
> >>   File "/tmp/ovirt-QZ1ucxWFfm/pythonlib/otopi/context.py", line 132, in
> >> _executeMethod
> >> method['method']()
> >>   File
> >> "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/ovirt-host-deploy/vdsm/
> packages.py",
> >> line 209, in _start
> >> self.services.state('vdsmd', True)
> >>   File "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/otopi/services/systemd.py",
> >> line 141, in state
> >> service=name,
> >> RuntimeError: Failed to start service 'vdsmd'
> >> 2016-12-27 01:29:29 ERROR otopi.context context._executeMethod:151
> Failed
> >> to execute stage 'Closing up': Failed to start service 'vdsmd'
> >> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:760
> >> ENVIRONMENT DUMP - BEGIN
> >> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV
> >> BASE/error=bool:'True'
> >> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV
> >> BASE/excep
> >>
> >> [2]
> >> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma
> ster/lastCompletedBuild/testReport/
>
>
> In the log I see
>
> Processing package vdsm-4.20.0-7.gitf851d1b.el7.centos.x86_64
>
> which is from Dec 22 (last Thursday). This is because of us missing a
> master-branch tag. v4.20.0 wrongly tagged on the same commit as that
> of v4.19.1, removed, and never placed properly.
>
> I've re-pushed v4.20.0 properly, and now merged a patch to trigger
> build-artifacts in master.
> http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/1544/
>
> When this is done, could you use it to take the artifacts and try again?
>
> Regards,
> Dan.
>
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra

Re: ost host addition failure

2016-12-27 Thread Gil Shinar 
After the following fix had been merged, we still had an issue with vm_run
but it had been fixed as well.
Master experimental is now working properly.

Thanks Dan
Gil

On Tue, Dec 27, 2016 at 10:24 AM, Dan Kenigsberg  wrote:

> On Tue, Dec 27, 2016 at 9:59 AM, Eyal Edri  wrote:
> >
> >
> > On Tue, Dec 27, 2016 at 9:56 AM, Eyal Edri  wrote:
> >>
> >> Any updates?
> >> The tests are still failing on vdsmd won't start from Sunday... master
> >> repos havn't been refreshed for a few days due to this.
> >>
> >> from host deploy log: [1]
> >> basic-suite-master-engine/_var_log_ovirt-engine/host-
> deploy/ovirt-host-deploy-20161227012930-192.168.201.4-14af2bf0.log
> >> the job links [2]
> >>
> >>
> >>
> >>
> >>
> >> [1]
> >> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
> master/lastCompletedBuild/artifact/exported-artifacts/
> basic_suite_master.sh-el7/exported-artifacts/test_logs/
> basic-suite-master/post-002_bootstrap.py/lago-
> >
> >
> > Now with the full link:
> > http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
> master/lastCompletedBuild/artifact/exported-artifacts/
> basic_suite_master.sh-el7/exported-artifacts/test_logs/
> basic-suite-master/post-002_bootstrap.py/lago-basic-suite-
> master-engine/_var_log_ovirt-engine/host-deploy/ovirt-host-
> deploy-20161227012930-192.168.201.4-14af2bf0.log
> >
> >>
> >>
> >>
> >>
> >> 016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
> >> plugin.execute:921 execute-output: ('/bin/systemctl', 'start',
> >> 'vdsmd.service') stdout:
> >>
> >> 2016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
> >> plugin.execute:926 execute-output: ('/bin/systemctl', 'start',
> >> 'vdsmd.service') stderr:
> >> A dependency job for vdsmd.service failed. See 'journalctl -xe' for
> >> details.
> >>
> >> 2016-12-27 01:29:29 DEBUG otopi.context context._executeMethod:142
> method
> >> exception
> >> Traceback (most recent call last):
> >>   File "/tmp/ovirt-QZ1ucxWFfm/pythonlib/otopi/context.py", line 132, in
> >> _executeMethod
> >> method['method']()
> >>   File
> >> "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/ovirt-host-deploy/
> vdsm/packages.py",
> >> line 209, in _start
> >> self.services.state('vdsmd', True)
> >>   File "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/otopi/services/systemd.py",
> >> line 141, in state
> >> service=name,
> >> RuntimeError: Failed to start service 'vdsmd'
> >> 2016-12-27 01:29:29 ERROR otopi.context context._executeMethod:151
> Failed
> >> to execute stage 'Closing up': Failed to start service 'vdsmd'
> >> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:760
> >> ENVIRONMENT DUMP - BEGIN
> >> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV
> >> BASE/error=bool:'True'
> >> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV
> >> BASE/excep
> >>
> >> [2]
> >> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
> master/lastCompletedBuild/testReport/
>
>
> In the log I see
>
> Processing package vdsm-4.20.0-7.gitf851d1b.el7.centos.x86_64
>
> which is from Dec 22 (last Thursday). This is because of us missing a
> master-branch tag. v4.20.0 wrongly tagged on the same commit as that
> of v4.19.1, removed, and never placed properly.
>
> I've re-pushed v4.20.0 properly, and now merged a patch to trigger
> build-artifacts in master.
> http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/1544/
>
> When this is done, could you use it to take the artifacts and try again?
>
> Regards,
> Dan.
>
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra

Re: ost host addition failure

2016-12-27 Thread Dan Kenigsberg 
On Tue, Dec 27, 2016 at 9:59 AM, Eyal Edri  wrote:
>
>
> On Tue, Dec 27, 2016 at 9:56 AM, Eyal Edri  wrote:
>>
>> Any updates?
>> The tests are still failing on vdsmd won't start from Sunday... master
>> repos havn't been refreshed for a few days due to this.
>>
>> from host deploy log: [1]
>> basic-suite-master-engine/_var_log_ovirt-engine/host-deploy/ovirt-host-deploy-20161227012930-192.168.201.4-14af2bf0.log
>> the job links [2]
>>
>>
>>
>>
>>
>> [1]
>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/lastCompletedBuild/artifact/exported-artifacts/basic_suite_master.sh-el7/exported-artifacts/test_logs/basic-suite-master/post-002_bootstrap.py/lago-
>
>
> Now with the full link:
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/lastCompletedBuild/artifact/exported-artifacts/basic_suite_master.sh-el7/exported-artifacts/test_logs/basic-suite-master/post-002_bootstrap.py/lago-basic-suite-master-engine/_var_log_ovirt-engine/host-deploy/ovirt-host-deploy-20161227012930-192.168.201.4-14af2bf0.log
>
>>
>>
>>
>>
>> 016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
>> plugin.execute:921 execute-output: ('/bin/systemctl', 'start',
>> 'vdsmd.service') stdout:
>>
>> 2016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
>> plugin.execute:926 execute-output: ('/bin/systemctl', 'start',
>> 'vdsmd.service') stderr:
>> A dependency job for vdsmd.service failed. See 'journalctl -xe' for
>> details.
>>
>> 2016-12-27 01:29:29 DEBUG otopi.context context._executeMethod:142 method
>> exception
>> Traceback (most recent call last):
>>   File "/tmp/ovirt-QZ1ucxWFfm/pythonlib/otopi/context.py", line 132, in
>> _executeMethod
>> method['method']()
>>   File
>> "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/ovirt-host-deploy/vdsm/packages.py",
>> line 209, in _start
>> self.services.state('vdsmd', True)
>>   File "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/otopi/services/systemd.py",
>> line 141, in state
>> service=name,
>> RuntimeError: Failed to start service 'vdsmd'
>> 2016-12-27 01:29:29 ERROR otopi.context context._executeMethod:151 Failed
>> to execute stage 'Closing up': Failed to start service 'vdsmd'
>> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:760
>> ENVIRONMENT DUMP - BEGIN
>> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV
>> BASE/error=bool:'True'
>> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV
>> BASE/excep
>>
>> [2]
>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/lastCompletedBuild/testReport/


In the log I see

Processing package vdsm-4.20.0-7.gitf851d1b.el7.centos.x86_64

which is from Dec 22 (last Thursday). This is because of us missing a
master-branch tag. v4.20.0 wrongly tagged on the same commit as that
of v4.19.1, removed, and never placed properly.

I've re-pushed v4.20.0 properly, and now merged a patch to trigger
build-artifacts in master.
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/1544/

When this is done, could you use it to take the artifacts and try again?

Regards,
Dan.
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra

Re: ost host addition failure

2016-12-27 Thread Eyal Edri 
On Tue, Dec 27, 2016 at 9:56 AM, Eyal Edri  wrote:

> Any updates?
> The tests are still failing on vdsmd won't start from Sunday... master
> repos havn't been refreshed for a few days due to this.
>
> from host deploy log: [1] basic-suite-master-engine/_
> var_log_ovirt-engine/host-deploy/ovirt-host-deploy-
> 20161227012930-192.168.201.4-14af2bf0.log
> the job links [2]
>
>
>
>
>
> [1] http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
> master/lastCompletedBuild/artifact/exported-artifacts/
> basic_suite_master.sh-el7/exported-artifacts/test_logs/
> basic-suite-master/post-002_bootstrap.py/lago-
>

Now with the full link:
http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/lastCompletedBuild/artifact/exported-artifacts/basic_suite_master.sh-el7/exported-artifacts/test_logs/basic-suite-master/post-002_bootstrap.py/lago-basic-suite-master-engine/_var_log_ovirt-engine/host-deploy/ovirt-host-deploy-20161227012930-192.168.201.4-14af2bf0.log


>
>
>
> 016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
> plugin.execute:921 execute-output: ('/bin/systemctl', 'start',
> 'vdsmd.service') stdout:
>
> 2016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd 
> plugin.execute:926 execute-output: ('/bin/systemctl', 'start', 
> 'vdsmd.service') stderr:
> A dependency job for vdsmd.service failed. See 'journalctl -xe' for details.
>
> 2016-12-27 01:29:29 DEBUG otopi.context context._executeMethod:142 method 
> exception
> Traceback (most recent call last):
>   File "/tmp/ovirt-QZ1ucxWFfm/pythonlib/otopi/context.py", line 132, in 
> _executeMethod
> method['method']()
>   File 
> "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/ovirt-host-deploy/vdsm/packages.py", 
> line 209, in _start
> self.services.state('vdsmd', True)
>   File "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/otopi/services/systemd.py", line 
> 141, in state
> service=name,
> RuntimeError: Failed to start service 'vdsmd'
> 2016-12-27 01:29:29 ERROR otopi.context context._executeMethod:151 Failed to 
> execute stage 'Closing up': Failed to start service 'vdsmd'
> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:760 
> ENVIRONMENT DUMP - BEGIN
> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV 
> BASE/error=bool:'True'
> 2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770 ENV 
> BASE/excep
>
> [2] 
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/lastCompletedBuild/testReport/
>
>
>
>
> On Sun, Dec 25, 2016 at 11:31 AM, Eyal Edri  wrote:
>
>> We should see it fixed here hopefully [1]
>>
>>
>> [1] http://jenkins.ovirt.org/view/All%20Running%20jobs/job/t
>> est-repo_ovirt_experimental_master/4412/console
>>
>> On Sun, Dec 25, 2016 at 11:19 AM, Dan Kenigsberg 
>> wrote:
>>
>>> On Sun, Dec 25, 2016 at 10:28 AM, Yaniv Kaul  wrote:
>>> >
>>> >
>>> > On Sun, Dec 25, 2016 at 9:47 AM, Dan Kenigsberg 
>>> wrote:
>>> >>
>>> >> Correct. https://gerrit.ovirt.org/#/c/69052/
>>> >>
>>> >> Can you try adding
>>> >> lago shell "$vm_name" -c "mkdir -p /var/log/ovirt-imageio-daemon/ &&
>>> >> chown vdsm:kvm /var/log/ovirt-imageio-daemon/"
>>> >
>>> >
>>> > How will it know what is the vdsm user before installing vdsm?
>>>
>>> You're right. a hack would have to `chmod a+rwx
>>> /var/log/ovirt-imageio-daemon/` instead.
>>>
>>> > Why not either:
>>> > 1. Fix it
>>>
>>> yes, that's why we've opened
>>> https://bugzilla.redhat.com/show_bug.cgi?id=143 ; now a fix is
>>> getting merged. I don't know when it is going to be ready in lago's
>>> repos.
>>>
>>> > -or-
>>> > 2. Revert the offending patch?
>>>
>>> I'm not aware of such patch. It's a race that has been there since
>>> ever, and I don't know why it suddenly pops up so often.
>>>
>>
>>
>>
>> --
>> Eyal Edri
>> Associate Manager
>> RHV DevOps
>> EMEA ENG Virtualization R
>> Red Hat Israel
>>
>> phone: +972-9-7692018 <+972%209-769-2018>
>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>
>
>
>
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R
> Red Hat Israel
>
> phone: +972-9-7692018 <+972%209-769-2018>
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>



-- 
Eyal Edri
Associate Manager
RHV DevOps
EMEA ENG Virtualization R
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra

Re: ost host addition failure

2016-12-26 Thread Eyal Edri 
Any updates?
The tests are still failing on vdsmd won't start from Sunday... master
repos havn't been refreshed for a few days due to this.

from host deploy log: [1]
basic-suite-master-engine/_var_log_ovirt-engine/host-deploy/ovirt-host-deploy-20161227012930-192.168.201.4-14af2bf0.log
the job links [2]





[1]
http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/lastCompletedBuild/artifact/exported-artifacts/basic_suite_master.sh-el7/exported-artifacts/test_logs/basic-suite-master/post-002_bootstrap.py/lago-


016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:921 execute-output: ('/bin/systemctl', 'start',
'vdsmd.service') stdout:

2016-12-27 01:29:29 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:926 execute-output: ('/bin/systemctl', 'start',
'vdsmd.service') stderr:
A dependency job for vdsmd.service failed. See 'journalctl -xe' for details.

2016-12-27 01:29:29 DEBUG otopi.context context._executeMethod:142
method exception
Traceback (most recent call last):
  File "/tmp/ovirt-QZ1ucxWFfm/pythonlib/otopi/context.py", line 132,
in _executeMethod
method['method']()
  File "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/ovirt-host-deploy/vdsm/packages.py",
line 209, in _start
self.services.state('vdsmd', True)
  File "/tmp/ovirt-QZ1ucxWFfm/otopi-plugins/otopi/services/systemd.py",
line 141, in state
service=name,
RuntimeError: Failed to start service 'vdsmd'
2016-12-27 01:29:29 ERROR otopi.context context._executeMethod:151
Failed to execute stage 'Closing up': Failed to start service 'vdsmd'
2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:760
ENVIRONMENT DUMP - BEGIN
2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770
ENV BASE/error=bool:'True'
2016-12-27 01:29:29 DEBUG otopi.context context.dumpEnvironment:770
ENV BASE/excep

[2] 
http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/lastCompletedBuild/testReport/




On Sun, Dec 25, 2016 at 11:31 AM, Eyal Edri  wrote:

> We should see it fixed here hopefully [1]
>
>
> [1] http://jenkins.ovirt.org/view/All%20Running%20jobs/job/
> test-repo_ovirt_experimental_master/4412/console
>
> On Sun, Dec 25, 2016 at 11:19 AM, Dan Kenigsberg 
> wrote:
>
>> On Sun, Dec 25, 2016 at 10:28 AM, Yaniv Kaul  wrote:
>> >
>> >
>> > On Sun, Dec 25, 2016 at 9:47 AM, Dan Kenigsberg 
>> wrote:
>> >>
>> >> Correct. https://gerrit.ovirt.org/#/c/69052/
>> >>
>> >> Can you try adding
>> >> lago shell "$vm_name" -c "mkdir -p /var/log/ovirt-imageio-daemon/ &&
>> >> chown vdsm:kvm /var/log/ovirt-imageio-daemon/"
>> >
>> >
>> > How will it know what is the vdsm user before installing vdsm?
>>
>> You're right. a hack would have to `chmod a+rwx
>> /var/log/ovirt-imageio-daemon/` instead.
>>
>> > Why not either:
>> > 1. Fix it
>>
>> yes, that's why we've opened
>> https://bugzilla.redhat.com/show_bug.cgi?id=143 ; now a fix is
>> getting merged. I don't know when it is going to be ready in lago's
>> repos.
>>
>> > -or-
>> > 2. Revert the offending patch?
>>
>> I'm not aware of such patch. It's a race that has been there since
>> ever, and I don't know why it suddenly pops up so often.
>>
>
>
>
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R
> Red Hat Israel
>
> phone: +972-9-7692018 <+972%209-769-2018>
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>



-- 
Eyal Edri
Associate Manager
RHV DevOps
EMEA ENG Virtualization R
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra