[JIRA] (OVIRT-780) engine upgrade job is disabling puppet on slaves
[ https://ovirt-jira.atlassian.net/browse/OVIRT-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evgheni Dereveanchin reassigned OVIRT-780: -- Assignee: Evgheni Dereveanchin (was: infra) > engine upgrade job is disabling puppet on slaves > > > Key: OVIRT-780 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-780 > Project: oVirt - virtualization made easy > Issue Type: Improvement >Reporter: Evgheni Dereveanchin >Assignee: Evgheni Dereveanchin >Priority: Low > Fix For: OVIRT-INFRA-NOV-2016 > > > I found a lot of out-of-sync slaves in Foreman and further investigation > revealed that puppet agent is disabled on them. > The journal indicates that it's the upgrade job doing this: > Oct 19 14:24:51 vm0079.workers-phx.ovirt.org sudo[18213]: jenkins : > TTY=unknown ; > PWD=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-4.0_el7_created > ; USER=root ; COMMAND=/bin/puppet agent --disable > Oct 19 14:24:54 vm0079.workers-phx.ovirt.org puppet-agent[18215]: Disabling > Puppet. > The job seems to re-enable puppet after it runs, but once there's several > consecutive jobs in the queue this causes puppet to be effectively turned off > for hours. > Do we really need this? Can we run engine upgrade jobs inside mock to not > affect the node itself? -- This message was sent by Atlassian JIRA (v1000.695.2#100025) ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
[JIRA] (OVIRT-780) engine upgrade job is disabling puppet on slaves
[ https://ovirt-jira.atlassian.net/browse/OVIRT-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eyal edri [Administrator] updated OVIRT-780: Resolution: Fixed Status: Done (was: To Do) > engine upgrade job is disabling puppet on slaves > > > Key: OVIRT-780 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-780 > Project: oVirt - virtualization made easy > Issue Type: Improvement >Reporter: Evgheni Dereveanchin >Assignee: infra >Priority: Low > > I found a lot of out-of-sync slaves in Foreman and further investigation > revealed that puppet agent is disabled on them. > The journal indicates that it's the upgrade job doing this: > Oct 19 14:24:51 vm0079.workers-phx.ovirt.org sudo[18213]: jenkins : > TTY=unknown ; > PWD=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-4.0_el7_created > ; USER=root ; COMMAND=/bin/puppet agent --disable > Oct 19 14:24:54 vm0079.workers-phx.ovirt.org puppet-agent[18215]: Disabling > Puppet. > The job seems to re-enable puppet after it runs, but once there's several > consecutive jobs in the queue this causes puppet to be effectively turned off > for hours. > Do we really need this? Can we run engine upgrade jobs inside mock to not > affect the node itself? -- This message was sent by Atlassian JIRA (v1000.482.6#100017) ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
[JIRA] (OVIRT-780) engine upgrade job is disabling puppet on slaves
[ https://ovirt-jira.atlassian.net/browse/OVIRT-780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=22212#comment-22212 ] Evgheni Dereveanchin commented on OVIRT-780: Logged ovirt-808 to track moving upgrade jobs to lago - that should fix the issue. In general, disabling puppet during a job run is not critical and is only visible when there's a lot of builds so dozens of nodes get stale on Foreman. I think we can close this case. > engine upgrade job is disabling puppet on slaves > > > Key: OVIRT-780 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-780 > Project: oVirt - virtualization made easy > Issue Type: Improvement >Reporter: Evgheni Dereveanchin >Assignee: infra >Priority: Low > > I found a lot of out-of-sync slaves in Foreman and further investigation > revealed that puppet agent is disabled on them. > The journal indicates that it's the upgrade job doing this: > Oct 19 14:24:51 vm0079.workers-phx.ovirt.org sudo[18213]: jenkins : > TTY=unknown ; > PWD=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-4.0_el7_created > ; USER=root ; COMMAND=/bin/puppet agent --disable > Oct 19 14:24:54 vm0079.workers-phx.ovirt.org puppet-agent[18215]: Disabling > Puppet. > The job seems to re-enable puppet after it runs, but once there's several > consecutive jobs in the queue this causes puppet to be effectively turned off > for hours. > Do we really need this? Can we run engine upgrade jobs inside mock to not > affect the node itself? -- This message was sent by Atlassian JIRA (v1000.482.6#100017) ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
[JIRA] (OVIRT-780) engine upgrade job is disabling puppet on slaves
[ https://ovirt-jira.atlassian.net/browse/OVIRT-780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=22204#comment-22204 ] eyal edri [Administrator] commented on OVIRT-780: - any action item here? > engine upgrade job is disabling puppet on slaves > > > Key: OVIRT-780 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-780 > Project: oVirt - virtualization made easy > Issue Type: Improvement >Reporter: Evgheni Dereveanchin >Assignee: infra >Priority: Low > > I found a lot of out-of-sync slaves in Foreman and further investigation > revealed that puppet agent is disabled on them. > The journal indicates that it's the upgrade job doing this: > Oct 19 14:24:51 vm0079.workers-phx.ovirt.org sudo[18213]: jenkins : > TTY=unknown ; > PWD=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-4.0_el7_created > ; USER=root ; COMMAND=/bin/puppet agent --disable > Oct 19 14:24:54 vm0079.workers-phx.ovirt.org puppet-agent[18215]: Disabling > Puppet. > The job seems to re-enable puppet after it runs, but once there's several > consecutive jobs in the queue this causes puppet to be effectively turned off > for hours. > Do we really need this? Can we run engine upgrade jobs inside mock to not > affect the node itself? -- This message was sent by Atlassian JIRA (v1000.482.6#100017) ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
[JIRA] (OVIRT-780) engine upgrade job is disabling puppet on slaves
[ https://ovirt-jira.atlassian.net/browse/OVIRT-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eyal edri [Administrator] updated OVIRT-780: Epic Link: OVIRT-400 > engine upgrade job is disabling puppet on slaves > > > Key: OVIRT-780 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-780 > Project: oVirt - virtualization made easy > Issue Type: Improvement >Reporter: Evgheni Dereveanchin >Assignee: infra >Priority: Low > > I found a lot of out-of-sync slaves in Foreman and further investigation > revealed that puppet agent is disabled on them. > The journal indicates that it's the upgrade job doing this: > Oct 19 14:24:51 vm0079.workers-phx.ovirt.org sudo[18213]: jenkins : > TTY=unknown ; > PWD=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-4.0_el7_created > ; USER=root ; COMMAND=/bin/puppet agent --disable > Oct 19 14:24:54 vm0079.workers-phx.ovirt.org puppet-agent[18215]: Disabling > Puppet. > The job seems to re-enable puppet after it runs, but once there's several > consecutive jobs in the queue this causes puppet to be effectively turned off > for hours. > Do we really need this? Can we run engine upgrade jobs inside mock to not > affect the node itself? -- This message was sent by Atlassian JIRA (v1000.482.6#100017) ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
[JIRA] (OVIRT-780) engine upgrade job is disabling puppet on slaves
[ https://ovirt-jira.atlassian.net/browse/OVIRT-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eyal edri [Administrator] updated OVIRT-780: Epic Link: OVIRT-400 > engine upgrade job is disabling puppet on slaves > > > Key: OVIRT-780 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-780 > Project: oVirt - virtualization made easy > Issue Type: Improvement >Reporter: Evgheni Dereveanchin >Assignee: infra >Priority: Low > > I found a lot of out-of-sync slaves in Foreman and further investigation > revealed that puppet agent is disabled on them. > The journal indicates that it's the upgrade job doing this: > Oct 19 14:24:51 vm0079.workers-phx.ovirt.org sudo[18213]: jenkins : > TTY=unknown ; > PWD=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-4.0_el7_created > ; USER=root ; COMMAND=/bin/puppet agent --disable > Oct 19 14:24:54 vm0079.workers-phx.ovirt.org puppet-agent[18215]: Disabling > Puppet. > The job seems to re-enable puppet after it runs, but once there's several > consecutive jobs in the queue this causes puppet to be effectively turned off > for hours. > Do we really need this? Can we run engine upgrade jobs inside mock to not > affect the node itself? -- This message was sent by Atlassian JIRA (v1000.482.6#100017) ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
[JIRA] (OVIRT-780) engine upgrade job is disabling puppet on slaves
[ https://ovirt-jira.atlassian.net/browse/OVIRT-780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=22000#comment-22000 ] Barak Korren commented on OVIRT-780: We need this because engine-setup does not know how to recover from a locked yum. Effectively this means that if Puppet runs while engine-setup is running, the engine-setup will fail. Engine upgrade cannot run inside mock because it needs access to things like systemd which are not typically found inside mock. We've discussed this in the past, we should have a ticket somewhere to migrate the upgrade jobs to Lago. > engine upgrade job is disabling puppet on slaves > > > Key: OVIRT-780 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-780 > Project: oVirt - virtualization made easy > Issue Type: Improvement >Reporter: Evgheni Dereveanchin >Assignee: infra >Priority: Low > > I found a lot of out-of-sync slaves in Foreman and further investigation > revealed that puppet agent is disabled on them. > The journal indicates that it's the upgrade job doing this: > Oct 19 14:24:51 vm0079.workers-phx.ovirt.org sudo[18213]: jenkins : > TTY=unknown ; > PWD=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-4.0_el7_created > ; USER=root ; COMMAND=/bin/puppet agent --disable > Oct 19 14:24:54 vm0079.workers-phx.ovirt.org puppet-agent[18215]: Disabling > Puppet. > The job seems to re-enable puppet after it runs, but once there's several > consecutive jobs in the queue this causes puppet to be effectively turned off > for hours. > Do we really need this? Can we run engine upgrade jobs inside mock to not > affect the node itself? -- This message was sent by Atlassian JIRA (v1000.456.2#100016) ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
