Adding lago-devel. Anyone from Lago project can help debug why it is taking so long to setup Lago (1:17 hours?)
Yaniv, was this run done only on CI, or someone tried to run it locally on a baremetal server / laptop? e. On Tue, Dec 22, 2015 at 2:50 PM, Yaniv Bronheim <ybron...@redhat.com> wrote: > OK. so for now the patch [1] skips all storage and OVSNetwork tests > Which still doesn't help much, single run takes more than 3hrs :/ (1:17hr, > for the setup :\ the rest for the tests) > > The output looks pretty good though :) > > [1] https://gerrit.ovirt.org/#/c/48268/ > > On Thu, Dec 17, 2015 at 12:13 PM, Petr Horacek <phora...@redhat.com> > wrote: > >> Hi, >> >> OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped >> if the package is not installed) and openvswitch service running (not >> started automatically). NetworkTest are not able to run together with >> OVSNetworkTest since we are doing ugly inheritance hacking there. My >> apologize for network tests difficulties, it will be resolved soon. If >> you want your life easier, temporary skip OVSNetworkTest suite and it >> should be OK. >> >> Best regards, >> Petr >> >> 2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <ybron...@redhat.com>: >> > exactly - just posted a patch again that sings all fails as broken . >> > we'll get the report soon and I'll publish it as well. hope the run will >> > take less time now >> > >> > On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsof...@redhat.com> wrote: >> >> >> >> Nice, but we cannot enable this until all the tests pass or disabled. >> >> >> >> There is no point in broken or flaky functional tests. >> >> >> >> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybron...@redhat.com> >> >> wrote: >> >> > So its not stable. It won't block merges and at least give us report >> >> > after >> >> > each merge. It takes really long time to run it (because of the tests >> >> > themselves. Lago things takes maximum 15minutes, but the run last for >> >> > more >> >> > than 2hrs right now and I suspect functional/storageTests.py gets >> stuck) >> >> > >> >> > Bellow you can see where we stand (before I added python-rtslib >> >> > package). >> >> > >> >> > Now, I still want to merge the patch >> https://gerrit.ovirt.org/#/c/48268/ >> >> > - >> >> > which enables this run after merges, and I still want you to consider >> >> > the >> >> > addition of Automation CI flag to our gerrit so that developer will >> be >> >> > able >> >> > to use it as a trigger for the check-merged.sh script run, just to >> see >> >> > if >> >> > their patch fixes\brakes something realted to the functional tests >> >> > >> >> > >> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ - >> >> > is >> >> > an example of how the run looks like. I still work to improve the >> output >> >> > >> >> > >> >> > Please reply and let me know if the idea around the automation flag >> is >> >> > acceptable by you.. and please review the patch for comments and >> acks. >> >> > We can ask dcaro to add the flag until Friday, otherwise we'll need >> to >> >> > delay >> >> > this effort after the holiday.. >> >> > >> >> > >> >> > functional.sosPluginTests.SosPluginTest >> >> > testSosPlugin OK >> >> > functional.vmRecoveryTests.RecoveryTests >> >> > test_vm_recovery FAIL >> >> > functional.vmQoSTests.VMQosTests >> >> > testSmallVMBallooning FAIL >> >> > functional.virtTests.VirtTest >> >> > testComplexVm FAIL >> >> > testHeadlessVm OK >> >> > testSimpleVm FAIL >> >> > testVmDefinitionGraphics('spice') FAIL >> >> > testVmDefinitionGraphics('vnc') OK >> >> > testVmDefinitionLegacyGraphics('qxl') FAIL >> >> > testVmDefinitionLegacyGraphics('vnc') OK >> >> > testVmDefinitionMultipleGraphics('spice', 'vnc') FAIL >> >> > testVmDefinitionMultipleGraphics('vnc', 'spice') FAIL >> >> > testVmWithCdrom('self') FAIL >> >> > testVmWithCdrom('specParams') FAIL >> >> > testVmWithCdrom('vmPayload') FAIL >> >> > testVmWithDevice('hotplugDisk') FAIL >> >> > testVmWithDevice('hotplugNic') FAIL >> >> > testVmWithDevice('smartcard') FAIL >> >> > testVmWithDevice('virtioNic') FAIL >> >> > testVmWithDevice('virtioRng') FAIL >> >> > testVmWithSla FAIL >> >> > testVmWithStorage('iscsi') SKIP: >> >> > python-rtslib is not installed. >> >> > testVmWithStorage('localfs') FAIL >> >> > testVmWithStorage('nfs') FAIL >> >> > functional.storageTests.StorageTest >> >> > testCreatePoolErrors OK >> >> > testStorage('glusterfs', 0) ERROR >> >> > testStorage('glusterfs', 3) ERROR >> >> > testStorage('iscsi', 0) SKIP: >> >> > python-rtslib is not installed. >> >> > testStorage('iscsi', 3) SKIP: >> >> > python-rtslib is not installed. >> >> > testStorage('localfs', 0) FAIL >> >> > testStorage('localfs', 3) FAIL >> >> > testStorage('nfs', 0) FAIL >> >> > testStorage('nfs', 3) FAIL >> >> > functional.networkTests.NetworkTest >> >> > testAddVlanedBridgeless ERROR >> >> > testAddVlanedBridgeless_oneCommand ERROR >> >> > testAfterNetworkSetupHook ERROR >> >> > testBeforeNetworkSetupHook ERROR >> >> > testBondHwAddress(False) ERROR >> >> > testBondHwAddress(True) ERROR >> >> > testBrokenNetworkReplacement(False) ERROR >> >> > testBrokenNetworkReplacement(True) ERROR >> >> > testDelNetworkBondAccumulation ERROR >> >> > testDelNetworkWithMTU(False) ERROR >> >> > testDelNetworkWithMTU(True) ERROR >> >> > testDelWithoutAdd ERROR >> >> > testDhclientLeases(4, 'default') ERROR >> >> > testDhclientLeases(4, 'local') ERROR >> >> > testDhclientLeases(6, None) ERROR >> >> > testDhcpReplaceNicWithBridge ERROR >> >> > testFailWithInvalidBondingName(False) ERROR >> >> > testFailWithInvalidBondingName(True) ERROR >> >> > testFailWithInvalidBridgeName ERROR >> >> > testFailWithInvalidIpConfig ERROR >> >> > testFailWithInvalidNic(False) ERROR >> >> > testFailWithInvalidNic(True) ERROR >> >> > testFailWithInvalidParams(False) ERROR >> >> > testFailWithInvalidParams(True) ERROR >> >> > testGetRouteDeviceTo ERROR >> >> > testHonorBlockingDhcp SKIP: >> >> > Slow >> >> > tests are disabled >> >> > testIpLinkWrapper ERROR >> >> > testLowerMtuDoesNotOverride FAIL >> >> > testNoBridgeLeftovers FAIL >> >> > testReconfigureBrNetWithVanishedPort ERROR >> >> > testRedefineBondedNetworkIPs ERROR >> >> > testRemovingBridgeDoesNotLeaveBridge ERROR >> >> > testReorderBondingOptions(False) ERROR >> >> > testReorderBondingOptions(True) ERROR >> >> > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR >> >> > testRestoreToBlockingDHCP ERROR >> >> > testSafeNetworkConfig(False) ERROR >> >> > testSafeNetworkConfig(True) ERROR >> >> > testSelectiveRestoreDuringUpgrade ERROR >> >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL >> >> > testSetupNetworkOutboundQos(False) ERROR >> >> > testSetupNetworkOutboundQos(True) FAIL >> >> > testSetupNetworksActiveSlave ERROR >> >> > testSetupNetworksAddBadParams(False) ERROR >> >> > testSetupNetworksAddBadParams(True) FAIL >> >> > testSetupNetworksAddBondWithManyVlans(False) ERROR >> >> > testSetupNetworksAddBondWithManyVlans(True) FAIL >> >> > testSetupNetworksAddDelBondedNetwork(False) ERROR >> >> > testSetupNetworksAddDelBondedNetwork(True) FAIL >> >> > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR >> >> > testSetupNetworksAddDelDhcp(False, (4,)) ERROR >> >> > testSetupNetworksAddDelDhcp(False, (6,)) ERROR >> >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL >> >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL >> >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL >> >> > testSetupNetworksAddManyVlans(False) ERROR >> >> > testSetupNetworksAddManyVlans(True) FAIL >> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR >> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL >> >> > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR >> >> > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR >> >> > testSetupNetworksAddOverExistingBond(False) ERROR >> >> > testSetupNetworksAddOverExistingBond(True) ERROR >> >> > testSetupNetworksAddVlan(False) ERROR >> >> > testSetupNetworksAddVlan(True) FAIL >> >> > testSetupNetworksConnectivityCheck FAIL >> >> > testSetupNetworksConvertVlanNetBridgeness ERROR >> >> > testSetupNetworksDelOneOfBondNets ERROR >> >> > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR >> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR >> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: >> >> > Slow >> >> > tests are disabled >> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR >> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: >> >> > Slow >> >> > tests are disabled >> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR >> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL >> >> > testSetupNetworksMtus(False) ERROR >> >> > testSetupNetworksMtus(True) SKIP: >> >> > This >> >> > test fails because the 2 different networks are getting configured >> with >> >> > the >> >> > same MTU. The test should assert that the reported MTUs are equal to >> the >> >> > requested ones. >> >> > testSetupNetworksMultiMTUsOverBond(False) ERROR >> >> > testSetupNetworksMultiMTUsOverBond(True) FAIL >> >> > testSetupNetworksMultiMTUsOverNic(False) ERROR >> >> > testSetupNetworksMultiMTUsOverNic(True) FAIL >> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR >> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL >> >> > testSetupNetworksNicless FAIL >> >> > testSetupNetworksNiclessBridgeless ERROR >> >> > testSetupNetworksOverDhcpIface ERROR >> >> > testSetupNetworksOverExistingBridge FAIL >> >> > testSetupNetworksReconfigureBridge FAIL >> >> > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR >> >> > testSetupNetworksRemoveSlavelessBond ERROR >> >> > testSetupNetworksResizeBond(False) ERROR >> >> > testSetupNetworksResizeBond(True) ERROR >> >> > testSetupNetworksStableBond(False) ERROR >> >> > testSetupNetworksStableBond(True) ERROR >> >> > testStaticNetworkConfig((4, 6)) FAIL >> >> > testStaticNetworkConfig((4,)) FAIL >> >> > testStaticNetworkConfig((6,)) FAIL >> >> > testStaticSourceRouting(False) ERROR >> >> > testStaticSourceRouting(True) FAIL >> >> > testTwiceAdd(False) ERROR >> >> > testTwiceAdd(True) ERROR >> >> > testVolatileConfig(False) ERROR >> >> > testVolatileConfig(True) ERROR >> >> > test_drop_initial_bond_slaves_ip_config FAIL >> >> > test_getVdsStats ERROR >> >> > test_remove_bond_under_network FAIL >> >> > test_remove_initial_network_nic_ip_config FAIL >> >> > test_rollback FAIL >> >> > test_setupNetworks_bond_with_custom_option ERROR >> >> > test_setupNetworks_on_external_bond ERROR >> >> > test_setupNetworks_on_external_vlaned_bond ERROR >> >> > functional.networkTestsOVS.OVSNetworkTest >> >> > testHonorBlockingDhcp SKIP: >> >> > Slow >> >> > tests are disabled >> >> > testLowerMtuDoesNotOverride OK >> >> > testNoBridgeLeftovers FAIL >> >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL >> >> > testSetupNetworkOutboundQos(True) OK >> >> > testSetupNetworksAddBadParams(True) OK >> >> > testSetupNetworksAddBondWithManyVlans(True) FAIL >> >> > testSetupNetworksAddDelBondedNetwork(True) FAIL >> >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL >> >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL >> >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL >> >> > testSetupNetworksAddManyVlans(True) OK >> >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL >> >> > testSetupNetworksAddVlan(True) FAIL >> >> > testSetupNetworksConnectivityCheck OK >> >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: >> >> > Slow >> >> > tests are disabled >> >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: >> >> > Slow >> >> > tests are disabled >> >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL >> >> > testSetupNetworksMtus(True) SKIP: >> >> > This >> >> > test fails because the 2 different networks are getting configured >> with >> >> > the >> >> > same MTU. The test should assert that the reported MTUs are equal to >> the >> >> > requested ones. >> >> > testSetupNetworksMultiMTUsOverBond(True) FAIL >> >> > testSetupNetworksMultiMTUsOverNic(True) FAIL >> >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL >> >> > testSetupNetworksNicless OK >> >> > testSetupNetworksOverExistingBridge OK >> >> > testSetupNetworksReconfigureBridge FAIL >> >> > testStaticNetworkConfig((4, 6)) FAIL >> >> > testStaticNetworkConfig((4,)) FAIL >> >> > testStaticNetworkConfig((6,)) FAIL >> >> > testStaticSourceRouting(True) FAIL >> >> > test_drop_initial_bond_slaves_ip_config FAIL >> >> > test_ovirtmgmtm_to_ovs FAIL >> >> > test_remove_bond_under_network SKIP: >> >> > OVS >> >> > does not support bridgeless networks >> >> > test_remove_initial_network_nic_ip_config OK >> >> > test_rollback FAIL >> >> > >> >> > >> >> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski < >> pklic...@redhat.com> >> >> > wrote: >> >> >> >> >> >> I like the idea but I have the same feelings as Francesco. I think >> that >> >> >> we >> >> >> need to make sure that functional tests for each vertical are stable >> >> >> before >> >> >> enabling this process. >> >> >> >> >> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <ee...@redhat.com> >> wrote: >> >> >>> >> >> >>> adding also infra team for visibility on the change in CI. >> >> >>> also inline. >> >> >>> >> >> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani < >> from...@redhat.com> >> >> >>> wrote: >> >> >>>> >> >> >>>> ----- Original Message ----- >> >> >>>> > From: "Yaniv Bronheim" <ybron...@redhat.com> >> >> >>>> > To: devel@ovirt.org, "Francesco Romani" <from...@redhat.com>, >> "Nir >> >> >>>> > Soffer" <nsof...@redhat.com>, "Piotr Kliczewski" >> >> >>>> > <pklic...@redhat.com> >> >> >>>> > Cc: "danken" <dan...@redhat.com>, "David Caro" >> >> >>>> > <dcaro...@redhat.com>, >> >> >>>> > "Eyal Edri" <ee...@redhat.com> >> >> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM >> >> >>>> > Subject: Automation CI for vdsm >> >> >>>> >> >> >>>> [...] >> >> >>>> > We want to allow developers to trigger the script once reviews >> and >> >> >>>> > verification are ready (last step before merge). To do so we >> agreed >> >> >>>> > to >> >> >>>> > add >> >> >>>> > Continues Integration flag for each vdsm patch. >> >> >>> >> >> >>> >> >> >>> This flag will be called 'Workflow' or we can name it otherwise, we >> >> >>> just >> >> >>> need to choose what makes sense. >> >> >>> David/Yaniv - Please correct me if I'm wrong. >> >> >>> >> >> >>>> >> >> >>>> Once this flag will be >> >> >>>> > signed with +1 it will trigger Jenkins CI to run the >> check-merged >> >> >>>> > script >> >> >>>> > (adding new button to gerrit is not an option - you can image >> that >> >> >>>> > flag as >> >> >>>> > a trigger button), on success Jenkins CI flag will turn to +2. >> on >> >> >>>> > fail >> >> >>>> > we'll get -1 and once new patchset is ready the developer will >> >> >>>> > remove >> >> >>>> > the >> >> >>>> > +1 and add it back to the Continues Integration flag to >> re-trigger >> >> >>>> > the >> >> >>>> > job. >> >> >>>> > >> >> >>>> > Please ack the process before we move on with that >> >> >>>> >> >> >>>> Sounds good, even though I'm a little scared (just gut feeling, no >> >> >>>> evidence >> >> >>>> whatsoever) that this could add even more complexity and >> fragility to >> >> >>>> the jenkins >> >> >>>> fleet. >> >> >>>> >> >> >>>> In the long run, when this is reliable, it will help greatly. >> >> >>>> In the short term, I'm scared because this can lead to false >> >> >>>> positives >> >> >>>> and bogus >> >> >>>> failures. >> >> >>>> >> >> >>>> Let me stress I don't have concrete item to share or specific >> flaws. >> >> >>>> >> >> >>>> As action item on me, I will find some time next week to check >> virt >> >> >>>> functional tests, >> >> >>>> to see if they need some fixes, work reliably and so forth >> >> >>>> >> >> >>>> > The patch for those scripts still under review and testing - >> >> >>>> > https://gerrit.ovirt.org/#/c/48268 >> >> >>>> >> >> >>>> Will review asap. >> >> >>>> >> >> >>>> -- >> >> >>>> Francesco Romani >> >> >>>> RedHat Engineering Virtualization R & D >> >> >>>> Phone: 8261328 >> >> >>>> IRC: fromani >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Eyal Edri >> >> >>> Supervisor, RHEV CI >> >> >>> EMEA ENG Virtualization R&D >> >> >>> Red Hat Israel >> >> >>> >> >> >>> phone: +972-9-7692018 >> >> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> >> >> >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Yaniv Bronhaim. >> > >> > >> > >> > >> > -- >> > Yaniv Bronhaim. >> > >> > _______________________________________________ >> > Devel mailing list >> > Devel@ovirt.org >> > http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > *Yaniv Bronhaim.* > > _______________________________________________ > Infra mailing list > in...@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -- Eyal Edri Associate Manager EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel