OK. so for now the patch [1] skips all storage and OVSNetwork tests Which still doesn't help much, single run takes more than 3hrs :/ (1:17hr, for the setup :\ the rest for the tests)
The output looks pretty good though :) [1] https://gerrit.ovirt.org/#/c/48268/ On Thu, Dec 17, 2015 at 12:13 PM, Petr Horacek <phora...@redhat.com> wrote: > Hi, > > OVSNetworkTest requires vdsm-hook-ovs installed (tests are not skipped > if the package is not installed) and openvswitch service running (not > started automatically). NetworkTest are not able to run together with > OVSNetworkTest since we are doing ugly inheritance hacking there. My > apologize for network tests difficulties, it will be resolved soon. If > you want your life easier, temporary skip OVSNetworkTest suite and it > should be OK. > > Best regards, > Petr > > 2015-12-16 17:25 GMT+01:00 Yaniv Bronheim <ybron...@redhat.com>: > > exactly - just posted a patch again that sings all fails as broken . > > we'll get the report soon and I'll publish it as well. hope the run will > > take less time now > > > > On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsof...@redhat.com> wrote: > >> > >> Nice, but we cannot enable this until all the tests pass or disabled. > >> > >> There is no point in broken or flaky functional tests. > >> > >> On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybron...@redhat.com> > >> wrote: > >> > So its not stable. It won't block merges and at least give us report > >> > after > >> > each merge. It takes really long time to run it (because of the tests > >> > themselves. Lago things takes maximum 15minutes, but the run last for > >> > more > >> > than 2hrs right now and I suspect functional/storageTests.py gets > stuck) > >> > > >> > Bellow you can see where we stand (before I added python-rtslib > >> > package). > >> > > >> > Now, I still want to merge the patch > https://gerrit.ovirt.org/#/c/48268/ > >> > - > >> > which enables this run after merges, and I still want you to consider > >> > the > >> > addition of Automation CI flag to our gerrit so that developer will be > >> > able > >> > to use it as a trigger for the check-merged.sh script run, just to see > >> > if > >> > their patch fixes\brakes something realted to the functional tests > >> > > >> > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ - > >> > is > >> > an example of how the run looks like. I still work to improve the > output > >> > > >> > > >> > Please reply and let me know if the idea around the automation flag is > >> > acceptable by you.. and please review the patch for comments and acks. > >> > We can ask dcaro to add the flag until Friday, otherwise we'll need to > >> > delay > >> > this effort after the holiday.. > >> > > >> > > >> > functional.sosPluginTests.SosPluginTest > >> > testSosPlugin OK > >> > functional.vmRecoveryTests.RecoveryTests > >> > test_vm_recovery FAIL > >> > functional.vmQoSTests.VMQosTests > >> > testSmallVMBallooning FAIL > >> > functional.virtTests.VirtTest > >> > testComplexVm FAIL > >> > testHeadlessVm OK > >> > testSimpleVm FAIL > >> > testVmDefinitionGraphics('spice') FAIL > >> > testVmDefinitionGraphics('vnc') OK > >> > testVmDefinitionLegacyGraphics('qxl') FAIL > >> > testVmDefinitionLegacyGraphics('vnc') OK > >> > testVmDefinitionMultipleGraphics('spice', 'vnc') FAIL > >> > testVmDefinitionMultipleGraphics('vnc', 'spice') FAIL > >> > testVmWithCdrom('self') FAIL > >> > testVmWithCdrom('specParams') FAIL > >> > testVmWithCdrom('vmPayload') FAIL > >> > testVmWithDevice('hotplugDisk') FAIL > >> > testVmWithDevice('hotplugNic') FAIL > >> > testVmWithDevice('smartcard') FAIL > >> > testVmWithDevice('virtioNic') FAIL > >> > testVmWithDevice('virtioRng') FAIL > >> > testVmWithSla FAIL > >> > testVmWithStorage('iscsi') SKIP: > >> > python-rtslib is not installed. > >> > testVmWithStorage('localfs') FAIL > >> > testVmWithStorage('nfs') FAIL > >> > functional.storageTests.StorageTest > >> > testCreatePoolErrors OK > >> > testStorage('glusterfs', 0) ERROR > >> > testStorage('glusterfs', 3) ERROR > >> > testStorage('iscsi', 0) SKIP: > >> > python-rtslib is not installed. > >> > testStorage('iscsi', 3) SKIP: > >> > python-rtslib is not installed. > >> > testStorage('localfs', 0) FAIL > >> > testStorage('localfs', 3) FAIL > >> > testStorage('nfs', 0) FAIL > >> > testStorage('nfs', 3) FAIL > >> > functional.networkTests.NetworkTest > >> > testAddVlanedBridgeless ERROR > >> > testAddVlanedBridgeless_oneCommand ERROR > >> > testAfterNetworkSetupHook ERROR > >> > testBeforeNetworkSetupHook ERROR > >> > testBondHwAddress(False) ERROR > >> > testBondHwAddress(True) ERROR > >> > testBrokenNetworkReplacement(False) ERROR > >> > testBrokenNetworkReplacement(True) ERROR > >> > testDelNetworkBondAccumulation ERROR > >> > testDelNetworkWithMTU(False) ERROR > >> > testDelNetworkWithMTU(True) ERROR > >> > testDelWithoutAdd ERROR > >> > testDhclientLeases(4, 'default') ERROR > >> > testDhclientLeases(4, 'local') ERROR > >> > testDhclientLeases(6, None) ERROR > >> > testDhcpReplaceNicWithBridge ERROR > >> > testFailWithInvalidBondingName(False) ERROR > >> > testFailWithInvalidBondingName(True) ERROR > >> > testFailWithInvalidBridgeName ERROR > >> > testFailWithInvalidIpConfig ERROR > >> > testFailWithInvalidNic(False) ERROR > >> > testFailWithInvalidNic(True) ERROR > >> > testFailWithInvalidParams(False) ERROR > >> > testFailWithInvalidParams(True) ERROR > >> > testGetRouteDeviceTo ERROR > >> > testHonorBlockingDhcp SKIP: > >> > Slow > >> > tests are disabled > >> > testIpLinkWrapper ERROR > >> > testLowerMtuDoesNotOverride FAIL > >> > testNoBridgeLeftovers FAIL > >> > testReconfigureBrNetWithVanishedPort ERROR > >> > testRedefineBondedNetworkIPs ERROR > >> > testRemovingBridgeDoesNotLeaveBridge ERROR > >> > testReorderBondingOptions(False) ERROR > >> > testReorderBondingOptions(True) ERROR > >> > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR > >> > testRestoreToBlockingDHCP ERROR > >> > testSafeNetworkConfig(False) ERROR > >> > testSafeNetworkConfig(True) ERROR > >> > testSelectiveRestoreDuringUpgrade ERROR > >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL > >> > testSetupNetworkOutboundQos(False) ERROR > >> > testSetupNetworkOutboundQos(True) FAIL > >> > testSetupNetworksActiveSlave ERROR > >> > testSetupNetworksAddBadParams(False) ERROR > >> > testSetupNetworksAddBadParams(True) FAIL > >> > testSetupNetworksAddBondWithManyVlans(False) ERROR > >> > testSetupNetworksAddBondWithManyVlans(True) FAIL > >> > testSetupNetworksAddDelBondedNetwork(False) ERROR > >> > testSetupNetworksAddDelBondedNetwork(True) FAIL > >> > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR > >> > testSetupNetworksAddDelDhcp(False, (4,)) ERROR > >> > testSetupNetworksAddDelDhcp(False, (6,)) ERROR > >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL > >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL > >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL > >> > testSetupNetworksAddManyVlans(False) ERROR > >> > testSetupNetworksAddManyVlans(True) FAIL > >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR > >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL > >> > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR > >> > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR > >> > testSetupNetworksAddOverExistingBond(False) ERROR > >> > testSetupNetworksAddOverExistingBond(True) ERROR > >> > testSetupNetworksAddVlan(False) ERROR > >> > testSetupNetworksAddVlan(True) FAIL > >> > testSetupNetworksConnectivityCheck FAIL > >> > testSetupNetworksConvertVlanNetBridgeness ERROR > >> > testSetupNetworksDelOneOfBondNets ERROR > >> > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR > >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR > >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: > >> > Slow > >> > tests are disabled > >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR > >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: > >> > Slow > >> > tests are disabled > >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR > >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL > >> > testSetupNetworksMtus(False) ERROR > >> > testSetupNetworksMtus(True) SKIP: > >> > This > >> > test fails because the 2 different networks are getting configured > with > >> > the > >> > same MTU. The test should assert that the reported MTUs are equal to > the > >> > requested ones. > >> > testSetupNetworksMultiMTUsOverBond(False) ERROR > >> > testSetupNetworksMultiMTUsOverBond(True) FAIL > >> > testSetupNetworksMultiMTUsOverNic(False) ERROR > >> > testSetupNetworksMultiMTUsOverNic(True) FAIL > >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR > >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL > >> > testSetupNetworksNicless FAIL > >> > testSetupNetworksNiclessBridgeless ERROR > >> > testSetupNetworksOverDhcpIface ERROR > >> > testSetupNetworksOverExistingBridge FAIL > >> > testSetupNetworksReconfigureBridge FAIL > >> > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR > >> > testSetupNetworksRemoveSlavelessBond ERROR > >> > testSetupNetworksResizeBond(False) ERROR > >> > testSetupNetworksResizeBond(True) ERROR > >> > testSetupNetworksStableBond(False) ERROR > >> > testSetupNetworksStableBond(True) ERROR > >> > testStaticNetworkConfig((4, 6)) FAIL > >> > testStaticNetworkConfig((4,)) FAIL > >> > testStaticNetworkConfig((6,)) FAIL > >> > testStaticSourceRouting(False) ERROR > >> > testStaticSourceRouting(True) FAIL > >> > testTwiceAdd(False) ERROR > >> > testTwiceAdd(True) ERROR > >> > testVolatileConfig(False) ERROR > >> > testVolatileConfig(True) ERROR > >> > test_drop_initial_bond_slaves_ip_config FAIL > >> > test_getVdsStats ERROR > >> > test_remove_bond_under_network FAIL > >> > test_remove_initial_network_nic_ip_config FAIL > >> > test_rollback FAIL > >> > test_setupNetworks_bond_with_custom_option ERROR > >> > test_setupNetworks_on_external_bond ERROR > >> > test_setupNetworks_on_external_vlaned_bond ERROR > >> > functional.networkTestsOVS.OVSNetworkTest > >> > testHonorBlockingDhcp SKIP: > >> > Slow > >> > tests are disabled > >> > testLowerMtuDoesNotOverride OK > >> > testNoBridgeLeftovers FAIL > >> > testSelectiveRestoreIgnoresVdsmRegParams FAIL > >> > testSetupNetworkOutboundQos(True) OK > >> > testSetupNetworksAddBadParams(True) OK > >> > testSetupNetworksAddBondWithManyVlans(True) FAIL > >> > testSetupNetworksAddDelBondedNetwork(True) FAIL > >> > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL > >> > testSetupNetworksAddDelDhcp(True, (4,)) FAIL > >> > testSetupNetworksAddDelDhcp(True, (6,)) FAIL > >> > testSetupNetworksAddManyVlans(True) OK > >> > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL > >> > testSetupNetworksAddVlan(True) FAIL > >> > testSetupNetworksConnectivityCheck OK > >> > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: > >> > Slow > >> > tests are disabled > >> > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: > >> > Slow > >> > tests are disabled > >> > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL > >> > testSetupNetworksMtus(True) SKIP: > >> > This > >> > test fails because the 2 different networks are getting configured > with > >> > the > >> > same MTU. The test should assert that the reported MTUs are equal to > the > >> > requested ones. > >> > testSetupNetworksMultiMTUsOverBond(True) FAIL > >> > testSetupNetworksMultiMTUsOverNic(True) FAIL > >> > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL > >> > testSetupNetworksNicless OK > >> > testSetupNetworksOverExistingBridge OK > >> > testSetupNetworksReconfigureBridge FAIL > >> > testStaticNetworkConfig((4, 6)) FAIL > >> > testStaticNetworkConfig((4,)) FAIL > >> > testStaticNetworkConfig((6,)) FAIL > >> > testStaticSourceRouting(True) FAIL > >> > test_drop_initial_bond_slaves_ip_config FAIL > >> > test_ovirtmgmtm_to_ovs FAIL > >> > test_remove_bond_under_network SKIP: > >> > OVS > >> > does not support bridgeless networks > >> > test_remove_initial_network_nic_ip_config OK > >> > test_rollback FAIL > >> > > >> > > >> > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski < > pklic...@redhat.com> > >> > wrote: > >> >> > >> >> I like the idea but I have the same feelings as Francesco. I think > that > >> >> we > >> >> need to make sure that functional tests for each vertical are stable > >> >> before > >> >> enabling this process. > >> >> > >> >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <ee...@redhat.com> wrote: > >> >>> > >> >>> adding also infra team for visibility on the change in CI. > >> >>> also inline. > >> >>> > >> >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani < > from...@redhat.com> > >> >>> wrote: > >> >>>> > >> >>>> ----- Original Message ----- > >> >>>> > From: "Yaniv Bronheim" <ybron...@redhat.com> > >> >>>> > To: de...@ovirt.org, "Francesco Romani" <from...@redhat.com>, > "Nir > >> >>>> > Soffer" <nsof...@redhat.com>, "Piotr Kliczewski" > >> >>>> > <pklic...@redhat.com> > >> >>>> > Cc: "danken" <dan...@redhat.com>, "David Caro" > >> >>>> > <dcaro...@redhat.com>, > >> >>>> > "Eyal Edri" <ee...@redhat.com> > >> >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM > >> >>>> > Subject: Automation CI for vdsm > >> >>>> > >> >>>> [...] > >> >>>> > We want to allow developers to trigger the script once reviews > and > >> >>>> > verification are ready (last step before merge). To do so we > agreed > >> >>>> > to > >> >>>> > add > >> >>>> > Continues Integration flag for each vdsm patch. > >> >>> > >> >>> > >> >>> This flag will be called 'Workflow' or we can name it otherwise, we > >> >>> just > >> >>> need to choose what makes sense. > >> >>> David/Yaniv - Please correct me if I'm wrong. > >> >>> > >> >>>> > >> >>>> Once this flag will be > >> >>>> > signed with +1 it will trigger Jenkins CI to run the check-merged > >> >>>> > script > >> >>>> > (adding new button to gerrit is not an option - you can image > that > >> >>>> > flag as > >> >>>> > a trigger button), on success Jenkins CI flag will turn to +2. on > >> >>>> > fail > >> >>>> > we'll get -1 and once new patchset is ready the developer will > >> >>>> > remove > >> >>>> > the > >> >>>> > +1 and add it back to the Continues Integration flag to > re-trigger > >> >>>> > the > >> >>>> > job. > >> >>>> > > >> >>>> > Please ack the process before we move on with that > >> >>>> > >> >>>> Sounds good, even though I'm a little scared (just gut feeling, no > >> >>>> evidence > >> >>>> whatsoever) that this could add even more complexity and fragility > to > >> >>>> the jenkins > >> >>>> fleet. > >> >>>> > >> >>>> In the long run, when this is reliable, it will help greatly. > >> >>>> In the short term, I'm scared because this can lead to false > >> >>>> positives > >> >>>> and bogus > >> >>>> failures. > >> >>>> > >> >>>> Let me stress I don't have concrete item to share or specific > flaws. > >> >>>> > >> >>>> As action item on me, I will find some time next week to check virt > >> >>>> functional tests, > >> >>>> to see if they need some fixes, work reliably and so forth > >> >>>> > >> >>>> > The patch for those scripts still under review and testing - > >> >>>> > https://gerrit.ovirt.org/#/c/48268 > >> >>>> > >> >>>> Will review asap. > >> >>>> > >> >>>> -- > >> >>>> Francesco Romani > >> >>>> RedHat Engineering Virtualization R & D > >> >>>> Phone: 8261328 > >> >>>> IRC: fromani > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Eyal Edri > >> >>> Supervisor, RHEV CI > >> >>> EMEA ENG Virtualization R&D > >> >>> Red Hat Israel > >> >>> > >> >>> phone: +972-9-7692018 > >> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Yaniv Bronhaim. > > > > > > > > > > -- > > Yaniv Bronhaim. > > > > _______________________________________________ > > Devel mailing list > > de...@ovirt.org > > http://lists.ovirt.org/mailman/listinfo/devel > -- *Yaniv Bronhaim.*
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra