exactly - just posted a patch again that sings all fails as broken . we'll get the report soon and I'll publish it as well. hope the run will take less time now
On Wed, Dec 16, 2015 at 6:19 PM, Nir Soffer <nsof...@redhat.com> wrote: > Nice, but we cannot enable this until all the tests pass or disabled. > > There is no point in broken or flaky functional tests. > > On Wed, Dec 16, 2015 at 5:34 PM, Yaniv Bronheim <ybron...@redhat.com> > wrote: > > So its not stable. It won't block merges and at least give us report > after > > each merge. It takes really long time to run it (because of the tests > > themselves. Lago things takes maximum 15minutes, but the run last for > more > > than 2hrs right now and I suspect functional/storageTests.py gets stuck) > > > > Bellow you can see where we stand (before I added python-rtslib package). > > > > Now, I still want to merge the patch https://gerrit.ovirt.org/#/c/48268/ > - > > which enables this run after merges, and I still want you to consider the > > addition of Automation CI flag to our gerrit so that developer will be > able > > to use it as a trigger for the check-merged.sh script run, just to see if > > their patch fixes\brakes something realted to the functional tests > > > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1480/ > - is > > an example of how the run looks like. I still work to improve the output > > > > > > Please reply and let me know if the idea around the automation flag is > > acceptable by you.. and please review the patch for comments and acks. > > We can ask dcaro to add the flag until Friday, otherwise we'll need to > delay > > this effort after the holiday.. > > > > > > functional.sosPluginTests.SosPluginTest > > testSosPlugin OK > > functional.vmRecoveryTests.RecoveryTests > > test_vm_recovery FAIL > > functional.vmQoSTests.VMQosTests > > testSmallVMBallooning FAIL > > functional.virtTests.VirtTest > > testComplexVm FAIL > > testHeadlessVm OK > > testSimpleVm FAIL > > testVmDefinitionGraphics('spice') FAIL > > testVmDefinitionGraphics('vnc') OK > > testVmDefinitionLegacyGraphics('qxl') FAIL > > testVmDefinitionLegacyGraphics('vnc') OK > > testVmDefinitionMultipleGraphics('spice', 'vnc') FAIL > > testVmDefinitionMultipleGraphics('vnc', 'spice') FAIL > > testVmWithCdrom('self') FAIL > > testVmWithCdrom('specParams') FAIL > > testVmWithCdrom('vmPayload') FAIL > > testVmWithDevice('hotplugDisk') FAIL > > testVmWithDevice('hotplugNic') FAIL > > testVmWithDevice('smartcard') FAIL > > testVmWithDevice('virtioNic') FAIL > > testVmWithDevice('virtioRng') FAIL > > testVmWithSla FAIL > > testVmWithStorage('iscsi') SKIP: > > python-rtslib is not installed. > > testVmWithStorage('localfs') FAIL > > testVmWithStorage('nfs') FAIL > > functional.storageTests.StorageTest > > testCreatePoolErrors OK > > testStorage('glusterfs', 0) ERROR > > testStorage('glusterfs', 3) ERROR > > testStorage('iscsi', 0) SKIP: > > python-rtslib is not installed. > > testStorage('iscsi', 3) SKIP: > > python-rtslib is not installed. > > testStorage('localfs', 0) FAIL > > testStorage('localfs', 3) FAIL > > testStorage('nfs', 0) FAIL > > testStorage('nfs', 3) FAIL > > functional.networkTests.NetworkTest > > testAddVlanedBridgeless ERROR > > testAddVlanedBridgeless_oneCommand ERROR > > testAfterNetworkSetupHook ERROR > > testBeforeNetworkSetupHook ERROR > > testBondHwAddress(False) ERROR > > testBondHwAddress(True) ERROR > > testBrokenNetworkReplacement(False) ERROR > > testBrokenNetworkReplacement(True) ERROR > > testDelNetworkBondAccumulation ERROR > > testDelNetworkWithMTU(False) ERROR > > testDelNetworkWithMTU(True) ERROR > > testDelWithoutAdd ERROR > > testDhclientLeases(4, 'default') ERROR > > testDhclientLeases(4, 'local') ERROR > > testDhclientLeases(6, None) ERROR > > testDhcpReplaceNicWithBridge ERROR > > testFailWithInvalidBondingName(False) ERROR > > testFailWithInvalidBondingName(True) ERROR > > testFailWithInvalidBridgeName ERROR > > testFailWithInvalidIpConfig ERROR > > testFailWithInvalidNic(False) ERROR > > testFailWithInvalidNic(True) ERROR > > testFailWithInvalidParams(False) ERROR > > testFailWithInvalidParams(True) ERROR > > testGetRouteDeviceTo ERROR > > testHonorBlockingDhcp SKIP: > Slow > > tests are disabled > > testIpLinkWrapper ERROR > > testLowerMtuDoesNotOverride FAIL > > testNoBridgeLeftovers FAIL > > testReconfigureBrNetWithVanishedPort ERROR > > testRedefineBondedNetworkIPs ERROR > > testRemovingBridgeDoesNotLeaveBridge ERROR > > testReorderBondingOptions(False) ERROR > > testReorderBondingOptions(True) ERROR > > testRestoreNetworksOnlyRestoreUnchangedDevices ERROR > > testRestoreToBlockingDHCP ERROR > > testSafeNetworkConfig(False) ERROR > > testSafeNetworkConfig(True) ERROR > > testSelectiveRestoreDuringUpgrade ERROR > > testSelectiveRestoreIgnoresVdsmRegParams FAIL > > testSetupNetworkOutboundQos(False) ERROR > > testSetupNetworkOutboundQos(True) FAIL > > testSetupNetworksActiveSlave ERROR > > testSetupNetworksAddBadParams(False) ERROR > > testSetupNetworksAddBadParams(True) FAIL > > testSetupNetworksAddBondWithManyVlans(False) ERROR > > testSetupNetworksAddBondWithManyVlans(True) FAIL > > testSetupNetworksAddDelBondedNetwork(False) ERROR > > testSetupNetworksAddDelBondedNetwork(True) FAIL > > testSetupNetworksAddDelDhcp(False, (4, 6)) ERROR > > testSetupNetworksAddDelDhcp(False, (4,)) ERROR > > testSetupNetworksAddDelDhcp(False, (6,)) ERROR > > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL > > testSetupNetworksAddDelDhcp(True, (4,)) FAIL > > testSetupNetworksAddDelDhcp(True, (6,)) FAIL > > testSetupNetworksAddManyVlans(False) ERROR > > testSetupNetworksAddManyVlans(True) FAIL > > testSetupNetworksAddNetworkToNicAfterBondBreaking(False) ERROR > > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL > > testSetupNetworksAddNetworkToNicAfterBondResizing(False) ERROR > > testSetupNetworksAddNetworkToNicAfterBondResizing(True) ERROR > > testSetupNetworksAddOverExistingBond(False) ERROR > > testSetupNetworksAddOverExistingBond(True) ERROR > > testSetupNetworksAddVlan(False) ERROR > > testSetupNetworksAddVlan(True) FAIL > > testSetupNetworksConnectivityCheck FAIL > > testSetupNetworksConvertVlanNetBridgeness ERROR > > testSetupNetworksDelOneOfBondNets ERROR > > testSetupNetworksDeletesTheBridgeOnlyWhenItIsReconfigured ERROR > > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(False)ERROR > > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: > Slow > > tests are disabled > > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(False)ERROR > > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: > Slow > > tests are disabled > > testSetupNetworksKeepNetworkOnBondAfterBondResizing(False) ERROR > > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL > > testSetupNetworksMtus(False) ERROR > > testSetupNetworksMtus(True) SKIP: > This > > test fails because the 2 different networks are getting configured with > the > > same MTU. The test should assert that the reported MTUs are equal to the > > requested ones. > > testSetupNetworksMultiMTUsOverBond(False) ERROR > > testSetupNetworksMultiMTUsOverBond(True) FAIL > > testSetupNetworksMultiMTUsOverNic(False) ERROR > > testSetupNetworksMultiMTUsOverNic(True) FAIL > > testSetupNetworksNetCompatibilityMultipleNetsSameNic(False) ERROR > > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL > > testSetupNetworksNicless FAIL > > testSetupNetworksNiclessBridgeless ERROR > > testSetupNetworksOverDhcpIface ERROR > > testSetupNetworksOverExistingBridge FAIL > > testSetupNetworksReconfigureBridge FAIL > > testSetupNetworksRemoveBondWithKilledEnslavedNics ERROR > > testSetupNetworksRemoveSlavelessBond ERROR > > testSetupNetworksResizeBond(False) ERROR > > testSetupNetworksResizeBond(True) ERROR > > testSetupNetworksStableBond(False) ERROR > > testSetupNetworksStableBond(True) ERROR > > testStaticNetworkConfig((4, 6)) FAIL > > testStaticNetworkConfig((4,)) FAIL > > testStaticNetworkConfig((6,)) FAIL > > testStaticSourceRouting(False) ERROR > > testStaticSourceRouting(True) FAIL > > testTwiceAdd(False) ERROR > > testTwiceAdd(True) ERROR > > testVolatileConfig(False) ERROR > > testVolatileConfig(True) ERROR > > test_drop_initial_bond_slaves_ip_config FAIL > > test_getVdsStats ERROR > > test_remove_bond_under_network FAIL > > test_remove_initial_network_nic_ip_config FAIL > > test_rollback FAIL > > test_setupNetworks_bond_with_custom_option ERROR > > test_setupNetworks_on_external_bond ERROR > > test_setupNetworks_on_external_vlaned_bond ERROR > > functional.networkTestsOVS.OVSNetworkTest > > testHonorBlockingDhcp SKIP: > Slow > > tests are disabled > > testLowerMtuDoesNotOverride OK > > testNoBridgeLeftovers FAIL > > testSelectiveRestoreIgnoresVdsmRegParams FAIL > > testSetupNetworkOutboundQos(True) OK > > testSetupNetworksAddBadParams(True) OK > > testSetupNetworksAddBondWithManyVlans(True) FAIL > > testSetupNetworksAddDelBondedNetwork(True) FAIL > > testSetupNetworksAddDelDhcp(True, (4, 6)) FAIL > > testSetupNetworksAddDelDhcp(True, (4,)) FAIL > > testSetupNetworksAddDelDhcp(True, (6,)) FAIL > > testSetupNetworksAddManyVlans(True) OK > > testSetupNetworksAddNetworkToNicAfterBondBreaking(True) FAIL > > testSetupNetworksAddVlan(True) FAIL > > testSetupNetworksConnectivityCheck OK > > testSetupNetworksEmergencyDevicesCleanupBondOverwrite(True) SKIP: > Slow > > tests are disabled > > testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(True) SKIP: > Slow > > tests are disabled > > testSetupNetworksKeepNetworkOnBondAfterBondResizing(True) FAIL > > testSetupNetworksMtus(True) SKIP: > This > > test fails because the 2 different networks are getting configured with > the > > same MTU. The test should assert that the reported MTUs are equal to the > > requested ones. > > testSetupNetworksMultiMTUsOverBond(True) FAIL > > testSetupNetworksMultiMTUsOverNic(True) FAIL > > testSetupNetworksNetCompatibilityMultipleNetsSameNic(True) FAIL > > testSetupNetworksNicless OK > > testSetupNetworksOverExistingBridge OK > > testSetupNetworksReconfigureBridge FAIL > > testStaticNetworkConfig((4, 6)) FAIL > > testStaticNetworkConfig((4,)) FAIL > > testStaticNetworkConfig((6,)) FAIL > > testStaticSourceRouting(True) FAIL > > test_drop_initial_bond_slaves_ip_config FAIL > > test_ovirtmgmtm_to_ovs FAIL > > test_remove_bond_under_network SKIP: OVS > > does not support bridgeless networks > > test_remove_initial_network_nic_ip_config OK > > test_rollback FAIL > > > > > > On Sun, Dec 13, 2015 at 10:07 PM, Piotr Kliczewski <pklic...@redhat.com> > > wrote: > >> > >> I like the idea but I have the same feelings as Francesco. I think that > we > >> need to make sure that functional tests for each vertical are stable > before > >> enabling this process. > >> > >> On Sun, Dec 13, 2015 at 8:34 AM, Eyal Edri <ee...@redhat.com> wrote: > >>> > >>> adding also infra team for visibility on the change in CI. > >>> also inline. > >>> > >>> On Fri, Dec 11, 2015 at 4:19 PM, Francesco Romani <from...@redhat.com> > >>> wrote: > >>>> > >>>> ----- Original Message ----- > >>>> > From: "Yaniv Bronheim" <ybron...@redhat.com> > >>>> > To: de...@ovirt.org, "Francesco Romani" <from...@redhat.com>, "Nir > >>>> > Soffer" <nsof...@redhat.com>, "Piotr Kliczewski" > >>>> > <pklic...@redhat.com> > >>>> > Cc: "danken" <dan...@redhat.com>, "David Caro" <dcaro...@redhat.com > >, > >>>> > "Eyal Edri" <ee...@redhat.com> > >>>> > Sent: Thursday, December 10, 2015 6:46:37 PM > >>>> > Subject: Automation CI for vdsm > >>>> > >>>> [...] > >>>> > We want to allow developers to trigger the script once reviews and > >>>> > verification are ready (last step before merge). To do so we agreed > to > >>>> > add > >>>> > Continues Integration flag for each vdsm patch. > >>> > >>> > >>> This flag will be called 'Workflow' or we can name it otherwise, we > just > >>> need to choose what makes sense. > >>> David/Yaniv - Please correct me if I'm wrong. > >>> > >>>> > >>>> Once this flag will be > >>>> > signed with +1 it will trigger Jenkins CI to run the check-merged > >>>> > script > >>>> > (adding new button to gerrit is not an option - you can image that > >>>> > flag as > >>>> > a trigger button), on success Jenkins CI flag will turn to +2. on > fail > >>>> > we'll get -1 and once new patchset is ready the developer will > remove > >>>> > the > >>>> > +1 and add it back to the Continues Integration flag to re-trigger > the > >>>> > job. > >>>> > > >>>> > Please ack the process before we move on with that > >>>> > >>>> Sounds good, even though I'm a little scared (just gut feeling, no > >>>> evidence > >>>> whatsoever) that this could add even more complexity and fragility to > >>>> the jenkins > >>>> fleet. > >>>> > >>>> In the long run, when this is reliable, it will help greatly. > >>>> In the short term, I'm scared because this can lead to false positives > >>>> and bogus > >>>> failures. > >>>> > >>>> Let me stress I don't have concrete item to share or specific flaws. > >>>> > >>>> As action item on me, I will find some time next week to check virt > >>>> functional tests, > >>>> to see if they need some fixes, work reliably and so forth > >>>> > >>>> > The patch for those scripts still under review and testing - > >>>> > https://gerrit.ovirt.org/#/c/48268 > >>>> > >>>> Will review asap. > >>>> > >>>> -- > >>>> Francesco Romani > >>>> RedHat Engineering Virtualization R & D > >>>> Phone: 8261328 > >>>> IRC: fromani > >>> > >>> > >>> > >>> > >>> -- > >>> Eyal Edri > >>> Supervisor, RHEV CI > >>> EMEA ENG Virtualization R&D > >>> Red Hat Israel > >>> > >>> phone: +972-9-7692018 > >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) > >> > >> > > > > > > > > -- > > Yaniv Bronhaim. > -- *Yaniv Bronhaim.*
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra