[ovirt-devel] Re: OST's basic suite UI sanity tests optimization

[Martin Perina]

On Fri, Mar 8, 2019 at 12:36 PM Marcin Sobczyk  wrote:

> Greg,
>
> that's a great finding and a very good starting point.
>
> If we want to stick with docker images and Firefox/Chrome testing, I still
> have some ideas, that would shorten the running time even more:
>
>- we do something like this:
>
> log("waiting %s sec for grid to initialize..." % GRID_STARTUP_DELAY)
> time.sleep(GRID_STARTUP_DELAY)
>
>   this is very inefficient. We can change that to something like I wrote
> here (_wait_for_selenium_hub):
>
>
> https://gerrit.ovirt.org/#/c/98135/2/common/test-scenarios-files/selenium/navigation/selenium_on_engine.py
>
>   This function probably needs some improvement (i.e. urllib3 spits out
> warnings on an unsuccessful connection attempt, so they would need to be
> silenced), but that's a far better approach than a simple sleep.
>
>- parallelize running Firefox and Chrome tests - there's no reason not
>to run them both at the same time. There's something called
>VectorThread in lago.utils. A simple example of usage can be found in
>'004_basic_sanity.py:955' (disk_operations function). This would have
>a nice side effect of getting rid of the ugly global ovirt_driver -
>each thread would have it's own.
>
>
>- maybe not a running-time improvement, but I think
>https://gerrit.ovirt.org/#/c/98127/ is still relevant - the way we
>call save_screenshot is ugly and much too verbose
>
> Right now, I have to switch my focus to some important stuff in VDSM - the
> OST patches were a continuation of a hackathon effort and something like a
> "side-project" ;) Still, I don't want the tread to die. I think there's a
> lot of room for improvements. I can rebase/improve some of my patches if
> you find them useful. Please keep me posted with your efforts!
>
> Regards, Marcin
> On 3/7/19 11:10 PM, Greg Sheremeta wrote:
>
> Marcin,
>
> It just dawned on me that the main reason 008's start_grid takes so long
> is that the docker images are fresh pulled every time. Several hundred MB,
> every time (ugh, sorry). We can and should cache them. What do you think
> about trying this before doing anything else? [it would also be a good time
> to update from actinium to the latest, iron.]
>
> @Barak Korren  you once mentioned to me we should
> cache these if they are ok to cache (they are). How do we do that?
>
>
Gal/Gallit/Barak, so is there any way how to store those docker containers
within image of lago VM which runs OST tests?

>
> docker.io/selenium/node-chrome-debug   3.9.1-actinium  327adc897d23
>   13 months ago   *904 MB*
> docker.io/selenium/node-firefox-debug   3.9.1-actinium  88649b420bd5
>   13 months ago   *814 MB*
>
> Greg
>
>
> On Tue, Mar 5, 2019 at 6:15 AM Greg Sheremeta  wrote:
>
>>
>> On Tue, Mar 5, 2019 at 4:55 AM Marcin Sobczyk 
>> wrote:
>>
>>> Hi,
>>> On 3/4/19 7:07 PM, Greg Sheremeta wrote:
>>>
>>> Hi,
>>>
>>> Thanks for trying to improve the tests!
>>>
>>> I'm reluctant to give up Firefox sanity tests on every commit, though.
>>> In fact, I wanted to add Edge and Safari, because those are also supported
>>> browsers. Just today a Firefox only issue was reported, so they are
>>> valuable.
>>>
>>> Was the Firefox-only issue detected by basic suite or some other tests?
>>>
>> It was reported by a developer. Because GWT compiles permutations per
>> browser, and each browser therefore loads completely separate JavaScript
>> payloads, it's just too easy for it to break in one browser and be fine in
>> the other, so I'm really not ok to remove Firefox.
>>
>> If Admin Portal was React where there is a single JavaScript payload
>> that's shared among all browsers, then I'd consider it.
>>
>>>
>>> Did you consider either leaving a grid up permanently or perhaps using a
>>> third party like saucelabs?
>>>
>>> I did consider simply having our own grid for the OST.
>>> There's even a thread somewhere on ovirt-devel, where someone found OST
>>> trying to connect to one of my VMs in Tel Aviv, where my own grid was
>>> running :D
>>> I couldn't make a public demo though - OST executors couldn't see my VM
>>> in tlv.
>>>
>>> This approach has 2 big flaws:
>>>
>>>- it requires quite a lot of resources for the grid to always be
>>>there for us
>>>
>>> What about Saucelabs or another third party free tool?
>>
>>
>>>
>>>- it makes OST running times somehow undeterministic - situations,
>>>where WebDriver has to wait for Selenium hub/nodes to be free, will
>>>probably take place
>>>
>>> The way I see basic suite's UI sanity tests, is that they're exactly
>>> what they're called - sanity tests.
>>> We do trivial checks like "can we log in to the webadmin site", "can we
>>> go to 'virtual machines' sub-page".
>>> I'm not in favor of dropping these completely - I think they make sense,
>>> but I also think we can live with a trimmed-down version that saves a lot
>>> of time.
>>> As I said - AFAIK QE have their own Selenium grid, where 

[ovirt-devel] Re: [URGENT] stop merging to vdsm master

[Sandro Bonazzola]

Adding missing devel list.

Il giorno mar 12 mar 2019 alle ore 10:24 Dafna Ron  ha
scritto:

> Hi,
>
> We have been continuously failing CQ on the vdsm project since 07-03-2019.
>
> The latest failure is still in vm migration test and root cause is still
> pointing to:
>
> https://gerrit.ovirt.org/#/c/97381/22 - virt: In FIPS mode, use SASL auth
> instead of qemu passwords
>
> Please stop merging until issue is resolved as we are risking further
> regressions being merged.
>
> Thanks,
> Dafna
>
>

-- 

SANDRO BONAZZOLA

MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV

Red Hat EMEA 

sbona...@redhat.com

___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GF7MLYRKXEIDZZRDNB6X7B6AY5SPRLV2/

[ovirt-devel] Re: [URGENT] stop merging to vdsm master

[Ryan Barry]

Very simply, let's revert the patch and fix OST offline to unblock CQ. This
was my suggestion if it could not be resolved by end of day yesterday.
Let's do it.

On Tue, Mar 12, 2019 at 6:18 AM Sandro Bonazzola 
wrote:

> Adding missing devel list.
>
> Il giorno mar 12 mar 2019 alle ore 10:24 Dafna Ron  ha
> scritto:
>
>> Hi,
>>
>> We have been continuously failing CQ on the vdsm project since
>> 07-03-2019.
>>
>> The latest failure is still in vm migration test and root cause is still
>> pointing to:
>>
>> https://gerrit.ovirt.org/#/c/97381/22 - virt: In FIPS mode, use SASL
>> auth instead of qemu passwords
>>
>> Please stop merging until issue is resolved as we are risking further
>> regressions being merged.
>>
>> Thanks,
>> Dafna
>>
>>
>
> --
>
> SANDRO BONAZZOLA
>
> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>
> Red Hat EMEA 
>
> sbona...@redhat.com
> 
>


-- 

Ryan Barry

Associate Manager - RHV Virt/SLA

rba...@redhat.comM: +16518159306 IM: rbarry

___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6WFPMDKF4NFT7QZQSCKI4W6UIZQLARN6/

[ovirt-devel] Re: [URGENT] stop merging to vdsm master

[Doron Fediuck]

On Tue, 12 Mar 2019 at 12:34, Ryan Barry  wrote:

> Very simply, let's revert the patch and fix OST offline to unblock CQ.
> This was my suggestion if it could not be resolved by end of day yesterday.
> Let's do it.
>

+1, as long as you make sure the relevant people are aware.

>
> On Tue, Mar 12, 2019 at 6:18 AM Sandro Bonazzola 
> wrote:
>
>> Adding missing devel list.
>>
>> Il giorno mar 12 mar 2019 alle ore 10:24 Dafna Ron  ha
>> scritto:
>>
>>> Hi,
>>>
>>> We have been continuously failing CQ on the vdsm project since
>>> 07-03-2019.
>>>
>>> The latest failure is still in vm migration test and root cause is still
>>> pointing to:
>>>
>>> https://gerrit.ovirt.org/#/c/97381/22 - virt: In FIPS mode, use SASL
>>> auth instead of qemu passwords
>>>
>>> Please stop merging until issue is resolved as we are risking further
>>> regressions being merged.
>>>
>>> Thanks,
>>> Dafna
>>>
>>>
>>
>> --
>>
>> SANDRO BONAZZOLA
>>
>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>>
>> Red Hat EMEA 
>>
>> sbona...@redhat.com
>> 
>>
>
>
> --
>
> Ryan Barry
>
> Associate Manager - RHV Virt/SLA
>
> rba...@redhat.comM: +16518159306 IM: rbarry
> 
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/TM6WRF6AUK5TVMZNGSFFYJU44USWEXFV/

[ovirt-devel] [hyperv] hvinfo: command line tool to get the HyperV enlightements status from within the guest

[Francesco Romani]

Hi all,


lately I've been involved again in hyperv support. I was tasked to help 
improve the testability of the hyperv guest configuration.


Hyperv support is a set of optimizations that libvirt/qemu offer to 
improve the runtime behaviour (stability, performance) of Windows guets.



See for example: https://libvirt.org/formatdomain.html#elementsFeatures

oVirt does support them (see for example 
https://github.com/oVirt/vdsm/blob/master/lib/vdsm/virt/libvirtxml.py#L243) 
and will keep supporting them


because they are a key feature for windows guests.


Up until now the easiest (only?) way to check if a hyperv optmization 
was really enabled was to inspect the libvirt XML and/or the QEMU 
command line flags


But we wanted to have another check.


Enter hvinfo: https://github.com/fromanirh/hvinfo

hvinfo is a simple tool that decodes CPUID informations according to the 
publicly-available HyperV specs 
(https://github.com/MicrosoftDocs/Virtualization-Documentation/tree/live/tlfs) 
and report what the guest sees.


It takes no arguments - just run it!-, it requires no special privileges 
and emits easy to consume JSON. It is designed to help and integrate 
into fully automated CI/QA.



Being a commandline tool, is hard to give a "screenshot", so let me just 
report sample output (admin privileges not actually required)


Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Users\admin> cd .\Downloads\
PS C:\Users\admin\Downloads> .\hvinfo.exe
{
  "HyperVsupport": true,
  "Features": {
    "GuestDebugging": false,
    "PerformanceMonitor": false,
    "PCPUDynamicPartitioningEvents": true,
    "HypercallInputParamsXMM": false,
    "VirtualGuestIdleState": false,
    "HypervisorSleepState": false,
    "NUMADistanceQuery": false,
    "TimerFrequenciesQuery": false,
    "SytheticMCEInjection": false,
    "GuestCrashMSR": false,
    "DebugMSR": false,
    "NPIEP": false,
    "DisableHypervisorAvailable": false,
    "ExtendedGvaRangesForFlushVirtualAddressList": false,
    "HypercallOutputReturnXMM": false,
    "SintPollingMode": false,
    "HypercallMsrLock": false,
    "UseDirectSyntheticTimers": false
  },
  "Recommendations": {
    "HypercallAddressSpaceSwitch": false,
    "HypercallLocalTLBFlush": false,
    "HypercallRemoteTLBFlush": false,
    "MSRAPICRegisters": true,
    "MSRSystemReset": false,
    "RelaxedTiming": true,
    "DMARemapping": false,
    "InterruptRemapping": false,
    "X2APICMSR": false,
    "DeprecatingAutoEOI": false,
    "SyntheticClusterIPI": false,
    "ExProcessorMasks": false,
    "Nested": false,
    "INTForMBECSyscalls": false,
    "NestedEVMCS": false,
    "SyncedTimeline": false,
    "DirectLocalFlushEntire": false,
    "NoNonArchitecturalCoreSharing": false,
    "SpinlockRetries": 8191
  }
}
PS C:\Users\admin\Downloads>


Caveat: the name of the features are the same of the spec, so we need 
mappings for oVirt flags, libvirt flags and so on.


For example

libvirt xml

domain.features.hyperv.relaxed[state="on"]

maps to hvinfo json

Features.Recommendations.RelaxedTiming=true


Feel free to test it and report any issue


Bests,


--
Francesco Romani
Senior SW Eng., Virtualization R&D
Red Hat
IRC: fromani github: @fromanirh
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/Z6OSC4RLC3TXINE2FV5NKYYHOI5L7PLN/

[ovirt-devel] ovirt-engine has been tagged (ovirt-engine-4.3.2.1)

[Tal Nisan]

___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NGASAOCOQETSZCDDW4YYFYOCDNDYIOBX/

[ovirt-devel] Re: [hyperv] hvinfo: command line tool to get the HyperV enlightements status from within the guest

[Michal Skrivanek]

I wonder if it makes sense to add this to qemu/ga, actually?
It takes time, but ultimately it’s the one common tool for guest side
stuff. At least worth asking if there is any interest, I guess

Thanks,
michal

> On 12 Mar 2019, at 13:30, Francesco Romani  wrote:
>
> Hi all,
>
>
> lately I've been involved again in hyperv support. I was tasked to help 
> improve the testability of the hyperv guest configuration.
>
> Hyperv support is a set of optimizations that libvirt/qemu offer to improve 
> the runtime behaviour (stability, performance) of Windows guets.
>
>
> See for example: https://libvirt.org/formatdomain.html#elementsFeatures
>
> oVirt does support them (see for example 
> https://github.com/oVirt/vdsm/blob/master/lib/vdsm/virt/libvirtxml.py#L243) 
> and will keep supporting them
>
> because they are a key feature for windows guests.
>
>
> Up until now the easiest (only?) way to check if a hyperv optmization was 
> really enabled was to inspect the libvirt XML and/or the QEMU command line 
> flags
>
> But we wanted to have another check.
>
>
> Enter hvinfo: https://github.com/fromanirh/hvinfo
>
> hvinfo is a simple tool that decodes CPUID informations according to the 
> publicly-available HyperV specs 
> (https://github.com/MicrosoftDocs/Virtualization-Documentation/tree/live/tlfs)
>  and report what the guest sees.
>
> It takes no arguments - just run it!-, it requires no special privileges and 
> emits easy to consume JSON. It is designed to help and integrate into fully 
> automated CI/QA.
>
>
> Being a commandline tool, is hard to give a "screenshot", so let me just 
> report sample output (admin privileges not actually required)
>
> Windows PowerShell
> Copyright (C) Microsoft Corporation. All rights reserved.
>
> PS C:\Users\admin> cd .\Downloads\
> PS C:\Users\admin\Downloads> .\hvinfo.exe
> {
>   "HyperVsupport": true,
>   "Features": {
> "GuestDebugging": false,
> "PerformanceMonitor": false,
> "PCPUDynamicPartitioningEvents": true,
> "HypercallInputParamsXMM": false,
> "VirtualGuestIdleState": false,
> "HypervisorSleepState": false,
> "NUMADistanceQuery": false,
> "TimerFrequenciesQuery": false,
> "SytheticMCEInjection": false,
> "GuestCrashMSR": false,
> "DebugMSR": false,
> "NPIEP": false,
> "DisableHypervisorAvailable": false,
> "ExtendedGvaRangesForFlushVirtualAddressList": false,
> "HypercallOutputReturnXMM": false,
> "SintPollingMode": false,
> "HypercallMsrLock": false,
> "UseDirectSyntheticTimers": false
>   },
>   "Recommendations": {
> "HypercallAddressSpaceSwitch": false,
> "HypercallLocalTLBFlush": false,
> "HypercallRemoteTLBFlush": false,
> "MSRAPICRegisters": true,
> "MSRSystemReset": false,
> "RelaxedTiming": true,
> "DMARemapping": false,
> "InterruptRemapping": false,
> "X2APICMSR": false,
> "DeprecatingAutoEOI": false,
> "SyntheticClusterIPI": false,
> "ExProcessorMasks": false,
> "Nested": false,
> "INTForMBECSyscalls": false,
> "NestedEVMCS": false,
> "SyncedTimeline": false,
> "DirectLocalFlushEntire": false,
> "NoNonArchitecturalCoreSharing": false,
> "SpinlockRetries": 8191
>   }
> }
> PS C:\Users\admin\Downloads>
>
>
> Caveat: the name of the features are the same of the spec, so we need 
> mappings for oVirt flags, libvirt flags and so on.
>
> For example
>
> libvirt xml
>
> domain.features.hyperv.relaxed[state="on"]
>
> maps to hvinfo json
>
> Features.Recommendations.RelaxedTiming=true
>
>
> Feel free to test it and report any issue
>
>
> Bests,
>
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R&D
> Red Hat
> IRC: fromani github: @fromanirh
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: 
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives: 
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/Z6OSC4RLC3TXINE2FV5NKYYHOI5L7PLN/
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5U4L25COXZDCZO2NMX6T7PGUE4NREVMC/