Re: [ovirt-devel] [lago-devel] ovirt tests failing on missing libxml2-python

2016-07-01 Thread Yaniv Kaul 
On Wed, Jun 29, 2016 at 11:15 PM, Barak Korren  wrote:

> On 29 June 2016 at 21:45, Yaniv Kaul  wrote:
> > On Mon, Jun 27, 2016 at 9:45 AM, Barak Korren 
> wrote:
> >>
> >> >
> >> >
> >> > It means that packages will be fetched EVERY time from outside, which
> >> > may be
> >> > slow(er).
> >> > Y.
> >> >
> >>
> >> We can (and mostly already have) setup simple caches to prevent that.
> >
> >
> > How do you set up cache on a developer's laptop?
> >
> We may have been unclear in our intentions, we want to make the
> pre-syncing optional not remove it completely. It does make sense on
> the laptop (sometimes), but not so much in the CI env.
>
> > The repo directory does not need to be cleaned every time.
>
> This is an assumption that may break if we end up having any corrupt
> or failing packages in the cache. It also make it hard to "go back in
> time" if we want to test without some update.
> (Cleaning corrupt caches an re-running is easy in a local setting, in
> CI you end up dealing with angry devs getting false '-1's)
>

True, and we don't want that. Developers have to trust the CI system.
This is an important point.


>
> > It can also be
> > resync'ed from a central repo - which still going to be faster than any
> > other fetching.
> > (hopefully sync'ed into the slave /dev/shm btw).
>
> It could be faster, but could also be slower if you end up fetching
> more then you have to. (if engine setup fails on missing dependency,
> you just spent needless time fetching VDSM dpes)
> Also fetching by itself may not be the bottleneck in all cases, it is
> surely slow when fetching from PHX to TLV, but when fetching from the
> Squid proxy's RAM inside PHX it can actually end up being faster then
> copying from the local disk.
>

I always fetch and store on /dev/shm/repostore
It's faster than anything else.

I did copy its content once to the disk, so when the host reboots, it
rsync's this to /dev/shm/repostore , then tests begin.
That perhaps is indeed not very needed in CI.


> >> The existing solution looks more like premature optimization gone badly
> >> IMO.
> >
> > Try to run ovirt-system-tests, clean the repo and re-run - it's 20-30
> > minutes at least longer - which is far more than what it takes to run the
> > whole test suite.
>
> I wonder how many of those minutes are spend on fetching things we
> actually need, and how much is spent on overhead. I suspect that
> without a local cache, the test run will be longer, but not as long as
> the pre-fetching+tests takes currently. More importantly, this may
> allow the CI to fail faster. I think we should at least test that.
>
> > I completely agree the manual maintenance is an annoyance, wish we had
> > something in between.
>
> Maybe we can take a middle ground, pre-fetch, but also enable external
> repos in CI (perhaps with some way to log and find out what was not
> pre-fetched).
>

This is what the code is supposed to do, I suspect. reposync syncs between
what you already have and what you fetch, no?
Y.


> --
> Barak Korren
> bkor...@redhat.com
> RHEV-CI Team
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] oVirt 4.0.1 First Release candidate is now available for testing

2016-07-01 Thread Rafael Martins 
The oVirt Project is pleased to announce the availability of the First
Release Candidate of oVirt 4.0.1 for testing, as of July 1st, 2016.

This is pre-release software. Please take a look at our community page[1]
to know how to ask questions and interact with developers and users.
All issues or bugs should be reported via oVirt Bugzilla[2].
This pre-release should not to be used in production.

This release is available now for:
* Fedora 23
* Red Hat Enterprise Linux 7.2 or later
* CentOS Linux (or similar) 7.2 or later

This release supports Hypervisor Hosts running:
* Red Hat Enterprise Linux 7.2 or later
* CentOS Linux (or similar) 7.2 or later
* Fedora 23
* oVirt Next Generation Node 4.0

See the release notes draft [3] for installation / upgrade instructions and
a list of new features and bugs fixed.

Notes:
* A new oVirt Live ISO is already available [4].
* A new oVirt Next Generation Node is already available [4]
* A new oVirt Engine Appliance is already available.
* Mirrors[5] might need up to one day to synchronize.

Additional Resources:
* Read more about the oVirt 4.0.1 release candidate highlights:
  http://www.ovirt.org/release/4.0.1/
* Get more oVirt Project updates on Twitter: https://twitter.com/ovirt
* Check out the latest project news on the oVirt blog:
  http://www.ovirt.org/blog/

[1] https://www.ovirt.org/community/
[2] https://bugzilla.redhat.com/enter_bug.cgi?classification=oVirt
[3] http://www.ovirt.org/release/4.0.1/
[4] http://resources.ovirt.org/pub/ovirt-4.0-pre/iso/
[5] http://www.ovirt.org/Repository_mirrors#Current_mirrors

--
Rafael Martins
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] [ANN] oVirt 3.6.7 Final Release is now available

2016-07-01 Thread Rafael Martins 
The oVirt Project is pleased to announce today the general availability of
oVirt 3.6.7.

This latest community release includes new features and bug fixes for

- oVirt Engine
- oVirt Engine Appliance
- oVirt Hosted Enging HA
- oVirt Hosted Engine Setup
- oVirt Engine SDK
- oVirt Engine Java SDK
- MOM
- OTOPI
- VDSM

oVirt is an open-source, openly-governed enterprise virtualization
management application, developed by a global community. You can use the
oVirt management interface (oVirt Engine) to manage hardware nodes, storage
and network resources, and to deploy and monitor virtual machines running
in your data center.

If you are familiar with VMware products, oVirt is conceptually similar to
vSphere. oVirt serves as the bedrock for Red Hat's Enterprise
Virtualization product, and it is the "upstream" project where new features
are developed prior to their inclusion in Red Hat's supported product
offering.

Notes:
* A new oVirt Live ISO and a new oVirt Engine Appliance are available.
* Mirrors might need up to one day to synchronize.


Additional Resources:
* Read more about the oVirt 3.6.7 release highlights:
  http://www.ovirt.org/release/3.6.7/
* Get more oVirt Project updates on Twitter: https://twitter.com/ovirt
* Check out the latest project news on the oVirt blog:
  http://www.ovirt.org/blog/

--
Rafael Martins
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [VDSM] Test fails only under make check

2016-07-01 Thread Nir Soffer 
On Fri, Jul 1, 2016 at 6:35 PM, Adam Litke  wrote:
> I have written a new test [1] and when running 'make check' I get a
> nasty ImportError (see below).  When running the same test using
> run_tests_local.sh directly it works fine.  Any ideas what might be
> going on?
>
> [1] https://gerrit.ovirt.org/#/c/60060/1/tests/storage_hsm_test.py
>
> ==
> ERROR: Failure: ImportError (No module named 'Queue')
> --
> Traceback (most recent call last):
>  File "/usr/lib/python3.4/site-packages/nose/failure.py", line 39, in
> runTest
>raise self.exc_val.with_traceback(self.tb)
>  File "/usr/lib/python3.4/site-packages/nose/loader.py", line 418, in
> loadTestsFromName
>addr.filename, addr.module)
>  File "/usr/lib/python3.4/site-packages/nose/importer.py", line 47,
> in importFromPath
>return self.importFromDir(dir_path, fqname)
>  File "/usr/lib/python3.4/site-packages/nose/importer.py", line 94,
> in importFromDir
>mod = load_module(part_fqname, fh, filename, desc)
>  File "/usr/lib64/python3.4/imp.py", line 235, in load_module
>return load_source(name, filename, file)
>  File "/usr/lib64/python3.4/imp.py", line 171, in load_source
>module = methods.load()
>  File "", line 1220, in load
>  File "", line 1200, in _load_unlocked
>  File "", line 1129, in _exec
>  File "", line 1471, in exec_module
>  File "", line 321, in
> _call_with_frames_removed
>  File "/home/alitke/src/vdsm/tests/storage_hsm_test.py", line 26, in
> 
>from storagetestlib import fake_file_env
>  File "/home/alitke/src/vdsm/tests/storagetestlib.py", line 24, in
> 
>from storagefakelib import FakeLVM
>  File "/home/alitke/src/vdsm/tests/storagefakelib.py", line 32, in
> 
>from storage import lvm as real_lvm
>  File "/home/alitke/src/vdsm/vdsm/storage/lvm.py", line 41, in
> 
>from vdsm.storage import devicemapper
>  File "/home/alitke/src/vdsm/lib/vdsm/storage/devicemapper.py", line
> 30, in 
>from vdsm.storage import misc
>  File "/home/alitke/src/vdsm/lib/vdsm/storage/misc.py", line 36, in
> 
>import Queue
> ImportError: No module named 'Queue'

The code you run is not compatible with python 3, and you did not
add the test module to the python 3 blacklist in tests/Makefile.am.

Nir
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] [VDSM] Test fails only under make check

2016-07-01 Thread Adam Litke 

I have written a new test [1] and when running 'make check' I get a
nasty ImportError (see below).  When running the same test using
run_tests_local.sh directly it works fine.  Any ideas what might be
going on?

[1] https://gerrit.ovirt.org/#/c/60060/1/tests/storage_hsm_test.py

==
ERROR: Failure: ImportError (No module named 'Queue')
--
Traceback (most recent call last):
 File "/usr/lib/python3.4/site-packages/nose/failure.py", line 39, in
runTest
   raise self.exc_val.with_traceback(self.tb)
 File "/usr/lib/python3.4/site-packages/nose/loader.py", line 418, in
loadTestsFromName
   addr.filename, addr.module)
 File "/usr/lib/python3.4/site-packages/nose/importer.py", line 47,
in importFromPath
   return self.importFromDir(dir_path, fqname)
 File "/usr/lib/python3.4/site-packages/nose/importer.py", line 94,
in importFromDir
   mod = load_module(part_fqname, fh, filename, desc)
 File "/usr/lib64/python3.4/imp.py", line 235, in load_module
   return load_source(name, filename, file)
 File "/usr/lib64/python3.4/imp.py", line 171, in load_source
   module = methods.load()
 File "", line 1220, in load
 File "", line 1200, in _load_unlocked
 File "", line 1129, in _exec
 File "", line 1471, in exec_module
 File "", line 321, in
_call_with_frames_removed
 File "/home/alitke/src/vdsm/tests/storage_hsm_test.py", line 26, in

   from storagetestlib import fake_file_env
 File "/home/alitke/src/vdsm/tests/storagetestlib.py", line 24, in

   from storagefakelib import FakeLVM
 File "/home/alitke/src/vdsm/tests/storagefakelib.py", line 32, in

   from storage import lvm as real_lvm
 File "/home/alitke/src/vdsm/vdsm/storage/lvm.py", line 41, in

   from vdsm.storage import devicemapper
 File "/home/alitke/src/vdsm/lib/vdsm/storage/devicemapper.py", line
30, in 
   from vdsm.storage import misc
 File "/home/alitke/src/vdsm/lib/vdsm/storage/misc.py", line 36, in

   import Queue
ImportError: No module named 'Queue'

--

--
Adam Litke
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] execCmd() and storing stdout and stderr in log file

2016-07-01 Thread Francesco Romani 
- Original Message -
> From: "Tomáš Golembiovský" 
> To: devel@ovirt.org
> Sent: Friday, July 1, 2016 4:43:45 PM
> Subject: [ovirt-devel] execCmd() and storing stdout and stderr in log file

[..]
> That's why I think having it present in execCmd() ready for everyone's
> use is the best solution. Unfortunately it seems that the code is a)
> essential on many places in vdsm and b) not properly covered by tests.
> Which makes it hard to touch. Also apparently some refactoring is either
> planned or already underway.
^^

This information comes from me, since IIRC we talked some time ago about
replace/refactor AsyncProc, like in https://gerrit.ovirt.org/#/c/49441/

Bests,

-- 
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] execCmd() and storing stdout and stderr in log file

2016-07-01 Thread Tomáš Golembiovský 
Hi,

I had a need recently to run a command with execCmd() and store it's
output and error to a log file, while still receiving it in the calling
code. Redirecting the error to output stream to have all in one stream
is also useful feature.

All this can be done in the calling code:

a)  On the shell level, by modyfing the command. This can be
intentionally dangerous because things like quoting of arguments has to
be considered and also could cause problems when wrappers (sudo, nice,
...) are used.

b)  By handling the writing to files in a code. This would add
unnecessary code duplication in a long run. (I don't think I'm the only
one who can see a potential in this.) Also for asynchronous process
runs, when storing both stderr & stdout in one file, it requires polling
and some stream magic. It would be better to have this done right and
only once so it can be properly tested.

That's why I think having it present in execCmd() ready for everyone's
use is the best solution. Unfortunately it seems that the code is a)
essential on many places in vdsm and b) not properly covered by tests.
Which makes it hard to touch. Also apparently some refactoring is either
planned or already underway.

What is the situation about refactoring that code area? Anyone working
on it? Do we have an estimation of time-frame for it?

Any suggestions/ideas?


Tomas

-- 
Tomáš Golembiovský 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Ovirt Node Next 4.0.0 host networking setup attempts to bridge an Infiniband interface

2016-07-01 Thread Giorgio Biacchi 
Hello list,
currently I have an up and running Ovirt 3.6 installation where every hypervisor
access the data domain via NFS over infiniband. All is working as expected, with
a couple of hooks I can set connected mode on ib0 and an MTU of 65520.

Now I'm testing Ovirt Node Next, but the behavior is different. When I try to
set up host networking the procedure attempts to bridge the network "ibstorage",
defined in Networks tab, with ib0 interface but a non ethernet interface cannot
be bridged that way.

There's any way I can obtain the desired result? Or somehow tell to the
hypervisor to not bridge that connection and to use ib0 directly?

Thanks
-- 
gb

PGP Key: http://pgp.mit.edu/
Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master won't compile

2016-07-01 Thread Alexander Wels 
On Friday, July 01, 2016 08:30:08 AM Tomas Jelinek wrote:
> - Original Message -
> 
> > From: "Alexander Wels" 
> > To: "Eyal Edri" 
> > Cc: devel@ovirt.org
> > Sent: Friday, July 1, 2016 1:51:48 PM
> > Subject: Re: [ovirt-devel] master won't compile
> > 
> > On Friday, July 01, 2016 12:14:08 PM Eyal Edri wrote:
> > > probably this patch is the culprit:
> > > https://gerrit.ovirt.org/#/c/60016/
> > > 
> > > On Fri, Jul 1, 2016 at 12:12 PM, Piotr Kliczewski <
> > 
> > I created [2] to fix the issue, I will backport it to the 4.0 branch.
> > 
> > [2] https://gerrit.ovirt.org/#/c/60061/
> 
> I have already backported it
> 

Noticed that after I had send the email, I blame lack of caffeine for that.

> > > piotr.kliczew...@gmail.com> wrote:
> > > > It looks that we have the same issue for 4.0 [1] as well.
> > > > 
> > > > 
> > > > 
> > > > [1]
> > > > http://jenkins.ovirt.org/job/ovirt-engine_4.0_check-merged-fc23-x86_64
> > > > /242
> > > > /console>
> > > > 
> > > > On Thu, Jun 30, 2016 at 9:57 PM, Alexander Wels  
wrote:
> > > > > On Thursday, June 30, 2016 03:56:25 PM Alexander Wels wrote:
> > > > >> On Thursday, June 30, 2016 03:54:16 PM Alexander Wels wrote:
> > > > >> > Hi,
> > > > >> > 
> > > > >> > I just rebased master and now it won't compile user
> > > > >> > portal with the following exception, looks like the for
> > > > >> > ACTION_TYPE_FAILED_SETUP_NETWORKS_OR_REFRESH
> > > > >> > _IN_PROGESS is missing in user portal:
> > > > >> > 
> > > > >> > [INFO]Generating method body for
> > > > >> > coloredText()
> > > > >> 
> > > > >> [INFO]Generating method body for coloredText()
> > > > > 
> > > > > Okay looks like my paste of the stack trace is getting snipped,
> > > > > basically
> > > > > looks like user portal is missing the message for that action key.
> > > > > ___
> > > > > Devel mailing list
> > > > > Devel@ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > 
> > > > ___
> > > > Devel mailing list
> > > > Devel@ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
> > 
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master won't compile

2016-07-01 Thread Tomas Jelinek 


- Original Message -
> From: "Alexander Wels" 
> To: "Eyal Edri" 
> Cc: devel@ovirt.org
> Sent: Friday, July 1, 2016 1:51:48 PM
> Subject: Re: [ovirt-devel] master won't compile
> 
> On Friday, July 01, 2016 12:14:08 PM Eyal Edri wrote:
> > probably this patch is the culprit:
> > https://gerrit.ovirt.org/#/c/60016/
> > 
> > On Fri, Jul 1, 2016 at 12:12 PM, Piotr Kliczewski <
> > 
> 
> I created [2] to fix the issue, I will backport it to the 4.0 branch.
> 
> [2] https://gerrit.ovirt.org/#/c/60061/

I have already backported it

> 
> > piotr.kliczew...@gmail.com> wrote:
> > > It looks that we have the same issue for 4.0 [1] as well.
> > > 
> > > 
> > > 
> > > [1]
> > > http://jenkins.ovirt.org/job/ovirt-engine_4.0_check-merged-fc23-x86_64/242
> > > /console>
> > > On Thu, Jun 30, 2016 at 9:57 PM, Alexander Wels  wrote:
> > > > On Thursday, June 30, 2016 03:56:25 PM Alexander Wels wrote:
> > > >> On Thursday, June 30, 2016 03:54:16 PM Alexander Wels wrote:
> > > >> > Hi,
> > > >> > 
> > > >> > I just rebased master and now it won't compile user
> > > >> > portal with the following exception, looks like the for
> > > >> > ACTION_TYPE_FAILED_SETUP_NETWORKS_OR_REFRESH
> > > >> > _IN_PROGESS is missing in user portal:
> > > >> > 
> > > >> > [INFO]Generating method body for
> > > >> > coloredText()
> > > >> 
> > > >> [INFO]Generating method body for coloredText()
> > > > 
> > > > Okay looks like my paste of the stack trace is getting snipped,
> > > > basically
> > > > looks like user portal is missing the message for that action key.
> > > > ___
> > > > Devel mailing list
> > > > Devel@ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > 
> > > ___
> > > Devel mailing list
> > > Devel@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master won't compile

2016-07-01 Thread Tomas Jelinek 
ok, on master fixed by Alex:
https://gerrit.ovirt.org/#/c/60061
and I have cherry-picked this fix now to 4.0:
https://gerrit.ovirt.org/#/c/60082/1

please note this issues shows up only if compiling with language permutations 
enabled.

- Original Message -
> From: "Eyal Edri" 
> To: "Piotr Kliczewski" , "Alona Kaplan" 
> 
> Cc: devel@ovirt.org
> Sent: Friday, July 1, 2016 11:14:08 AM
> Subject: Re: [ovirt-devel] master won't compile
> 
> probably this patch is the culprit:
> https://gerrit.ovirt.org/#/c/60016/
> 
> On Fri, Jul 1, 2016 at 12:12 PM, Piotr Kliczewski <
> piotr.kliczew...@gmail.com > wrote:
> 
> 
> It looks that we have the same issue for 4.0 [1] as well.
> 
> 
> 
> [1]
> http://jenkins.ovirt.org/job/ovirt-engine_4.0_check-merged-fc23-x86_64/242/console
> 
> On Thu, Jun 30, 2016 at 9:57 PM, Alexander Wels < aw...@redhat.com > wrote:
> > On Thursday, June 30, 2016 03:56:25 PM Alexander Wels wrote:
> >> On Thursday, June 30, 2016 03:54:16 PM Alexander Wels wrote:
> >> > Hi,
> >> > 
> >> > I just rebased master and now it won't compile user
> >> > portal with the following exception, looks like the for
> >> > ACTION_TYPE_FAILED_SETUP_NETWORKS_OR_REFRESH
> >> > _IN_PROGESS is missing in user portal:
> >> > 
> >> > [INFO] Generating method body for
> >> > coloredText()
> >> 
> >> [INFO] Generating method body for coloredText()
> > 
> > Okay looks like my paste of the stack trace is getting snipped, basically
> > looks like user portal is missing the message for that action key.
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> 
> 
> 
> --
> Eyal Edri
> Associate Manager
> RHEV DevOps
> EMEA ENG Virtualization R
> Red Hat Israel
> 
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master won't compile

2016-07-01 Thread Piotr Kliczewski 
It looks that we have the same issue for 4.0 [1] as well.



[1] 
http://jenkins.ovirt.org/job/ovirt-engine_4.0_check-merged-fc23-x86_64/242/console

On Thu, Jun 30, 2016 at 9:57 PM, Alexander Wels  wrote:
> On Thursday, June 30, 2016 03:56:25 PM Alexander Wels wrote:
>> On Thursday, June 30, 2016 03:54:16 PM Alexander Wels wrote:
>> > Hi,
>> >
>> > I just rebased master and now it won't compile user
>> > portal with the following exception, looks like the for
>> > ACTION_TYPE_FAILED_SETUP_NETWORKS_OR_REFRESH
>> > _IN_PROGESS is missing in user portal:
>> >
>> > [INFO]Generating method body for
>> > coloredText()
>>
>> [INFO]Generating method body for coloredText()
>
> Okay looks like my paste of the stack trace is getting snipped, basically
> looks like user portal is missing the message for that action key.
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel