Re: [ovirt-devel] Problem when loading images via web interface

2018-01-23 Thread Yaniv Kaul 
On Tue, Jan 23, 2018 at 10:39 PM, Dmitry Semenov  wrote:

> While loading disk image (via the web interface) in cluster01 on
> storage01, storage02 - everything is going well.
> While loading disk image (via the web interface) in cluster02 on
> storage03, storage04 - the problem occurs, the image isn't loaded, the
> process stops at the stage: paused by System (at the same time loading
> straightly through API goes without problems).
>
> screenshot: https://yadi.sk/i/9WtkDlT23Riqxp
>
> Logs are applied (engine.log): https://pastebin.com/54k5j7hC


Can you also share vdsm logs, at least from 01c04x09.unix.local ? It seems
to have failed there.
Y.


>
>
> image size: ~1.3 GB
>
> my scheme:
>
> data_center_01
>   cluster01
> host01  \
> host02  - storage01, storage02
> host03  /
>
>   cluster02
> host04  \
> host05  - storage03, storage04
> host06  /
>
> HostedEngine in cluster01
> oVirt: Version 4.2.0.2-1.el7.centos
>
> --
> Best regards,
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ACTION-REQUIRED] Making accurate CI for oVirt 4.2

2018-01-23 Thread Dan Kenigsberg 
On Wed, Jan 24, 2018 at 8:35 AM, Barak Korren  wrote:
> On 23 January 2018 at 18:44, Martin Sivak  wrote:
>> Hi Barak,
>>
>> can you please please add links to the proper repositories and/or
>> directories when you send something like this? I really helps us when
>> we do not have to search through all the jenkins and other infra
>> repositories for which is the correct one. Because I really do not
>> remember all the places that need to change out of my head.
>
> See below.
>
>> So what you are asking for here is basically that we edit the files
>> here [1] and create a 4.2_build-artifacts job using copy and paste,
>> right? Or is there some other place that needs to change as well?
>
> Yep. technically this should amount to a single change to a single
> file (See below). The important part is making the right decision for
> each project, understanding its consequences, and realizing the
> actions that would be needed for changing that decision in the future.
>
>> [1] 
>> https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;f=jobs/confs/projects;h=5a59dfea545da98e252eb6c8d95a92d08708a22d;hb=cd75bb9eb3353652384ed89777fc15d71d1f9e36
>
> There is only one file** you need to maintain that is (currently) not
> in your own project's repo***.
> Each project has such a file at [1].
>
> Documentation for the contents of that file can be found here: [2].
>
> There is no need to copy-paste much - the existing file should contain
> a mapping of project branches to oVirt versions. Typically what would
> be needed is just to add a single entry to the map. For example, for
> engine it would be:
>
> version:
> - master:
> branch: master
> - 4.2:
> branch: master
>...

If project maintainers opt for this "Route 2", it is their personal
responsibility to change the above "master" to "ovirt-4.2" branch
*BEFORE* they create their stable branch ovirt-4.2. If they fail to do
so, CI would get "dirty" with 4.3 packages.  Barak hinted to this a
bit too mildly.

>
> ** Bigger projects can spread configuration across multiple files, but
> this is rarely needed.
> *** This applies only to Gerrit projects. GitHub projects have
> everything configured in their own repo. See [3].
>  Specifically for engine, the map appears twice in the file, this
> should probably be re-factored.
>
> [1]: 
> https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;f=jobs/confs/projects;hb=refs/heads/master
> [2]: 
> http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_Gerrit/index.html
> [3]: 
> http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_GitHub/index.html
>
>
>
> --
> Barak Korren
> RHV DevOps team , RHCE, RHCi
> Red Hat EMEA
> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ACTION-REQUIRED] Making accurate CI for oVirt 4.2

2018-01-23 Thread Barak Korren 
On 23 January 2018 at 18:44, Martin Sivak  wrote:
> Hi Barak,
>
> can you please please add links to the proper repositories and/or
> directories when you send something like this? I really helps us when
> we do not have to search through all the jenkins and other infra
> repositories for which is the correct one. Because I really do not
> remember all the places that need to change out of my head.

See below.

> So what you are asking for here is basically that we edit the files
> here [1] and create a 4.2_build-artifacts job using copy and paste,
> right? Or is there some other place that needs to change as well?

Yep. technically this should amount to a single change to a single
file (See below). The important part is making the right decision for
each project, understanding its consequences, and realizing the
actions that would be needed for changing that decision in the future.

> [1] 
> https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;f=jobs/confs/projects;h=5a59dfea545da98e252eb6c8d95a92d08708a22d;hb=cd75bb9eb3353652384ed89777fc15d71d1f9e36

There is only one file** you need to maintain that is (currently) not
in your own project's repo***.
Each project has such a file at [1].

Documentation for the contents of that file can be found here: [2].

There is no need to copy-paste much - the existing file should contain
a mapping of project branches to oVirt versions. Typically what would
be needed is just to add a single entry to the map. For example, for
engine it would be:

version:
- master:
branch: master
- 4.2:
branch: master
   ...

** Bigger projects can spread configuration across multiple files, but
this is rarely needed.
*** This applies only to Gerrit projects. GitHub projects have
everything configured in their own repo. See [3].
 Specifically for engine, the map appears twice in the file, this
should probably be re-factored.

[1]: 
https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;f=jobs/confs/projects;hb=refs/heads/master
[2]: 
http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_Gerrit/index.html
[3]: 
http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_GitHub/index.html



-- 
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Problem when loading images via web interface

2018-01-23 Thread Dmitry Semenov 
While loading disk image (via the web interface) in cluster01 on storage01, 
storage02 - everything is going well.
While loading disk image (via the web interface) in cluster02 on storage03, 
storage04 - the problem occurs, the image isn't loaded, the process stops at 
the stage: paused by System (at the same time loading straightly through API 
goes without problems).

screenshot: https://yadi.sk/i/9WtkDlT23Riqxp

Logs are applied (engine.log): https://pastebin.com/54k5j7hC

image size: ~1.3 GB

my scheme:

data_center_01
  cluster01
    host01  \
    host02  - storage01, storage02
    host03  /

  cluster02
    host04  \
    host05  - storage03, storage04
    host06  /

HostedEngine in cluster01
oVirt: Version 4.2.0.2-1.el7.centos

-- 
Best regards,
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ACTION-REQUIRED] Making accurate CI for oVirt 4.2

2018-01-23 Thread Martin Sivak 
Hi Barak,

can you please please add links to the proper repositories and/or
directories when you send something like this? I really helps us when
we do not have to search through all the jenkins and other infra
repositories for which is the correct one. Because I really do not
remember all the places that need to change out of my head.

So what you are asking for here is basically that we edit the files
here [1] and create a 4.2_build-artifacts job using copy and paste,
right? Or is there some other place that needs to change as well?

[1] 
https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;f=jobs/confs/projects;h=5a59dfea545da98e252eb6c8d95a92d08708a22d;hb=cd75bb9eb3353652384ed89777fc15d71d1f9e36


Best regards

Martin Sivak

On Tue, Jan 23, 2018 at 1:52 PM, Barak Korren  wrote:
> Hi,
>
> This message is aimed for project maintainers. Other developers may
> also find it interesting to have a glimpse at the oVirt-wide test and
> composition processes.
>
> TL;DR: To get accurate CI for oVirt 4.2, most projects
>need to add 4.2 jobs in YAML.
>
> Before I can explain what the current issue is and which action is
> required, I'm need to provide a brief overview into how oVirt CI
> works.
>
> oVirt CI has two major components to it:
> 1. The STDCI component which is used to build and test individual
> projects. Most developers interact with this on a daily basis as it is
> responding on GitHub and Gerrit events.
> 2. The "change-queue" (CQ) component which is used to automatically
> compose the whole of oVirt from its sub projects  and run system tests
> (OST) on it. This component is used to gather the information that is
> used by the infra team to compose the "OST failure report" you can
> occasionally see being sent to this list. The change queue is also
> used to automatically maintain the 'tested' and '*-snapshot' (AKA
> nightly) repositories.
>
> The way the CQ composes oVirt is by looking at the post-merge STDCI
> 'build-artifacts' jobs, and collecting together artifacts built by
> jobs that target a specific oVirt version into that version's change
> queue. Essentially the '*_master_build-artifacts-*' jobs go into the
> 'ovirt-master' change queue, the '*_4.1_build-artifacts-*' jobs go
> into the 'ovirt-4.1' change queue etc.
>
> Over the course of the oVirt 4.2 development, most project used their
> 'master' branch, which is typically mapped to '*_master_*' jobs, for
> developing 4.2 code. So the 'ovirt-master' CQ provided good indication
> of the state of 4.2 code.
>
> As projects started addeing 4.2 branches, we have created an
> 'ovirt-4.2' CQ to gather them. We did so under the assumption that
> most projects will branch soon after. The assumption turned up to be
> wrong as most projects did not yet fork and may not do so in the near
> future.
>
> As some projects did fork, the end result is that currently:
>
>  ___there is no accurate representation of oVirt 4.2 in CI___
>
> 'ovirt-master' CQ no longer represents oVirt 4.2 as some projects
> already have some 4.3 code in their 'master' branches.
>
> 'ovirt-4.2' CQ does not represent oVirt 4.2 as most projects do not
> push artifacts into it.
>
> To get any benefit from CI, we need to have it represent what we are
> going to release. This means that at this point we need all projects
> to have '*_4.2_build-artifacts-*' jobs that map to the code that is
> intended to be included in oVirt 4.2. Projects can either:
>
> 1. Create 4.2 branches and map the new jobs to them.
> 2. Keep 4.2 development in 'master' and create '4.2' jobs that map to it.
>
> Taking route #2 means a commitment to not adding any 4.3 code to the
> 'master' branch. Please keep it, as rolling back "too new" builds from
> the various repos and caches we have is very difficult.
>
> --
> Barak Korren
> RHV DevOps team , RHCE, RHCi
> Red Hat EMEA
> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] unboundid-ldapsdk-4.0.4-1.fc27 is now available for testing

2018-01-23 Thread Sandro Bonazzola 
On Fedora 27 we had previously 4.0.1. The new releases 4.0.4 will be soon
available in updates-testing repository and includes several bug fixes and
a few enhancements.  You can see release notes here:
https://github.com/pingidentity/ldapsdk/releases

In oVirt this package is required by ovirt-engine-extension-aaa-ldap.

If you're going to test the package, please provide karma:
https://bodhi.fedoraproject.org/updates/unboundid-ldapsdk-4.0.4-1.fc27

Thanks,

-- 

SANDRO BONAZZOLA

ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D

Red Hat EMEA 

TRIED. TESTED. TRUSTED. 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] [ACTION-REQUIRED] Making accurate CI for oVirt 4.2

2018-01-23 Thread Barak Korren 
Hi,

This message is aimed for project maintainers. Other developers may
also find it interesting to have a glimpse at the oVirt-wide test and
composition processes.

TL;DR: To get accurate CI for oVirt 4.2, most projects
   need to add 4.2 jobs in YAML.

Before I can explain what the current issue is and which action is
required, I'm need to provide a brief overview into how oVirt CI
works.

oVirt CI has two major components to it:
1. The STDCI component which is used to build and test individual
projects. Most developers interact with this on a daily basis as it is
responding on GitHub and Gerrit events.
2. The "change-queue" (CQ) component which is used to automatically
compose the whole of oVirt from its sub projects  and run system tests
(OST) on it. This component is used to gather the information that is
used by the infra team to compose the "OST failure report" you can
occasionally see being sent to this list. The change queue is also
used to automatically maintain the 'tested' and '*-snapshot' (AKA
nightly) repositories.

The way the CQ composes oVirt is by looking at the post-merge STDCI
'build-artifacts' jobs, and collecting together artifacts built by
jobs that target a specific oVirt version into that version's change
queue. Essentially the '*_master_build-artifacts-*' jobs go into the
'ovirt-master' change queue, the '*_4.1_build-artifacts-*' jobs go
into the 'ovirt-4.1' change queue etc.

Over the course of the oVirt 4.2 development, most project used their
'master' branch, which is typically mapped to '*_master_*' jobs, for
developing 4.2 code. So the 'ovirt-master' CQ provided good indication
of the state of 4.2 code.

As projects started addeing 4.2 branches, we have created an
'ovirt-4.2' CQ to gather them. We did so under the assumption that
most projects will branch soon after. The assumption turned up to be
wrong as most projects did not yet fork and may not do so in the near
future.

As some projects did fork, the end result is that currently:

 ___there is no accurate representation of oVirt 4.2 in CI___

'ovirt-master' CQ no longer represents oVirt 4.2 as some projects
already have some 4.3 code in their 'master' branches.

'ovirt-4.2' CQ does not represent oVirt 4.2 as most projects do not
push artifacts into it.

To get any benefit from CI, we need to have it represent what we are
going to release. This means that at this point we need all projects
to have '*_4.2_build-artifacts-*' jobs that map to the code that is
intended to be included in oVirt 4.2. Projects can either:

1. Create 4.2 branches and map the new jobs to them.
2. Keep 4.2 development in 'master' and create '4.2' jobs that map to it.

Taking route #2 means a commitment to not adding any 4.3 code to the
'master' branch. Please keep it, as rolling back "too new" builds from
the various repos and caches we have is very difficult.

-- 
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] [ OST Failure Report ] [ oVirt 4.1 ] [ 23-01-2018 ] [ 002_bootstrap.add_cluster ]

2018-01-23 Thread Dafna Ron 
Hi,

We are failing test 002_bootstrap.add_cluster on 4.1 suite.
this is an issue we encountered on 4.2 and we think is related to libvirt.



*Link and headline of suspected patches: Patch is not related to the
underline issue: *

https://gerrit.ovirt.org/#/c/85609/7 - spec: update dependencies for
CVE-2017-5754,CVE-2017-5753,CVE-2017-5715






*Link to
Job:http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1527
Link to
all
logs:http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1527/artifact/
(Relevant)
error snippet from the log: *

 File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
129, in wrapped_test
test()
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
  File 
"/home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/basic-suite-4.1/test-scenarios/002_bootstrap.py",
line 211, in add_cluster
add_cluster_4(prefix)
  File 
"/home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/basic-suite-4.1/test-scenarios/002_bootstrap.py",
line 237, in add_cluster_4
cpu_family = prefix.virt_env.get_ovirt_cpu_family()
  File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 151,
in get_ovirt_cpu_family
','.join(cpu_map[host.cpu_vendor].iterkeys())
RuntimeError: Unsupported CPU model: Haswell-noTSX-IBRS. Supported
models: 
IvyBridge,Westmere,Skylake,Penryn,Haswell,Broadwell,Nehalem,Skylake-Client,Broadwell-noTSX,Conroe,SandyBridge,Haswell-noTSX

**
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ 23-06-2018 ] [ 001_initialize_engine.initialize_engine ]

2018-01-23 Thread Dafna Ron 
Hi,

We are failing test "001_initialize_engine.initialize_engine" in basic
suite.
Can you please check this issue?


*Link and headline of suspected patches:
https://gerrit.ovirt.org/#/c/85809/  -
*























































*packaging: raise if network provider name is duplicatedLink to
Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5047/
Link
to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5047/artifact/
(Relevant)
error snippet from the log: [ INFO  ] Stage: Setup validation[ ERROR
] Failed to execute stage 'Setup validation': could not connect to server:
Connection refusedIs the server running on host "localhost"
(::1) and acceptingTCP/IP connections on port 5432?
could not connect to server: Connection refusedIs the
server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432? [ INFO  ] Stage: Clean up
Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20180123025743-gngssk.log[
INFO  ] Generating answer file
'/var/lib/ovirt-engine/setup/answers/20180123025751-setup.conf'[ INFO  ]
Stage: Pre-termination[ INFO  ] Stage: Termination[ ERROR ] Execution of
setup failed('FATAL Internal error (main): could not connect to server:
Connection refused\n\tIs the server running on host "localhost" (::1) and
accepting\n\tTCP/IP connections on port 5432?\ncould not connect to server:
Connection refused\n\tIs the server running on host "localhost" (127.0.0.1)
and accepting\n\tTCP/IP connections on port 5432?\n',)2018-01-23
07:57:51,174::ssh.py::ssh::96::lago.ssh::DEBUG::Command 178de848 on
lago-basic-suite-master-engine  errors: Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/otopi/__main__.py", line 88, in
maininstaller.execute()  File
"/usr/lib/python2.7/site-packages/otopi/main.py", line 157, in execute
self.context.runSequence()  File
"/usr/lib/python2.7/site-packages/otopi/context.py", line 765, in
runSequenceutil.raiseExceptionInformation(infos[0])  File
"/usr/lib/python2.7/site-packages/otopi/util.py", line 81, in
raiseExceptionInformationexec('raise info[1], None, info[2]')  File
"/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in
_executeMethodmethod['method']()  File
"/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py",
line 894, in _validate_provider_uniquenessexisting_provider_id =
self._check_provider_exists()  File
"/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py",
line 138, in _check_provider_existsownConnection=True,  File
"/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py",
line 238, in executedatabase=database,  File
"/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py",
line 172, in connectsslmode=sslmode,  File
"/usr/lib64/python2.7/site-packages/psycopg2/__init__.py", line 164, in
connectconn = _connect(dsn, connection_factory=connection_factory,
async=async)OperationalError: could not connect to server: Connection
refusedIs the server running on host "localhost" (::1) and
acceptingTCP/IP connections on port 5432?could not connect to
server: Connection refusedIs the server running on host "localhost"
(127.0.0.1) and acceptingTCP/IP connections on port 5432?*
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel