[ovirt-devel] Re: Write To a Storage Domain

[Arik Hadas]

On Mon, Mar 25, 2024 at 9:21 AM Shafi Mohammed 
wrote:

> Hi Guys ,
>
> I am writing a tool to migrate VMs from KVM to Ovirt .
>
> I created a storage domain and tried to expose it to NBD server to write
> the data from the source Vm disk . But im facing a nbd permission issue  .
> Even a copy or write is restricted .
>
> Actual Command
> qemu-nbd -f qcow2 /rhev/data-center/mnt/192.168.108.27:
> _storage/d62b04f8-973f-4168-9e69-1f334a4968b6/images/cd6b9fc4-ef8a-4f40-ac8d-f18d355223d0/d2a57e6f-029e-46cc-85f8-ce151d027dcb
>
>
>
> *qemu-nbd: Failed to blk_new_open
> '/rhev/data-center/mnt/192.168.108.27:_storage/d62b04f8-973f-4168-9e69-1f334a4968b6/images/cd6b9fc4-ef8a-4f40-ac8d-f18d355223d0/d2a57e6f-029e-46cc-85f8-ce151d027dcb':
> Could not open
> '/rhev/data-center/mnt/192.168.108.27:_storage/d62b04f8-973f-4168-9e69-1f334a4968b6/images/cd6b9fc4-ef8a-4f40-ac8d-f18d355223d0/d2a57e6f-029e-46cc-85f8-ce151d027dcb':
> Permission denied *
> Please suggest me if Ovirt has any API  to open a disk from a
> storage domain and write data to it
>

just out of curiosity, did you look at the built-in functionality of
importing VMs from KVM via Libvirt?
https://www.ovirt.org/develop/release-management/features/virt/KvmToOvirt.html


>
>
>
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/76KMPY2W7VPQCZI7U3Q4K6OVBO3NVUNS/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/AWS2JAY5C5ZPPQSI3ZNMV5PJ5YQC5I6Z/

[ovirt-devel] Re: Update project settings for required CI tests in vdsm and ovirt-imageio

[Arik Hadas]

On Mon, Jul 18, 2022 at 10:24 AM Albert Esteve  wrote:

> Hi all,
>
> We have recently updated the CI tests in the vdsm repository, and split
> storage tests into 'test-storage-root" and "test-storage-user".
>
> However, the latter is still named "test-storage" (
> https://github.com/oVirt/vdsm/blob/master/.github/workflows/ci.yml#L13)
> as it was a required CI job in the project settings. Ideally, we would need
> a project owner (or someone with write permissions) to remove
> "test-storage" from the project requirements and add "test-storage-user"
> instead, for consistency.
>
> Similarly, in ovirt-imageio, centos-8 and fedora-35 shall be removed in
> favor of their current counterparts:
> https://github.com/oVirt/ovirt-imageio/pull/99
> Otherwise, the branch above cannot be integrated.
>
> I do not have the permissions, but in my fork I found the place where to
> change it (i.e., the branches settings):
> [image: image.png]
>
> Could someone with permissions take a look? Thanks in advance!
>

Update: we've made the changes above
Thanks Albert!


> BR,
>
> Albert Esteve
>
> Senior Software Engineer
>
> Red Hat 
>
> aest...@redhat.com
> 
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/73ATIUFEIFO3XU2TP3YQ75VUXJO6IRIS/

[ovirt-devel] Re: oVirt 4.5 Beta Test Day | Installing on Ceph

[Arik Hadas]

@Shani Leviim  can you please take a look?

On Thu, Apr 7, 2022 at 4:27 PM Sandro Bonazzola  wrote:

> @Arik Hadas  @Benny Zlotnik  can
> you please have a look?
>
> Il giorno gio 7 apr 2022 alle ore 15:20 Jöran Malek 
> ha scritto:
>
>> Result for today sofar:
>>
>> * Installation on iSCSI works fine (didn't test iSCSI multipath)
>> * Copy from iSCSI to Managed Block Storage works
>> * Creating a VM from that Managed Block Storage disk works
>> ** Starting a VM with managed block storage volume works
>> ** Creating a template from a vm with storage on Managed Block Storage
>> doesn't work, error:
>>
>> 2022-04-07 15:02:44,984+02 ERROR
>>> [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService]
>>> (default task-18) [] Permutation name: BD850CA753ED62B70AE455D832D2466E
>>> 2022-04-07 15:02:44,984+02 ERROR
>>> [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService]
>>> (default task-18) [] Uncaught exception:
>>> com.google.gwt.core.client.JavaScriptException: (TypeError) : Cannot read
>>> properties of undefined (reading 'a')
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.vm.HwOnlyCoreUnitToVmBaseBuilder.$postBuild(HwOnlyCoreUnitToVmBaseBuilder.java:24)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.vm.CoreUnitToVmBaseBuilder.postBuild(CoreUnitToVmBaseBuilder.java:33)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.vm.HwOnlyCoreUnitToVmBaseBuilder.postBuild(HwOnlyCoreUnitToVmBaseBuilder.java:24)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.CompositeBuilder$LastBuilder.build(CompositeBuilder.java:45)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.BaseSyncBuilder.build(BaseSyncBuilder.java:13)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.BaseSyncBuilder.build(BaseSyncBuilder.java:13)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.CompositeBuilder.build(CompositeBuilder.java:21)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.CompositeBuilder.build(CompositeBuilder.java:21)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.BuilderExecutor.$build(BuilderExecutor.java:61)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.builders.BuilderExecutor.build(BuilderExecutor.java:28)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.models.vms.VmListModel.$postNameUniqueCheck(VmListModel.java:1337)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.models.vms.VmListModel.$lambda$14(VmListModel.java:1315)
>>> at
>>> org.ovirt.engine.ui.uicommonweb.models.vms.VmListModel$lambda$14$Type.onSuccess(VmListModel.java:1315)
>>> at
>>> org.ovirt.engine.ui.frontend.Frontend$1.$onSuccess(Frontend.java:239)
>>> at
>>> org.ovirt.engine.ui.frontend.Frontend$1.onSuccess(Frontend.java:239)
>>> at
>>> org.ovirt.engine.ui.frontend.communication.OperationProcessor$1.$onSuccess(OperationProcessor.java:133)
>>> at
>>> org.ovirt.engine.ui.frontend.communication.OperationProcessor$1.onSuccess(OperationProcessor.java:133)
>>> at
>>> org.ovirt.engine.ui.frontend.communication.GWTRPCCommunicationProvider$3$1.$onSuccess(GWTRPCCommunicationProvider.java:161)
>>> at
>>> org.ovirt.engine.ui.frontend.communication.GWTRPCCommunicationProvider$3$1.onSuccess(GWTRPCCommunicationProvider.java:161)
>>> at
>>> com.google.gwt.user.client.rpc.impl.RequestCallbackAdapter.onResponseReceived(RequestCallbackAdapter.java:198)
>>> at
>>> com.google.gwt.http.client.Request.$fireOnResponseReceived(Request.java:233)
>>> at
>>> com.google.gwt.http.client.RequestBuilder$1.onReadyStateChange(RequestBuilder.java:409)
>>> at Unknown.eval(webadmin-0.js)
>>> at com.google.gwt.core.client.impl.Impl.apply(Impl.java:306)
>>> at com.google.gwt.core.client.impl.Impl.entry0(Impl.java:345)
>>> at Unknown.eval(webadmin-0.js)
>>
>>
>>
>> * Importing a Template from Glance into iSCSI-Volume, then copying that
>> disk to Managed Block Storage works.
>> * Removing the original imported disk, leaving only the managed block
>> storage disk, works
>> * Creating a VM of that template doesn't work
>> (vdsm and engine attached)
>> ___
>> Devel mailing list -- devel@ovirt.org
>> To unsubscribe send an email to devel-le...@ovirt.org
>> P

[ovirt-devel] Re: Proposing Eyal Shenitsky as ovirt-engine backend maintener

[Arik Hadas]

+1

On Wed, Aug 11, 2021 at 3:36 PM Martin Perina  wrote:

>
> On Wed, Aug 11, 2021 at 2:03 PM Ritesh Chikatwar 
> wrote:
>
>> +1 from my side
>>
>> On Wed, Aug 11, 2021 at 5:15 PM Benny Zlotnik 
>> wrote:
>>
>>> Hi,
>>>
>>> Eyal has been working on ovirt-engine for a while now and is currently
>>> the most prolific active contributor from the storage team.
>>>
>>> As such, I would like to propose Eyal as an ovirt-engine maintainer
>>>
>>
> +1 from me
>
___
>>> Devel mailing list -- devel@ovirt.org
>>> To unsubscribe send an email to devel-le...@ovirt.org
>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/Y7QQSLHBUUU6W2BTFB57BHHWEJ3ELQHH/
>>>
>> ___
>> Devel mailing list -- devel@ovirt.org
>> To unsubscribe send an email to devel-le...@ovirt.org
>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/LSZZRCJBKSZL2LX5YOKJNYVHWI4NFJW6/
>>
>
>
> --
> Martin Perina
> Manager, Software Engineering
> Red Hat Czech s.r.o.
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PNPF5WWB76M2GORQ2PGOZDXWZDPOYLTX/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/2E2SQBPI3N4YWPA4W4GYFYAHU2Z2MRQF/

[ovirt-devel] Re: lago is dead, long live ost!

[Arik Hadas]

On Tue, Jul 27, 2021 at 2:57 PM Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

> Hi,
> we worked for a while on removing our dependency on lago that has been
> more or less abandoned for a long time now. We got to the minimal feature
> set that is simple to replace with bunch of virsh commands, and most of the
> advanced logic is implemented in pytest. We’ve just merged the last patches
> that freed us up from lago for local and beaker CI runs.
>
> There’s a new ost.sh wrapper for the simple operations of running the
> suite, inspecting the environment and shutting it down.
> Hopefully it’s self explanatory….
>
> ./ost.sh command [arguments]
>
> run   [...]
> initializes the workspace with preinstalled distro ost-images,
> launches VMs and runs the whole suite
> add extra repos with --custom-repo=url
> skip check that extra repo is actually used with
> --skip-custom-repos-check
> status
> show environment status, VM details
> shell  [command ...]
> opens ssh connection
> console 
> opens virsh console
> destroy
> stop and remove the running environment
>

> Right now lago and run_suite.sh still works and it is still used by the
> mock-based jenkins.ovirt.org runs. It will go away in future.
>

nice
so if run_suite.sh is deprecated, how about replacing it in the
documentation with that new thingy?


>
> Thanks,
> michal
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/WJMNDQR5QNRQIJ64KUGTS3NXZJTA2MGN/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DLUZCDUOLKAIGOXLPHLLS7YFK7SASHF2/

[ovirt-devel] Re: 2FA requirement for oVirt GitHub org?

[Arik Hadas]

On Mon, Jun 28, 2021 at 3:26 PM Sandro Bonazzola 
wrote:

> HI, just an heads up that turning on the 2FA will have some side effects:
>
> Require two-factor authentication for everyone in the oVirt organization.
> Members, billing managers, and outside collaborators who do not have
> two-factor authentication enabled for their personal account will be
> removed from the organization and will receive an email notifying them
> about the change.
>
> So before turning it on, we need to ensure we are not going to disrupt the
> org.
>

+1
I don't know if it's that way by default when enabling the 2FA but when
other organizations enabled it, users were allowed to keeping signing in
using password for several weeks and only then those that didn't set it up
were dropped from the organization


>
> Il giorno lun 28 giu 2021 alle ore 14:20 Martin Perina 
> ha scritto:
>
>>
>>
>> On Mon, Jun 28, 2021 at 11:58 AM Sandro Bonazzola 
>> wrote:
>>
>>> +1 for enabling 2FA on my side
>>>
>>
>> +1
>>
>>>
>>> Il giorno lun 28 giu 2021 alle ore 11:50 Janos Pasztor 
>>> ha scritto:
>>>
 Hey folks,

 I just noticed that we don't have 2FA enforcement on the oVirt GitHub
 org.

 [image: image.png]
 Is this just an oversight or is it intentional? May I propose to turn
 2FA enforcement on?

 Janos
 ___
 Devel mailing list -- devel@ovirt.org
 To unsubscribe send an email to devel-le...@ovirt.org
 Privacy Statement: https://www.ovirt.org/privacy-policy.html
 oVirt Code of Conduct:
 https://www.ovirt.org/community/about/community-guidelines/
 List Archives:
 https://lists.ovirt.org/archives/list/devel@ovirt.org/message/SASNDGLK3FBKB5LPDXVMLJKB7GXK73KZ/

>>>
>>>
>>> --
>>>
>>> Sandro Bonazzola
>>>
>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>>>
>>> Red Hat EMEA 
>>>
>>> sbona...@redhat.com
>>> 
>>>
>>> *Red Hat respects your work life balance. Therefore there is no need to
>>> answer this email out of your office hours.*
>>>
>>>
>>> ___
>>> Devel mailing list -- devel@ovirt.org
>>> To unsubscribe send an email to devel-le...@ovirt.org
>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CDSTENA6B4YNUDX27RJUXZAH4JLZD4BF/
>>>
>>
>>
>> --
>> Martin Perina
>> Manager, Software Engineering
>> Red Hat Czech s.r.o.
>>
>
>
> --
>
> Sandro Bonazzola
>
> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>
> Red Hat EMEA 
>
> sbona...@redhat.com
> 
>
> *Red Hat respects your work life balance. Therefore there is no need to
> answer this email out of your office hours.*
>
>
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/AI56Q2PSY3PN22FMRTUD3JJNEQUYCY6U/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/HZWWLUGAJXPRKL5O5CDNK7TU4YZZHIYD/

[ovirt-devel] Re: Migrate VMs between DCs

[Arik Hadas]

On Wed, Mar 17, 2021 at 12:01 AM Daniel Gurgel <
danieldemoraisgur...@gmail.com> wrote:

> Nir, thank you for your help and practical examples. I'll try to simulate.
>
> In fact, most companies that actually have different offices do not share
> the same SAN/Storage structure , there are physical limitations and network
> details and time that make procedures impossible - As I said, storages that
> work differently, are also a limiting factor.
>
> Export Domain, we can count today, but in six months, we don't know if
> it'll still be there.
>

Even if export domains will still be there, you'd probably want to switch
to an alternative that is better tested these days and that would comply
with recent/future changes.
Let me be more concrete here - we're adding TPM [1] + have already added
NVRAM and there's no plan to preserve them when exporting and importing
from an export domain.

Nir touched the trade-off between the alternatives that were described and
I'll add to that:
1. Detach+attach date domain is indeed nice in the sense that you don't
copy data but if you intend to use it as an alternative to export domains,
be aware that you have to have all the disks of the VM/template on the
storage domain(s) you detach and when detaching a storage domain, the
entities (VMs/templates) are "unregistered" (i.e., removed) from the
original data center.
2. Export/Import OVAs is the closest replacement for export domains - if
you want the ability to export a VM/template from one data center and
import it elsewhere while ensuring the VM/template includes all its
resources (disks, TPM, NVRAM, etc) at the target data center and without
affecting the original data center, that's the mechanism I'd go with. We
made significant fixes for that recently so I'd suggest updating to 4.4.4.

[1]
https://www.ovirt.org/develop/release-management/features/virt/tpm-device.html



>
> Most hypervisors support Live Migrate/Off line migrate between "data
> centers". It would be great to have this feature up to oVirt 4.5.x
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/WYRHNO2V6RF4FSIDJCLCR2DMVAI4LLJR/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CCQ7GGQV5D2O7XWNAJ77DH6FRRF3NU56/

[ovirt-devel] Re: Engine upgrade fails with virtio_scsi_multi_queues_enabled does not exist

[Arik Hadas]

On Tue, Mar 16, 2021 at 11:13 AM Vojtech Juranek 
wrote:

> On Tuesday, 16 March 2021 08:48:46 CET Benny Zlotnik wrote:
> > I think you had an issue because you clashed with the script number
> > (or something like that, since you have a patch with an upgrade
> > script), I think if you run manually the SQL in[1] and rerun
> > engine-setup it should sort it out
>
> Thanks Benny and Didi, indeed, in my patch I *had* upgrade script
> 04_04_1010_something, which I renamed after to 04_04_1020_something after
> I
> rebased on top of gerrit #112642. However, DB remembers it and skips
> 04_04_1010_*.
>

BTW, an old trick to prevent this from happening is to name the script
locally with index % 10 != 0 until it's merged


>
> Running 04_04_1010_add_virtio_scsi_multi_queues_enabled_to_vm_static.sq
> manually fixed the issue.
>

> Thanks!
> Vojta
>
> > [1]
> >
> https://gerrit.ovirt.org/c/ovirt-engine/+/112642/16/packaging/dbscripts/upg
> > rade/04_04_1010_add_virtio_scsi_multi_queues_enabled_to_vm_static.sql#1
>
> > On Tue, Mar 16, 2021 at 9:40 AM Vojtech Juranek 
> > wrote:
> > >
> > >
> > > On Tuesday, 16 March 2021 08:29:07 CET Ales Musil wrote:
> > >
> > > > On Tue, Mar 16, 2021 at 8:24 AM Vojtech Juranek  >
> > > > wrote:
> > >
> > > > > On Tuesday, 16 March 2021 07:08:31 CET Yedidyah Bar David wrote:
> > > > >
> > > > > > On Mon, Mar 15, 2021 at 10:48 PM Vojtech Juranek
> > > > > > 
> > > > >
> > > > >
> > > > >
> > > > > wrote:
> > > > >
> > > > > > > Hi,
> > > > > > > when I try to upgrade my engine with built artifacts (e.g.
> [1]),
> > > > > > > it
> > > > >
> > > > >
> > > > >
> > > > > fails
> > > > >
> > > > >
> > > > >
> > > > > > > with:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > psql:/usr/share/ovirt-engine/dbscripts/create_views.sql:1000:
> > > > > > > ERROR:
> > > > > > > column vm_templates.virtio_scsi_multi_queues_enabled does not
> > > > > > > exist
> > > > >
> > > > >
> > > > >
> > > > > LINE
> > > > >
> > > > >
> > > > >
> > > > > > > 86: vm_templates.virtio_scsi_multi_queues_enabled AS
> > > > > > > virtio_...>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >  ^
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > FATAL: Cannot execute sql command:
> > > > > > > --file=/usr/share/ovirt-engine/dbscripts/create_views.sql
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Same issue happens when I tried to update my dev engine with
> > > > > > > local
> > > > >
> > > > >
> > > > >
> > > > > build.
> > > > >
> > > > >
> > > > >
> > > > > > > However, OST is passing [2].
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Any idea what's wrong? I don't see anything wrong in [3], which
> > > > > > > is
> > > > > > > probably the culprit, but I haven't much experience with
> > > > > > > engine/db
> > > > > > > though. Or maybe some issue on my side?
> > > > > >
> > > > > >
> > > > > >
> > > > > > I suppose this failure happens in engine-setup, right? Please
> share
> > > > > > its logs. Thanks.
> > > > >
> > > > >
> > > > >
> > > > > attached
> > > > >
> > > > >
> > > > >
> > > > > > Best regards,
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Thanks
> > > > > > > Vojta
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > >
> > > > >
> > > > >
> > > > >
> https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/11174/
> > > > > art
> > > > >
> > > > >
> > > > >
> > > > > > > ifact/check-patch.el8.x86_64/ [2]
> > > > > > > https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7880/
> > > > > > > [3]
> > > > >
> > > > >
> > > > >
> > > > >
> https://github.com/oVirt/ovirt-engine/commit/52caf76e60ace2266d8283c13
> > > > > 1d4
> > > > >
> > > > >
> > > > >
> > > > > > > 673a2dd79289___
> Devel
> > > > >
> > > > >
> > > > >
> > > > > mailing
> > > > >
> > > > >
> > > > >
> > > > > > > list -- devel@ovirt.org
> > > > > > > To unsubscribe send an email to devel-le...@ovirt.org
> > > > > > > Privacy Statement: https://www.ovirt.org/privacy-policy.html
> > > > > > > oVirt Code of Conduct:
> > > > > > > https://www.ovirt.org/community/about/community-guidelines/
> List
> > > > >
> > > > >
> > > > >
> > > > > > > Archives:
> > > > >
> > > > >
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DWCUVM4H
> > > > > RGV
> > > > >
> > > > >
> > > > >
> > > > > > > SUNLCS3TWL4LHKXDD2NPO/
> > > > >
> > > > >
> > > > >
> > > > > ___
> > > > > Devel mailing list -- devel@ovirt.org
> > > > > To unsubscribe send an email to devel-le...@ovirt.org
> > > > > Privacy Statement: https://www.ovirt.org/privacy-policy.html
> > > > > oVirt Code of Conduct:
> > > > > https://www.ovirt.org/community/about/community-guidelines/
> > > > > List Archives:
> > > > >
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/TDSBKKXB
> > > > > 42Y4
> > > > > V644AI3PDRMV5QWWUF5J/
> > > >
> > > > Hi,
> > > > it happened to me as well on my dev engine.
> > > > I

[ovirt-devel] Re: Issue running OST basic suite master

[Arik Hadas]

On Thu, Feb 4, 2021 at 11:15 AM Marcin Sobczyk  wrote:

> Hi,
>
> On 2/4/21 10:05 AM, Galit Rosenthal wrote:
> > Hi,
> >
> > Basic suite master is failing [1]
> >
> > on
> basic-suite-master.test-scenarios.test_004_basic_sanity.test_import_vm1
> >
> > Can someone have a look?
> Already being handled. I'll post a patch soon.
>

Already being handled by @Lucia Jelinkova   as well
Seems like a consequence of [1] but need to investigate why it fails the
import...

[1] https://gerrit.ovirt.org/#/c/ovirt-engine/+/112976/


>
> Regards, Marcin
>
> >
> > Regards,
> > Galit
> >
> > [1]
> >
> https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_el8nightly/294/consoleFull
> > <
> https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_el8nightly/294/consoleFull
> >
> >
> > --
> >
> > Galit Rosenthal
> >
> > SOFTWARE ENGINEER
> >
> > Red Hat
> >
> > ga...@redhat.com  T: 972-9-7692230
> > 
> >
> > 
> >
> >
> > ___
> > Devel mailing list -- devel@ovirt.org
> > To unsubscribe send an email to devel-le...@ovirt.org
> > Privacy Statement: https://www.ovirt.org/privacy-policy.html
> > oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> > List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/L6KDQLQG3DYT6XVJANQQNCZBCSHBHGPX/
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7WPWCHJYG2ZORKMWKZR5IDCJ4UWFHUD3/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/OGVIU3FUUU6OWTEFAGDB2YETV3I52KJW/

[ovirt-devel] Re: OST fails with NPE in VmDeviceUtils.updateUsbSlots

[Arik Hadas]

On Thu, Jan 28, 2021 at 11:21 AM Marcin Sobczyk  wrote:

> Hi,
>
> On 1/28/21 9:43 AM, Arik Hadas wrote:
> > Hi,
> > Seems like our changes to bios type handing lead to that.
> > Interestingly, OST passed on the patches..
> Can you please provide more info on the verification process?
>

Sure.
The OST job [1] passed on PS 16 of [2] and there was no change on the patch
between PS 16 and PS 17 that got in.

[1]
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/7706/
[2] https://gerrit.ovirt.org/#/c/ovirt-engine/+/111657/


>
> Regards, Marcin
>
> > Anyway, we look into it.
> > Thanks for bringing this to our attention!
> >
> > On Thu, Jan 28, 2021 at 12:13 AM Vojtech Juranek  > <mailto:vjura...@redhat.com>> wrote:
> >
> > Hi,
> > OST fails constantly in test_check_snapshot_with_memory [1] with
> > NPE  in
> > VmDeviceUtils.updateUsbSlots [2]. Build with any additional
> > changes (custom
> > repo) is on [3].
> >
> > Unfortunately, I wasn't able to find the root cause. Could someone
> > please take
> > a look?
> >
> > Thanks
> > Vojta
> >
> > [1]
> >
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull
> > <
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull>
> > [2]
> >
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/exported-artifacts/test_logs/basic-suite-master/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log
> > <
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/exported-artifacts/test_logs/basic-suite-master/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log
> >
> > [3]
> >
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/
> > <
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/
> >___
> > Devel mailing list -- devel@ovirt.org <mailto:devel@ovirt.org>
> > To unsubscribe send an email to devel-le...@ovirt.org
> > <mailto:devel-le...@ovirt.org>
> > Privacy Statement: https://www.ovirt.org/privacy-policy.html
> > <https://www.ovirt.org/privacy-policy.html>
> > oVirt Code of Conduct:
> > https://www.ovirt.org/community/about/community-guidelines/
> > <https://www.ovirt.org/community/about/community-guidelines/>
> > List Archives:
> >
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56UDVBD53EOTUFPDXOPC/
> > <
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56UDVBD53EOTUFPDXOPC/
> >
> >
> >
> > ___
> > Devel mailing list -- devel@ovirt.org
> > To unsubscribe send an email to devel-le...@ovirt.org
> > Privacy Statement: https://www.ovirt.org/privacy-policy.html
> > oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> > List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6FCPAHKN6ECUTMHPPH37UPPMBDUTPDGL/
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/HG73VPO2KFUJ2AT5EO525BRJJNJ3MM5F/

[ovirt-devel] Re: OST fails with NPE in VmDeviceUtils.updateUsbSlots

[Arik Hadas]

Hi,
Seems like our changes to bios type handing lead to that.
Interestingly, OST passed on the patches..
Anyway, we look into it.
Thanks for bringing this to our attention!

On Thu, Jan 28, 2021 at 12:13 AM Vojtech Juranek 
wrote:

> Hi,
> OST fails constantly in test_check_snapshot_with_memory [1] with NPE  in
> VmDeviceUtils.updateUsbSlots [2]. Build with any additional changes
> (custom
> repo) is on [3].
>
> Unfortunately, I wasn't able to find the root cause. Could someone please
> take
> a look?
>
> Thanks
> Vojta
>
> [1]
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/consoleFull
> [2]
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7720/artifact/exported-artifacts/test_logs/basic-suite-master/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log
> [3]
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7718/parameters/
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/M5LFINFRHR3T56UDVBD53EOTUFPDXOPC/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6FCPAHKN6ECUTMHPPH37UPPMBDUTPDGL/

[ovirt-devel] Re: My web console(remote viewer) cannot connect to other cluster's vm.

[Arik Hadas]

On Thu, Jan 14, 2021 at 10:58 AM tommy  wrote:

> Cannot add the apice on ovirt vm's console page.
>
> Please refer the attached file.
>

You need to change the 'Video Type' to QXL first.


>
>
>
>
>
>
> -Original Message-
> From: ybard...@redhat.com  On Behalf Of Yedidyah Bar
> David
> Sent: Thursday, January 14, 2021 4:52 PM
> To: tommy 
> Cc: oVirt development list 
> Subject: Re: [ovirt-devel] Re: My web console(remote viewer) cannot
> connect to other cluster's vm.
>
> On Thu, Jan 14, 2021 at 10:46 AM tommy  wrote:
> >
> > Yeah.
> >
> > Ping is Ok,telnet is also ok.
> >
> > I want know how to enable spice protocol on my ovirt env ?
>
> I think Edit VM should be enough. If it's grayed-out even when the machine
> is down, then I do not know.
>
> >
> >
> >
> > -Original Message-
> > From: ybard...@redhat.com  On Behalf Of Yedidyah
> > Bar David
> > Sent: Thursday, January 14, 2021 4:38 PM
> > To: tommy 
> > Cc: oVirt development list 
> > Subject: Re: [ovirt-devel] Re: My web console(remote viewer) cannot
> connect to other cluster's vm.
> >
> > On Thu, Jan 14, 2021 at 10:02 AM tommy  wrote:
> > >
> > > I tried it .
> > >
> > > 1. The service is well.
> > >
> > > 2.The firewall is ok.
> > >
> > > 3.The CA Cert is installed.
> > >
> > > But can not disappear anything ,just appear:Loading
> >
> > Did you check name resolution from the client? E.g.:
> >
> > ping ooeng.tltd.com
> >
> > ssh -p 6100 -v ooeng.tltd.com
> >
> > >
> > >
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: devel-boun...@ovirt.org  On Behalf Of
> > > tommy
> > > Sent: Thursday, January 14, 2021 3:53 PM
> > > To: 'Yedidyah Bar David' 
> > > Cc: 'oVirt development list' 
> > > Subject: [ovirt-devel] Re: My web console(remote viewer) cannot
> connect to other cluster's vm.
> > >
> > >
> > > Can't connect to websocket proxy server wss://ooeng.tltd.com:6100.
> Please check that:
> > > websocket proxy service is running,
> > > firewalls are properly set,
> > > websocket proxy certificate is trusted by your browser. Default CA
> certificate.
> > >
> > >
> > > I try to using novnc, but it raese such error.
> > > How to check the websocked proxy service ?
> > >
> > >
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: devel-boun...@ovirt.org  On Behalf Of
> > > Yedidyah Bar David
> > > Sent: Thursday, January 14, 2021 3:37 PM
> > > To: tommy 
> > > Cc: oVirt development list 
> > > Subject: [ovirt-devel] Re: My web console(remote viewer) cannot
> connect to other cluster's vm.
> > >
> > > On Thu, Jan 14, 2021 at 9:24 AM tommy  wrote:
> > > >
> > > > I got the point!
> > > >
> > > > There are two env, one is 4.3.6 and the other is 4.4.3.
> > > >
> > > > On 4.3.6, I only can connect to engvm, and there only one
> concole:vnc, I can not using spice, maybe not installed and configured.
> > >
> > > Not sure we support changing the engine VM console type. The engine vm
> is special, some changes are not allowed.
> > >
> > > Please try a different VM.
> > >
> > > > On 4.4.3, I can connect any vm, but is using spice protocol. I
> changed vm to using VNC, it still can not work!
> > >
> > > Did you try also novnc?
> > >
> > > If it works, there might be a problem with your client
> (remote-viewer?). Which one do you use and which version?
> > >
> > > >
> > > > Now, I should install spice server and configure it to see if ovirt
> can using it.
> > > >
> > > > Instaling is vary easy, but what should I do to configure ovirt to
> using spice ?
> > >
> > > You do not need to install a spice server (inside the VM), and even if
> you do, oVirt will not use it.
> > > The spice server here is part of the qemu process.
> > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: ybard...@redhat.com  On Behalf Of
> > > > Yedidyah Bar David
> > > > Sent: Thursday, January 14, 2021 3:07 PM
> > > > To: tommy 
> > > > Cc: oVirt development list 
> > > > Subject: Re: [ovirt-devel] Re: My web console(remote viewer) cannot
> connect to other cluster's vm.
> > > >
> > > > On Thu, Jan 14, 2021 at 9:04 AM tommy  wrote:
> > > > >
> > > > > Thanks for your replay.
> > > > >
> > > > > I have tried using IP instead of hostname, but it still not work.
> > > > >
> > > > > I want choose spice instead of vnc to have a try. But the button
> can not be chosen, what should I do to activate the spiece ?
> > > >
> > > > Edit the VM -> Console. Will likely require rebooting the VM.
> > > >
> > > > Best regards,
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -Original Message-
> > > > > From: devel-boun...@ovirt.org  On
> > > > > Behalf Of Yedidyah Bar David
> > > > > Sent: Thursday, January 14, 2021 2:55 PM
> > > > > To: tommy 
> > > > > Cc: oVirt development list 
> > > > > Subject: [ovirt-devel] Re: My web console(remote viewer) cannot
> connect to other cluster's vm.
> > > > >
> > > > > Hi,
> > > > >
> > > > > On 

[ovirt-devel] Re: basic suite master fails on test_add_vm_pool

[Arik Hadas]

On Wed, Jan 13, 2021 at 9:15 AM Galit Rosenthal  wrote:

> Hi,
>
> Basic suite master fails on  test_add_vm_pool [1]:
>
> 2021-01-13 04:49:22,106+01 INFO
> [org.ovirt.engine.core.bll.AddVmPoolCommand] (default task-1)
> [dfd8e322-0ae8-4b41-ab77-a06df07f7e1f] Lock freed to object
> 'EngineLock:{exclusiveLocks='[test-pool=VM_POOL_NAME]', sharedLocks=''}'
>
> 2021-01-13 04:49:22,164+01 DEBUG 
> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
> (default task-1) [dfd8e322-0ae8-4b41-ab77-a06df07f7e1f] method: 
> getByNameForDataCenter, params: [ef42b9cc-d4e4-4400-bed1-076c81262682, 
> test-pool-1, null, false], timeElapsed: 33ms
> 2021-01-13 04:49:22,164+01 DEBUG 
> [org.ovirt.engine.core.bll.IsVmWithSameNameExistQuery] (default task-1) 
> [dfd8e322-0ae8-4b41-ab77-a06df07f7e1f] Query IsVmWithSameNameExistQuery took 
> 33 ms
> 2021-01-13 04:49:22,165+01 ERROR [org.ovirt.engine.core.bll.AddVmPoolCommand] 
> (default task-1) [] Command 'org.ovirt.engine.core.bll.AddVmPoolCommand' 
> failed: null
> 2021-01-13 04:49:22,165+01 ERROR [org.ovirt.engine.core.bll.AddVmPoolCommand] 
> (default task-1) [] Exception: java.lang.NullPointerException
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommonVmPoolCommand.buildAddVmParameters(CommonVmPoolCommand.java:316)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommonVmPoolCommand.addVmsToPool(CommonVmPoolCommand.java:250)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommonVmPoolCommand.executeCommand(CommonVmPoolCommand.java:232)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1175)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1333)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2009)
>   at 
> org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:140)
>   at 
> org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:79)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1393)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:425)
>   at 
> deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13)
>
>
> Can some one take a look?
>
>
Yes, we're on it


> Thanks
>
> Galit
>
> [1]
> https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_el8nightly/225/artifact/exported-artifacts/test_logs/basic-suite-master/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log
> --
>
> Galit Rosenthal
>
> SOFTWARE ENGINEER
>
> Red Hat 
>
> ga...@redhat.comT: 972-9-7692230
> 
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DSFTRFQLNHTRL4DZ56N5FSPYMKCY6TYL/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ZYOCDXWEYKVE3X7C53DTK6V53NCFI3XI/

[ovirt-devel] Re: Storage OST Failure

[Arik Hadas]

On Wed, Dec 2, 2020 at 5:27 PM Benny Zlotnik  wrote:

> The failing test was merged two days ago[1]
> A snapshot is created when the VM isn't running as well?
>

Yes, the reason is twofold:
1. When you create the snapshot and clone the disks from the snapshot, the
VM can start during the clone operation (no need to wait for the clone
operation to complete before starting the VM)
2. That way the implementation for both running and non-running VM is the
same

We took this approach for export VM to OVA for quite some time


> My guess is because snapshot removal runs detached from the parent[2]
> (which is correct IMHO), it doesn't have the same correlation_id, so
> all_jobs_finished[3] doesn't wait for it. Perhaps snapshot creation/removal
> can be skipped if it's a cold clone?
>

To be honest, I'm not sure that I remember the reason for detaching from
the parent.
I think it was related to compensation but anyway I believe we should
preserve the execution context, and by that probably preserve the
correlation id.
What do you think?


>
> [1] https://gerrit.ovirt.org/c/ovirt-system-tests/+/112039
> [2]
> https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/CloneVmCommand.java#L273
> [3]
> https://github.com/oVirt/ovirt-system-tests/blob/master/basic-suite-master/test-scenarios/test_004_basic_sanity.py#L518
>
> On Wed, Dec 2, 2020 at 2:49 PM Liran Rotenberg 
> wrote:
>
>>
>>
>> On Wed, Dec 2, 2020 at 2:03 PM Nir Soffer  wrote:
>>
>>> On Wed, Dec 2, 2020 at 1:50 PM Steven Rosenberg 
>>> wrote:
>>> >
>>> > Dear Personnel,
>>> >
>>> > It seems that the OST is failing due to a storage issue [1]. The
>>> failure is on the test_verify_and_remove_cloned_vm test which is not
>>> changed on the patch [2] .
>>> >
>>> > The engine log prints the error:
>>> >
>>> > 2020-12-01 13:07:48,297+01 ERROR
>>> [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
>>> (default task-1) []
>>> >
>>> > An error occurred while fetching unregistered disks from Storage
>>> Domain id '26d55c27-bf61-4f01-b5a3-d9a6a01c3e8a'
>>> > 2020-12-01 13:07:48,298+01 DEBUG
>>> [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
>>> (default task-1) [] Skipping format update for domain
>>> '26d55c27-bf61-4f01-b5a3-d9a6a01c3e8a' (type 'ImportExport')
>>> >
>>> > and returns the following error:
>>> >
>>> > 2020-12-01 13:17:46,784+01 ERROR
>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>> task-2) [] Operation Failed:
>>> >
>>> > [Cannot remove VM. The VM is performing an operation on a Snapshot.
>>> Please wait for the operation to finish, and try again.]
>>>
>>> Looks like bad synchronization with the previous test, not waiting
>>> until the other tests changing
>>> the snapshot has completed.
>>>
>> +Shmuel Melamud  +Arik Hadas 
>> That seems correct. We had https://bugzilla.redhat.com/1177156, as part
>> of clone VM there is a snapshot created and later removed. It was merged
>> pretty long ago (Oct 19).
>> Maybe in the current run you had no luck in the timing but that is just
>> an indicator this test might be flaky and should have better handling.
>>
>>>
>>> > Maybe there is a recent change causing a regression here?
>>> >
>>> >
>>> >
>>> > [1]
>>> https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/14137/
>>> > [2]
>>> https://github.com/oVirt/ovirt-system-tests/blob/master/basic-suite-master/test-scenarios/test_004_basic_sanity.py#L526
>>> >
>>> > With Best Regards.
>>> >
>>> > Steven.
>>> >
>>> >
>>> ___
>>> Devel mailing list -- devel@ovirt.org
>>> To unsubscribe send an email to devel-le...@ovirt.org
>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YVUXG2W7G7TIGMGSAQIIFOMCRN63ZDV2/
>>>
>> ___
>> Devel mailing list -- devel@ovirt.org
>> To unsubscribe send an email to devel-le...@ovirt.org
>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5327PJXJILJWAWHWI57GYSN4YNWCYJ2C/
>>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/U4LHWMQ75Z74GRJ7QGKLX3E6NBH5L3KM/

[ovirt-devel] Re: callback hanging making commands executing serial

[Arik Hadas]

On Mon, Oct 19, 2020 at 10:44 AM Benny Zlotnik  wrote:

> IMHO (without knowing well the ansible related stuff), a sensible
> solution would be to make CreateOva async, this would require allowing
> to use AnsibleExecutor in an async manner, but this seems possible
> since AnsibleExecutor#runCommand already performs polling.
> So if the polling is extracted and will instead run in a callback of
> CreateOvaCommand (similar to VM jobs handling in MergeCommand) it
> should work properly. But then again, I'm not very familiar with the
> ansible stuff so it might not be feasible.
>

+1
That's neat!
It wasn't possible before when we invoked the playbook with ProcessBuilder
but now that we use Ansible-runner, that sounds like the right approach to
me (which minimizes the execution time of the 'execute' phase so it could
also fix [1])

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1797553



>
> On Mon, Oct 19, 2020 at 10:27 AM Martin Perina  wrote:
> >
> > Adding Artur ...
> >
> > On Mon, Oct 19, 2020 at 9:06 AM Liran Rotenberg 
> wrote:
> >>
> >> Hi all,
> >> Investigating a bug about serial execution of the same command more
> than once shows a problem with our callbacks.
> >>
> >> The problem is felt when we have synchronized operation (such as
> running ansible runner service) that takes a bit time to be executed.
> >> The problem is, when running multiple commands and one has synchronized
> long operation within a child command, it will hang out all the commands
> running on the engine using callbacks.
> >> The good example is given in the bug, since export OVA command using
> ansible(sync) and the pack_ova script takes time to finish.
> >> This is not really noticeable with short synchronized commands, but it
> does execute serial for them as well instead of parallel.
> >> Running export to OVA command and then other commands with callbacks
> will get them hanging if the export command callback started and it reached
> to performNextOperation.
> >>
> >> Some technical details:
> >> We have one thread in CommandCallbacksPoller[1] that runs, collects the
> current command on the engine and processes them.
> >> Once we have the above scenario (let's say 2 commands), the first one
> will go into callback.doPolling in invokeCallbackMethodsImpl. [2]
> >> In that case it will go to ChildCommandsCallbackBase::doPolling,
> eventually to childCommandsExecutingEnded. [3]
> >> While there are more actions to perform we will do performNextOperation
> [4], which calls executeNextOperation (in the bug case [5]).
> >> When the next operation is long and synchronized, this will block the
> CommandCallbacksPoller thread and only when it finishes the thread is
> released and the callbacks continue working.
> >>
> >> Any idea how to solve this issue?
> >>
> >> Regards,
> >> Liran
> >>
> >> [1] -
> https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/tasks/CommandCallbacksPoller.java#L52-L55
> >> [2] -
> https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/tasks/CommandCallbacksPoller.java#L175
> >> [3] -
> https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/ChildCommandsCallbackBase.java#L80
> >> [4] -
> https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/SerialChildCommandsExecutionCallback.java#L32
> >> [5] -
> https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/exportimport/ExportVmToOvaCommand.java#L199-L231
> >>
> >
> >
> > --
> > Martin Perina
> > Manager, Software Engineering
> > Red Hat Czech s.r.o.
> > ___
> > Devel mailing list -- devel@ovirt.org
> > To unsubscribe send an email to devel-le...@ovirt.org
> > Privacy Statement: https://www.ovirt.org/privacy-policy.html
> > oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> > List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GBOXNWVC3ATCCGUFOVSY6DIP7WPZ5KHZ/
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/W45G7FUW736ODBJAELCBU65KLMS2DHB4/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QVGGBECL5G7CHZUIGTEC

[ovirt-devel] Re: Proposing Benny Zlotnik as an Engine Storage maintainer

[Arik Hadas]

+1

On Fri, Feb 21, 2020 at 8:53 PM Tal Nisan  wrote:

> Hi everyone,
> Benny joined the Storage team in November 2016 and since then played a key
> role in investigating very complex customer bugs around oVirt engine as
> well as contributing to features such as DR, Cinberlib, new export/import
> mechanism and also rewrote and still maintains the LSM mechanism.
> Given his big contribution and knowledge around the engine I'd like to
> nominate Benny as an engine storage maintainer.
>
> Your thoughts please.
> Tal.
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/G52XI2QDSJNAOMZURGAFQJH2RZIHJBRX/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/2YZXDG2L2T3DCAMXSV5YDBBJ56J4ZTFR/

[ovirt-devel] Proposing Andrej Krejcir as a maintainer of engine core

[Arik Hadas]

Dear engine-core maintainers,

I'd like to propose Andrej Krejcir as a maintainer of oVirt engine backend.

Since Jul 2015, Andrej has been contributing to ovirt-engine, mostly in SLA
areas (scheduling, quota, etc) and recently being more and more involved
also in the VIRT areas. This results in ~200 patches to engine master alone.

Personally, I reviewed some of Andrej's contributions and noticed reviews
made by Andrej to others. I sincerely believe Andrej would be a good
addition to the maintainers' group.

Thanks in advance for your response!
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5BHTBPVVAKVKQBZBXZA2QX3PITEVXXRW/

[ovirt-devel] Re: VDSM and safelease

[Arik Hadas]

On Fri, Apr 5, 2019 at 8:33 PM Michal Skrivanek 
wrote:

>
>
> On 5 Apr 2019, at 09:31, Martin Tessun  wrote:
>
> Hi Sandro, Michal,
>
> On 4/5/19 9:01 AM, Sandro Bonazzola wrote:
>
>
>
> Il giorno gio 4 apr 2019 alle ore 19:15 Michal Skrivanek <
> michal.skriva...@redhat.com> ha scritto:
>
>>
>>
>> On 3 Apr 2019, at 13:24, Martin Perina  wrote:
>>
>>
>>
>> On Wed, Apr 3, 2019 at 10:22 AM Martin Tessun  wrote:
>>
>>> On 4/3/19 8:25 AM, Sandro Bonazzola wrote:
>>>
>>> So, according to the thread we have a few action items:
>>> - Decide if we'll drop export domain and iso domain in 4.4
>>>
>>>
>> Just please don't forget that 4.4 engine will need to support 4.2, 4.3
>> and 4.4 hosts and we will need to allow migrating VMs from 4.2/4.3 hosts
>> (EL7) to 4.4 hosts (EL8), so we need to be careful about implications of
>> removing ISO and export data domains
>>
>>
>> In general we can’t remove anything while the corresponding cluster level
>> is still supported. So feel free to drop anything we used in <4.2, but
>> think twice and (run that through virt team at least) before you remove
>> anything used in 4.2+
>>
>
> Ok, so: do we need to take safelease with us in 4.4? If the answer is yes,
> I need to request to find a maintainer for it since we'll need to package
> it for RHEL 8 / CentOS 8.
> This is currently blocking pre-integration testing with RHEL 8 Beta so it
> needs to be addressed quickly in order to be able to proceed.
>
>
> First question: Up to which clusterlevel is safelease needed? Is it needed
> in 4.2 cluster level?
>
>
> Also 4.3
>
> If so: safelease is probably only required on RHEL 7 hypervisors then. In
> that case we don't need it for RHEL 8 Hypervisors.
>
>
> If we won’t have it on RHEL 8/4.4 then you are effectively cutting off
> <4.4 support in 4.4 vdsm. Which also means no live migration between 4.3
> compat level which means no way to upgrade to 4.4 with running VMs.
> I do not think that’s acceptable
>
> So in case we remove safelease from RHV 4.4/RHEL 8 based hypervisors we
> should be fine.
> In case safelease is needed for the engine, we need to think if we want to
> move the engine to RHEL 8 in that case.
>
>
> Cheers,
> Martin
>
>
>
>
>>
>>> If we do so, we still need a way to clean these old domains up, aka
>>> moving the ISOs to a Data Domain or "migrating" an existing ISO domain into
>>> a data Domain.
>>> Export Domain is probably easier, as the OVAs can simply be copied to a
>>> central location. Maybe having an export domain available as a second way
>>> to upload VMs (e.g. for bulk-imports) still makes sense. Esp. as I believe
>>> v2v is relying on export domains today.
>>>
>>> So while I am in favor of getting rid of unneeded code, we need to think
>>> about the benefits they both have and how to get them implemented in case
>>> we agree on removing the old domains.
>>>
>>> So what are the benefits of the ISO domain:
>>>
>>> - Easy to add new ISOs (just copy them to the NFS location
>>> - simple way of sharing between DCs/RHV-Ms
>>> - Having a central place for the ISOs
>>>
>>> The 3rd item can be achieved by admins simply using one Storage Domain
>>> just for ISOs.
>>> The 2nd would probably require some sort of read-only SDs or a way to
>>> have one SD shared between DCs with just one DC having write-access.
>>> The 1st one is probably hardest, as there is no easy way of adding data
>>> to the Storage Domain without tooling. Maybe there are also other ways of
>>> achieving this, the above are just my ideas.
>>>
>>
Just want to mention in this context the integration between oVirt and
muCommander that I'm working on to allow downloading, uploading and
deleting disk images from oVirt storage domains via a lightweight,
cross-platform, open source file manager with a dual-pane interface. A demo
is available: https://youtu.be/daCSlDB_3Jc
Specifically, muCommander could identify ISO files and upload them with the
appropriate image content-type.


>>> Next - what are the benefits of Export Domain:
>>>
>>> - Unattended Import
>>> - Bulk Im- and Export
>>> - Central location
>>> - Easily sharable between DCs/RHV-Ms
>>>
>>> The 4th one is already achieved as we have a common Import/Export tool,
>>> so the OVAs can be easily shared and used by different DCs/RHV-Ms
>>> The 3rd one is something that could be easily achieved administrately.
>>> The 2nd one is already more complicated, but can probably be solved with
>>> Ansible (as the 1st one probably as well).
>>>
>>>
>>> So from my PoV it is easiest to remove the Export Domain while still
>>> having all needed features available. The ISO domain seems a bit harder to
>>> me.
>>> Please think about how to solve this, before we decide on removing both
>>> of them.
>>>
>>>
>>> Cheers,
>>> Martin
>>>
>>>
>>> - Move requirements from safelease to vdsm for numactl, dmidecode and
>>> virt-v2v if not already done
>>> - Elect a maintainer for safelease for 4.3 scope
>>> - Deprecate safelease in 4.3 and remove it on master if we agree on
>>>

[ovirt-devel] Re: Proposing Shmuel Melamud as a Virt backend maintainer

[Arik Hadas]

On Wed, Nov 14, 2018 at 9:22 AM Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

> Hi all,
> I’d like to propose Shmuel as a backend maintainer for the virt area. He’s
> a good candidate with his 200+ patches and insight into complex parts of VM
> pools, templates, q35 support, v2v, sparsify...
> Feel free to bother him more with review requests;)
>

As I reviewed most of Shmuel's patches over the years, Shmuel is familiar
with most of the virt areas in ovirt-engine and has been demonstrating a
good understanding of both the technical and functional sides. I trust
Shmuel for judging and advising future changes in a way that would drive
the project forward.
So that's a +1 from me!


>
> Thanks,
> michal
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/TBYGMRTCF3BUMZ4ZC46DI6BQONNUMWOV/
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/P5PCH572PAMOYVSKMJ3Y4L7MHKON5SOU/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt master (ovirt-engine) ] [ 03.06.18 ] [098_ovirt_provider_ovn.use_ovn_provider ]

[Arik Hadas]

Merged

בתאריך יום א׳, 3 ביוני 2018, 17:23, מאת Arik Hadas ‏:

> Testing: https://gerrit.ovirt.org/#/c/91898/
>
> On Sun, Jun 3, 2018 at 4:27 PM, Arik Hadas  wrote:
>
>> I'm looking
>>
>> On Sun, Jun 3, 2018 at 3:52 PM, Gal Ben Haim  wrote:
>>
>>> Specifically, it failed to unplug a nic from a VM.
>>>
>>>
>>> Suspected patches:
>>>
>>>
>>> https://gerrit.ovirt.org/#/c/91195/
>>>
>>> Link to Job:
>>>
>>>
>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8019/
>>>
>>> Link to all logs:
>>>
>>>
>>>
>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8019/artifact/exported-artifacts/basic-suit-master-el7/
>>>
>>> (Relevant) error snippet from the log:
>>>
>>> 
>>>
>>> 2018-06-03 06:35:30,764-04 INFO  
>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HotUnplugNicVDSCommand] (default 
>>> task-1) [32c965fd] FINISH, HotUnplugNicVDSCommand, return: , log id: 
>>> 18d9b4e9
>>> 2018-06-03 06:35:30,764-04 DEBUG 
>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
>>> (default task-1) [32c965fd] method: runVdsCommand, params: [HotUnplugNic, 
>>> VmNicDeviceVDSParameters:{hostId='d0f5b521-ed04-4b9e-9524-7f8d4be842d0', 
>>> vm.vm_name='vm0', nic='VmNic:{id='9419aca6-2fae-4851-a902-a6b5a88ac691', 
>>> vnicProfileId='8d12fcbb-23e8-432d-845a-ab2b97039955', speed='1', 
>>> type='3', macAddress='00:1a:4a:16:01:0e', linked='true', 
>>> vmId='ec4303e9-2b7a-451d-9f43-836c29767652', vmTemplateId='null'}', 
>>> vmDevice='VmDevice:{id='VmDeviceId:{deviceId='9419aca6-2fae-4851-a902-a6b5a88ac691',
>>>  vmId='ec4303e9-2b7a-451d-9f43-836c29767652'}', device='bridge', 
>>> type='INTERFACE', specParams='[]', address='', managed='true', 
>>> plugged='true', readOnly='false', deviceAlias='', customProperties='[]', 
>>> snapshotId='null', logicalName='null', hostDevice='null'}'}], timeElapsed: 
>>> 55ms
>>> 2018-06-03 06:35:30,765-04 ERROR 
>>> [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] 
>>> (default task-1) [32c965fd] Command 
>>> 'org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand' 
>>> failed: EngineException: 
>>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: 
>>> VDSGenericException: VDSErrorException: Failed to HotUnplugNicVDS, error = 
>>> General Exception: ('macAddr',), code = 100 (Failed with error 
>>> GeneralException and code 100)
>>> 2018-06-03 06:35:30,774-04 DEBUG 
>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
>>> (default task-1) [32c965fd] method: get, params: 
>>> [d0f5b521-ed04-4b9e-9524-7f8d4be842d0], timeElapsed: 3ms
>>> 2018-06-03 06:35:30,781-04 DEBUG 
>>> [org.ovirt.engine.core.utils.timer.FixedDelayJobListener] 
>>> (DefaultQuartzScheduler5) [] Rescheduling 
>>> DEFAULT.org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData#-9223372036854775795
>>>  as there is no unfired trigger.
>>> 2018-06-03 06:35:30,781-04 DEBUG 
>>> [org.ovirt.engine.core.utils.timer.FixedDelayJobListener] 
>>> (DefaultQuartzScheduler6) [] Rescheduling 
>>> DEFAULT.org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices#-9223372036854775801
>>>  as there is no unfired trigger.
>>> 2018-06-03 06:35:30,782-04 ERROR 
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>>> (default task-1) [32c965fd] EVENT_ID: 
>>> NETWORK_DEACTIVATE_VM_INTERFACE_FAILURE(1,015), Failed to unplug Network 
>>> Interface eth2 (VirtIO) from VM vm0. (User: admin@internal-authz)
>>> 2018-06-03 06:35:30,791-04 DEBUG 
>>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>>  (default task-1) [32c965fd] Compiled stored procedure. Call string i
>>>
>>>
>>> 
>>>
>>>
>>> --
>>> *GAL bEN HAIM*
>>> RHV DEVOPS
>>>
>>
>>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PGXYFLNE67WWNPMNJQQXHH5KV2D44XJK/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt master (ovirt-engine) ] [ 03.06.18 ] [098_ovirt_provider_ovn.use_ovn_provider ]

[Arik Hadas]

Testing: https://gerrit.ovirt.org/#/c/91898/

On Sun, Jun 3, 2018 at 4:27 PM, Arik Hadas  wrote:

> I'm looking
>
> On Sun, Jun 3, 2018 at 3:52 PM, Gal Ben Haim  wrote:
>
>> Specifically, it failed to unplug a nic from a VM.
>>
>>
>> Suspected patches:
>>
>>
>> https://gerrit.ovirt.org/#/c/91195/
>>
>> Link to Job:
>>
>>
>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8019/
>>
>> Link to all logs:
>>
>>
>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-test
>> er/8019/artifact/exported-artifacts/basic-suit-master-el7/
>>
>> (Relevant) error snippet from the log:
>>
>> 
>>
>> 2018-06-03 06:35:30,764-04 INFO  
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HotUnplugNicVDSCommand] (default 
>> task-1) [32c965fd] FINISH, HotUnplugNicVDSCommand, return: , log id: 18d9b4e9
>> 2018-06-03 06:35:30,764-04 DEBUG 
>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
>> (default task-1) [32c965fd] method: runVdsCommand, params: [HotUnplugNic, 
>> VmNicDeviceVDSParameters:{hostId='d0f5b521-ed04-4b9e-9524-7f8d4be842d0', 
>> vm.vm_name='vm0', nic='VmNic:{id='9419aca6-2fae-4851-a902-a6b5a88ac691', 
>> vnicProfileId='8d12fcbb-23e8-432d-845a-ab2b97039955', speed='1', 
>> type='3', macAddress='00:1a:4a:16:01:0e', linked='true', 
>> vmId='ec4303e9-2b7a-451d-9f43-836c29767652', vmTemplateId='null'}', 
>> vmDevice='VmDevice:{id='VmDeviceId:{deviceId='9419aca6-2fae-4851-a902-a6b5a88ac691',
>>  vmId='ec4303e9-2b7a-451d-9f43-836c29767652'}', device='bridge', 
>> type='INTERFACE', specParams='[]', address='', managed='true', 
>> plugged='true', readOnly='false', deviceAlias='', customProperties='[]', 
>> snapshotId='null', logicalName='null', hostDevice='null'}'}], timeElapsed: 
>> 55ms
>> 2018-06-03 06:35:30,765-04 ERROR 
>> [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] 
>> (default task-1) [32c965fd] Command 
>> 'org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand' 
>> failed: EngineException: 
>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: 
>> VDSGenericException: VDSErrorException: Failed to HotUnplugNicVDS, error = 
>> General Exception: ('macAddr',), code = 100 (Failed with error 
>> GeneralException and code 100)
>> 2018-06-03 06:35:30,774-04 DEBUG 
>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
>> (default task-1) [32c965fd] method: get, params: 
>> [d0f5b521-ed04-4b9e-9524-7f8d4be842d0], timeElapsed: 3ms
>> 2018-06-03 06:35:30,781-04 DEBUG 
>> [org.ovirt.engine.core.utils.timer.FixedDelayJobListener] 
>> (DefaultQuartzScheduler5) [] Rescheduling 
>> DEFAULT.org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData#-9223372036854775795
>>  as there is no unfired trigger.
>> 2018-06-03 06:35:30,781-04 DEBUG 
>> [org.ovirt.engine.core.utils.timer.FixedDelayJobListener] 
>> (DefaultQuartzScheduler6) [] Rescheduling 
>> DEFAULT.org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices#-9223372036854775801
>>  as there is no unfired trigger.
>> 2018-06-03 06:35:30,782-04 ERROR 
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>> (default task-1) [32c965fd] EVENT_ID: 
>> NETWORK_DEACTIVATE_VM_INTERFACE_FAILURE(1,015), Failed to unplug Network 
>> Interface eth2 (VirtIO) from VM vm0. (User: admin@internal-authz)
>> 2018-06-03 06:35:30,791-04 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>  (default task-1) [32c965fd] Compiled stored procedure. Call string i
>>
>>
>> 
>>
>>
>> --
>> *GAL bEN HAIM*
>> RHV DEVOPS
>>
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/WCWBPV5R4UQFBHRPKA4KSJHCLDNC2WHW/

[ovirt-devel] Re: [ OST Failure Report ] [ oVirt master (ovirt-engine) ] [ 03.06.18 ] [098_ovirt_provider_ovn.use_ovn_provider ]

[Arik Hadas]

I'm looking

On Sun, Jun 3, 2018 at 3:52 PM, Gal Ben Haim  wrote:

> Specifically, it failed to unplug a nic from a VM.
>
>
> Suspected patches:
>
>
> https://gerrit.ovirt.org/#/c/91195/
>
> Link to Job:
>
>
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8019/
>
> Link to all logs:
>
>
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-
> tester/8019/artifact/exported-artifacts/basic-suit-master-el7/
>
> (Relevant) error snippet from the log:
>
> 
>
> 2018-06-03 06:35:30,764-04 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HotUnplugNicVDSCommand] (default 
> task-1) [32c965fd] FINISH, HotUnplugNicVDSCommand, return: , log id: 18d9b4e9
> 2018-06-03 06:35:30,764-04 DEBUG 
> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
> (default task-1) [32c965fd] method: runVdsCommand, params: [HotUnplugNic, 
> VmNicDeviceVDSParameters:{hostId='d0f5b521-ed04-4b9e-9524-7f8d4be842d0', 
> vm.vm_name='vm0', nic='VmNic:{id='9419aca6-2fae-4851-a902-a6b5a88ac691', 
> vnicProfileId='8d12fcbb-23e8-432d-845a-ab2b97039955', speed='1', 
> type='3', macAddress='00:1a:4a:16:01:0e', linked='true', 
> vmId='ec4303e9-2b7a-451d-9f43-836c29767652', vmTemplateId='null'}', 
> vmDevice='VmDevice:{id='VmDeviceId:{deviceId='9419aca6-2fae-4851-a902-a6b5a88ac691',
>  vmId='ec4303e9-2b7a-451d-9f43-836c29767652'}', device='bridge', 
> type='INTERFACE', specParams='[]', address='', managed='true', 
> plugged='true', readOnly='false', deviceAlias='', customProperties='[]', 
> snapshotId='null', logicalName='null', hostDevice='null'}'}], timeElapsed: 
> 55ms
> 2018-06-03 06:35:30,765-04 ERROR 
> [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] 
> (default task-1) [32c965fd] Command 
> 'org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand' failed: 
> EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: 
> VDSGenericException: VDSErrorException: Failed to HotUnplugNicVDS, error = 
> General Exception: ('macAddr',), code = 100 (Failed with error 
> GeneralException and code 100)
> 2018-06-03 06:35:30,774-04 DEBUG 
> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] 
> (default task-1) [32c965fd] method: get, params: 
> [d0f5b521-ed04-4b9e-9524-7f8d4be842d0], timeElapsed: 3ms
> 2018-06-03 06:35:30,781-04 DEBUG 
> [org.ovirt.engine.core.utils.timer.FixedDelayJobListener] 
> (DefaultQuartzScheduler5) [] Rescheduling 
> DEFAULT.org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData#-9223372036854775795
>  as there is no unfired trigger.
> 2018-06-03 06:35:30,781-04 DEBUG 
> [org.ovirt.engine.core.utils.timer.FixedDelayJobListener] 
> (DefaultQuartzScheduler6) [] Rescheduling 
> DEFAULT.org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices#-9223372036854775801
>  as there is no unfired trigger.
> 2018-06-03 06:35:30,782-04 ERROR 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
> (default task-1) [32c965fd] EVENT_ID: 
> NETWORK_DEACTIVATE_VM_INTERFACE_FAILURE(1,015), Failed to unplug Network 
> Interface eth2 (VirtIO) from VM vm0. (User: admin@internal-authz)
> 2018-06-03 06:35:30,791-04 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>  (default task-1) [32c965fd] Compiled stored procedure. Call string i
>
>
> 
>
>
> --
> *GAL bEN HAIM*
> RHV DEVOPS
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/2KJPV6MML2K2YLIWAXD7OJXSTEFF3SR7/

[ovirt-devel] Re: Propose Milan Zamazal as virt maintainer

[Arik Hadas]

On Thu, May 31, 2018 at 1:46 AM, Nir Soffer  wrote:

> On Wed, May 30, 2018 at 11:23 AM Francesco Romani 
> wrote:
>
>> Hi all,
>>
>> Milan Zamazal has been working on the oVirt project for more than 2.5
>> years.
>> Let me highlight some of his many contributions to the project:
>>
>> - Since January this year - 2018, he's been a maintainer for the stable
>> branches.
>> - He developed important features like memory hotunplug, which required
>> tight cooperation
>>   and communication with the other layers of the stack (libvirt, qemu).
>> - He is a mentor in the Outreachy program, which lead to creation of the
>> oVirt Log Analyzer: https://github.com/mz-pdm/ovirt-log-analyzer
>> - He contributed more than 290 patches to the Vdsm project in master
>> branch alone, excluding backports and contributions to Engine
>> - He contributed and is contributing testcases and fixes to the oVirt
>> System Test suite, a tool which was already pivotal in ensuring the
>> quality of the oVirt project.
>>
>> As reviewer, Milan is responsive and his comments are always
>> comprehensive and  well focused,
>> with strong attitude towards getting things done and done right.
>>
>> Milan also demonstrated his ability to adapt to the needs of the project:
>> - he demonstrated careful and thoughtful patch management while
>> maintaining the stable branches
>> - he also demonstrated he's not shy to tackle large and needed changes
>> during the 4.1 and 4.2 cycles,
>>   when we deeply reorganized the XML processing in the virt code.
>>
>> For those reasons, and many more, I think he will be a good addition to
>> the maintainers team, and I propose him as virt co-maintainer.
>>
>> Please share your thoughts
>>
>
> +1
>

FWIW, +1


>
> Nir
>
>
> ___
> Devel mailing list -- devel@ovirt.org
> To unsubscribe send an email to devel-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-
> guidelines/
> List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/
> message/QELJ7YNZVW65HXN5KGRKYUR5F334BA2X/
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YIMB6QHQ37DVL46Z6OEULJEMSYGDTM2R/

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 2018-04-04 ] [006_migrations.prepare_migration_attachments_ipv6]

[Arik Hadas]

On Wed, Apr 11, 2018 at 12:45 PM, Alona Kaplan  wrote:

>
>
> On Tue, Apr 10, 2018 at 6:52 PM, Gal Ben Haim  wrote:
>
>> I'm seeing the same error in [1], during 006_migrations.migrate_vm.
>>
>> [1] http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1650/
>>
>
> Seems like another bug. The migration failed since for some reason the vm
> is already defined on the destination host.
>
> 2018-04-10 11:08:08,685-0400 ERROR (jsonrpc/0) [api] FINISH create
> error=Virtual machine already exists (api:129)
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 122, in
> method
> ret = func(*args, **kwargs)
> File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 191, in create
> raise exception.VMExists()
> VMExists: Virtual machine already exists
>
>
Milan, Francesco, could it be that because of [1] that appears on the
destination host right after shutting down the VM, it remained defined on
that host?

[1] 2018-04-10 11:01:40,005-0400 ERROR (libvirt/events) [vds] Error running
VM callback (clientIF:683)

Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 646, in
dispatchLibvirtEvents

v.onLibvirtLifecycleEvent(event, detail, None)

AttributeError: 'NoneType' object has no attribute 'onLibvirtLifecycleEvent'



>
>
>>
>>
>> On Tue, Apr 10, 2018 at 4:14 PM, Alona Kaplan 
>> wrote:
>>
>>> Hi all,
>>>
>>> Looking at the log it seems that the new GetCapabilitiesAsync is
>>> responsible for the mess.
>>>
>>> -
>>> * 08:29:47 - engine loses connectivity to host 
>>> 'lago-basic-suite-4-2-host-0'.*
>>>
>>>
>>>
>>> *- Every 3 seconds a getCapabalititiesAsync request is sent to the host 
>>> (unsuccessfully).*
>>>
>>>  * before each "getCapabilitiesAsync" the monitoring lock is taken 
>>> (VdsManager,refreshImpl)
>>>
>>>  * "getCapabilitiesAsync" immediately fails and throws 
>>> 'VDSNetworkException: java.net.ConnectException: Connection refused'. The 
>>> exception is caught by 
>>> 'GetCapabilitiesAsyncVDSCommand.executeVdsBrokerCommand' which calls 
>>> 'onFailure' of the callback and re-throws the exception.
>>>
>>>  catch (Throwable t) {
>>> getParameters().getCallback().onFailure(t);
>>> throw t;
>>>  }
>>>
>>> * The 'onFailure' of the callback releases the "monitoringLock" 
>>> ('postProcessRefresh()->afterRefreshTreatment()-> if (!succeeded) 
>>> lockManager.releaseLock(monitoringLock);')
>>>
>>> * 'VdsManager,refreshImpl' catches the network exception, marks 
>>> 'releaseLock = true' and *tries to release the already released lock*.
>>>
>>>   The following warning is printed to the log -
>>>
>>>   WARN  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] 
>>> (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Trying to release 
>>> exclusive lock which does not exist, lock key: 
>>> 'ecf53d69-eb68-4b11-8df2-c4aa4e19bd93VDS_INIT'
>>>
>>>
>>>
>>>
>>> *- 08:30:51 a successful getCapabilitiesAsync is sent.*
>>>
>>>
>>> *- 08:32:55 - The failing test starts (Setup Networks for setting ipv6).
>>> *
>>>
>>> * SetupNetworks takes the monitoring lock.
>>>
>>> *- 08:33:00 - ResponseTracker cleans the getCapabilitiesAsync requests from 
>>> 4 minutes ago from its queue and prints a VDSNetworkException: Vds timeout 
>>> occured.*
>>>
>>>   * When the first request is removed from the queue 
>>> ('ResponseTracker.remove()'), the
>>> *'Callback.onFailure' is invoked (for the second time) -> monitoring lock 
>>> is released (the lock taken by the SetupNetworks!).*
>>>
>>>   * *The other requests removed from the queue also try to release the 
>>> monitoring lock*, but there is nothing to release.
>>>
>>>   * The following warning log is printed -
>>> WARN  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] 
>>> (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Trying to release 
>>> exclusive lock which does not exist, lock key: 
>>> 'ecf53d69-eb68-4b11-8df2-c4aa4e19bd93VDS_INIT'
>>>
>>> - *08:33:00 - SetupNetwork fails on Timeout ~4 seconds after is started*. 
>>> Why? I'm not 100% sure but I guess the late processing of the 
>>> 'getCapabilitiesAsync' that causes losing of the monitoring lock and the 
>>> late + mupltiple processing of failure is root cause.
>>>
>>>
>>> Ravi, 'getCapabilitiesAsync' failure is treated twice and the lock is 
>>> trying to be released three times. Please share your opinion regarding how 
>>> it should be fixed.
>>>
>>>
>>> Thanks,
>>>
>>> Alona.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sun, Apr 8, 2018 at 1:21 PM, Dan Kenigsberg 
>>> wrote:
>>>
 On Sun, Apr 8, 2018 at 9:21 AM, Edward Haas  wrote:

>
>
> On Sun, Apr 8, 2018 at 9:15 AM, Eyal Edri  wrote:
>
>> Was already done by Yaniv - https://gerrit.ovirt.org/#/c/89851.
>> Is it still failing?
>>
>> On Sun, Apr 8, 2018 at 8:59 AM, Barak Korren 
>> wrote:
>>
>>> On 7 April 2018 at 00:30, Dan

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 19-03-2018 ] [ 004_basic_sanity.verify_vm_import ]

[Arik Hadas]

On Tue, Mar 20, 2018 at 5:13 PM, Dafna Ron  wrote:

> Thank you.
> can you please also add the fix here?
>

https://gerrit.ovirt.org/#/c/89250/


>
>
> On Tue, Mar 20, 2018 at 3:03 PM, Arik Hadas  wrote:
>
>>
>>
>> On Mon, Mar 19, 2018 at 6:02 PM, Dafna Ron  wrote:
>>
>>> Hi,
>>>
>>> We have a failed change on test 004_basic_sanity.verify_vm_import.
>>> I can't see anything in the logs that suggests there was a problem with
>>> the import and the error in the junit also suggest this can possibly be in
>>> the api version.
>>>
>>> can you please check the issue?
>>>
>>
>> We found the issue - it happens only when importing VM as new entity
>> (clone), SetVmStatus is not called in that case and so the VM remains in
>> status ImageLockd. Shmuel will post a fix shortly.
>>
>>
>>>
>>> *Link and headline of suspected patches: *
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *core: Call endAction() of all child commands in ImportVmCommand -
>>> https://gerrit.ovirt.org/#/c/89165/
>>> <https://gerrit.ovirt.org/#/c/89165/>Link to
>>> Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362
>>> <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362>Link to
>>> all
>>> logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/artifact/exported-artifacts/basic-suit-4.2-el7/test_logs/basic-suite-4.2/post-004_basic_sanity.py/
>>> <http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/artifact/exported-artifacts/basic-suit-4.2-el7/test_logs/basic-suite-4.2/post-004_basic_sanity.py/>(Relevant)
>>> error snippet from the log: 2018-03-19 11:01:59,679-04 INFO
>>> [org.ovirt.engine.core.bll.exportimport.ImportVmCommand]
>>> (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Lock freed to object
>>> 'EngineLock:{exclusiveLocks='[imported_vm=VM_NAME,
>>> 3494a94b-c253-440b-a6cb-c4c818b19ae1=VM]',
>>> sharedLocks='[cc4255e8-8e1d-4608-a3fe-a2096ed4e9f9=REMOTE_VM]'}'*
>>> Error Message
>>>
>>> False != True after 180 seconds
>>>
>>> Stacktrace
>>>
>>> Traceback (most recent call last):
>>>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>>> testMethod()
>>>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
>>> self.test(*self.arg)
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, 
>>> in wrapped_test
>>> test()
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in 
>>> wrapper
>>> return func(get_test_prefix(), *args, **kwargs)
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in 
>>> wrapper
>>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>>>   File 
>>> "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py",
>>>  line 596, in verify_vm_import
>>> lambda: vm_service.get().status == types.VmStatus.DOWN
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 267, 
>>> in assert_true_within_short
>>> assert_equals_within_short(func, True, allowed_exceptions)
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 251, 
>>> in assert_equals_within_short
>>> func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, 
>>> in assert_equals_within
>>> '%s != %s after %s seconds' % (res, value, timeout)
>>> AssertionError: False != True after 180 seconds
>>>
>>> **
>>>
>>> ___
>>> Devel mailing list
>>> Devel@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 19-03-2018 ] [ 004_basic_sanity.verify_vm_import ]

[Arik Hadas]

On Mon, Mar 19, 2018 at 6:02 PM, Dafna Ron  wrote:

> Hi,
>
> We have a failed change on test 004_basic_sanity.verify_vm_import.
> I can't see anything in the logs that suggests there was a problem with
> the import and the error in the junit also suggest this can possibly be in
> the api version.
>
> can you please check the issue?
>

We found the issue - it happens only when importing VM as new entity
(clone), SetVmStatus is not called in that case and so the VM remains in
status ImageLockd. Shmuel will post a fix shortly.


>
> *Link and headline of suspected patches: *
>
>
>
>
>
>
>
>
>
> *core: Call endAction() of all child commands in ImportVmCommand -
> https://gerrit.ovirt.org/#/c/89165/
> Link to
> Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362
> Link to
> all
> logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/artifact/exported-artifacts/basic-suit-4.2-el7/test_logs/basic-suite-4.2/post-004_basic_sanity.py/
> (Relevant)
> error snippet from the log: 2018-03-19 11:01:59,679-04 INFO
> [org.ovirt.engine.core.bll.exportimport.ImportVmCommand]
> (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Lock freed to object
> 'EngineLock:{exclusiveLocks='[imported_vm=VM_NAME,
> 3494a94b-c253-440b-a6cb-c4c818b19ae1=VM]',
> sharedLocks='[cc4255e8-8e1d-4608-a3fe-a2096ed4e9f9=REMOTE_VM]'}'*
> Error Message
>
> False != True after 180 seconds
>
> Stacktrace
>
> Traceback (most recent call last):
>   File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
> testMethod()
>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
> self.test(*self.arg)
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in 
> wrapped_test
> test()
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in 
> wrapper
> return func(get_test_prefix(), *args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in 
> wrapper
> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>   File 
> "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py",
>  line 596, in verify_vm_import
> lambda: vm_service.get().status == types.VmStatus.DOWN
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 267, in 
> assert_true_within_short
> assert_equals_within_short(func, True, allowed_exceptions)
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 251, in 
> assert_equals_within_short
> func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions
>   File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in 
> assert_equals_within
> '%s != %s after %s seconds' % (res, value, timeout)
> AssertionError: False != True after 180 seconds
>
> **
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Unremovable disks created through the API

[Arik Hadas]

On Wed, Mar 7, 2018 at 11:48 PM, Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
>
> On 07 Mar 2018, at 14:20, Arik Hadas  wrote:
>
>
>
> On Wed, Mar 7, 2018 at 1:41 PM, Richard W.M. Jones 
> wrote:
>
>> On Wed, Mar 07, 2018 at 01:26:39PM +0200, Arik Hadas wrote:
>> > Interesting, that contradicts my intuition - I would imagine that most
>> of
>> > the things are actually known (the things that appear in the top-level
>> part
>> > of the domain xml: memory size, memory size, num of CPUs, name,.. ) and
>> > only things that depend on the content of the disks or things that
>> depend
>> > on installations during the conversion are unknown.
>> > But anyway, it is enough IMO to send the name, memory, CPU and size of
>> the
>> > disks to present something useful to the user and make the necessary
>> > validations at that point.
>>
>> Some of those things are known, but they didn't seem to me to be that
>> interesting for oVirt to know in advance.  In any case what's
>> precisely known before conversion is:
>>
>> (1) The 'source' struct and sub-structs:
>>
>> https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8
>> bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L59
>>
>> (2) The 'overlay' struct (one per disk):
>>
>> https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8
>> bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L175
>>
>> Note only virtual disk size is known, which is near to useless for
>> provisioning storage.
>>
>> (3) The 'target' struct (one per disk):
>>
>> https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8
>> bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L191
>>
>> What's unknown are guest capabilities (hence nothing about what
>> devices should be presented to the guest), inspection data, target bus
>> mapping, real size of disks, etc.
>>
>>
> I see. I think it is sufficient - the information from the 'source' struct
> seems enough just to produce a representative VM entity in the database
> that would be reflected in the UI with status 'importing' and for general
> validations,
>
>
> For having an entity, yes. For meaningful validations, not really.
>

I was thinking mostly about validating the name and the amount of free
space in the target storage domain.
But in the future, when/if the imported VMs could be from different
architecture (PPC?) or different CPU type (q35?) we can also validate that
the target cluster is valid.


> It's not so interesting to see much in oVirt, at least not for the
> virt-v2v command line usage
>
>
Functionality-wise, I agree.
But in a serious management application you don't expect to see memory=0 or
memory=N/A when you open the general tab of the VM that is being imported.
If we can easily get the information from the 'source' struct, I think we
should.

> and the estimated size on the 'target' struct would be enough for storage
> validations and optionally for choosing the right target storage domain.
> The other things are relatively hidden in oVirt's UI and can be added at
> the last phase.
>
>
> What I do not like that much about is that you change the VM at the end
> substantially, and for the import duration it's locked anyway
> The name has a value, and progress, but again, only for a oVirt GUI user -
> and when you are driving the process from cmdline it's just not that
> important
>

Right, but I tend to think that it's better to generalize the process in a
way that the cmdline flow will make an extra step(s) rather than having
separate and different flows.


>
> It's not so difficult to do it like that "externally" - just blindly
> create a completely stub VM in your caller app(wrapper; another
> integration), and then replace it. We do not need to push that into v2v
> directly
>
>
It depends on the amount and quality of validations you want to make at the
beginning of the process and whether or not ovirt-engine should instruct
v2v where to upload the disks to (maybe there should be an option to upload
a disk without specifying the target storage domain so it will be selected
automatically).
v2v will anyway need to trigger a call to ovirt-engine in order to create
that context, that wrapper command. If the data we need at the beginning is
already available to v2v at that point, providing it to that call shouldn't
be that complex, right?

>
> BTW, that's how import from VMware/Xen currently works - we add a VM
> entity based on the domain XML we get fr

Re: [ovirt-devel] [ovirt-users] Unremovable disks created through the API

[Arik Hadas]

On Wed, Mar 7, 2018 at 3:37 PM, Richard W.M. Jones 
wrote:

> On Wed, Mar 07, 2018 at 03:12:58PM +0200, Arik Hadas wrote:
> > If for some predefined period of time no new disk is added or an upload
> > doesn't make any progress (assuming the uploads are done sequentially),
> to
> > fail the import operation and that would roll back the resources (disks,
> > VMs) that were created as part of the import process.
>
> At the moment we're actually trying to remove the disk on failure.
> However the disk_service.remove() API does nothing (doesn't even
> fail).  Perhaps because the transfer isn't finalized on the error
> path?
>

That's weird, it should fail to acquire the disk's lock.
In any case, I think that removing the disk this way would be wrong as we
also store an entity in the image_transfer table that should be removed
(otherwise, another attempt to upload the same disk would probably fail).
Daniel/Nir, I can't find a way to cancel an ongoing upload-image operation
through the API (as we have in the webadmin), am I missing it or is it
missing?


>
> Anyway the code - which needs review - is:
>
> https://www.redhat.com/archives/libguestfs/2018-March/msg00024.html
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~
> rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> Fedora Windows cross-compiler. Compile Windows programs, test, and
> build Windows installers. Over 100 libraries supported.
> http://fedoraproject.org/wiki/MinGW
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Unremovable disks created through the API

[Arik Hadas]

On Wed, Mar 7, 2018 at 1:41 PM, Richard W.M. Jones 
wrote:

> On Wed, Mar 07, 2018 at 01:26:39PM +0200, Arik Hadas wrote:
> > Interesting, that contradicts my intuition - I would imagine that most of
> > the things are actually known (the things that appear in the top-level
> part
> > of the domain xml: memory size, memory size, num of CPUs, name,.. ) and
> > only things that depend on the content of the disks or things that depend
> > on installations during the conversion are unknown.
> > But anyway, it is enough IMO to send the name, memory, CPU and size of
> the
> > disks to present something useful to the user and make the necessary
> > validations at that point.
>
> Some of those things are known, but they didn't seem to me to be that
> interesting for oVirt to know in advance.  In any case what's
> precisely known before conversion is:
>
> (1) The 'source' struct and sub-structs:
>
> https://github.com/libguestfs/libguestfs/blob/
> ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L59
>
> (2) The 'overlay' struct (one per disk):
>
> https://github.com/libguestfs/libguestfs/blob/
> ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L175
>
> Note only virtual disk size is known, which is near to useless for
> provisioning storage.
>
> (3) The 'target' struct (one per disk):
>
> https://github.com/libguestfs/libguestfs/blob/
> ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L191
>
> What's unknown are guest capabilities (hence nothing about what
> devices should be presented to the guest), inspection data, target bus
> mapping, real size of disks, etc.
>
>
I see. I think it is sufficient - the information from the 'source' struct
seems enough just to produce a representative VM entity in the database
that would be reflected in the UI with status 'importing' and for general
validations, and the estimated size on the 'target' struct would be enough
for storage validations and optionally for choosing the right target
storage domain. The other things are relatively hidden in oVirt's UI and
can be added at the last phase.

BTW, that's how import from VMware/Xen currently works - we add a VM entity
based on the domain XML we get from vCenter and at the last phase add the
missing parts when getting the generated OVF from virt-v2v. So for
instance, the VM would have no graphics device until that last phase.


> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~
> rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-top is 'top' for virtual machines.  Tiny program with many
> powerful monitoring features, net stats, disk stats, logging, etc.
> http://people.redhat.com/~rjones/virt-top
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Unremovable disks created through the API

[Arik Hadas]

On Wed, Mar 7, 2018 at 1:01 PM, Tomáš Golembiovský 
wrote:

> Hi,
>
> this sounds like a good idea in general. Few interconnected questions
> though...
>
> On Wed, 7 Mar 2018 10:42:31 +0200
> Arik Hadas  wrote:
>
> > IMHO, the process should be comprised of:
> > 1. virt-v2v calls an API with the (probably partial since the OS and
> other
> > things are unknown at that point) OVF configuration
> > 2. virt-v2v uploads the disks
> > 3. virt-v2v provides the up-to-date configuration
> >
> > Step #1 will enable ovirt-engine:
> > 1. Most importantly, to cleanup uploaded disks in case of an error during
> > the import process. Otherwise, we require the client to clean them up,
> > which can be challenging (e.g., if the virt-v2v process crashes).
>
> Who will handle the removal in case of problems? Engine after timeout?
> Or is the only benefit that administrator can remove all disks in one
> step by removing the VM?
>

So I was thinking that the wrapper command will hold the amount of disks
that should be uploaded for the VM.
If for some predefined period of time no new disk is added or an upload
doesn't make any progress (assuming the uploads are done sequentially), to
fail the import operation and that would roll back the resources (disks,
VMs) that were created as part of the import process.


>
> Note that the uploads do not timeout at the moment. However strange that
> might be. So I assume removing the disks/VM will be impossible anyway
> because of locking.
>

Yeah, I think no timeout was defined because uploading from the browser is
relatively fragile and we didn't want the upload to fail, and the partial
disks to be removed, due to browser issues but rather to be able to resume
their upload. But different logic can be implemented in the wrapping
command.
As for locking, we don't have to call RemoveDisk but instead, to terminate
the upload which will eventually remove the disks.


>
>
> > 2. To inform the user that the process has started - so he won't get
> > surprised by seeing disks being uploaded suddenly. That will give a
> context
> > to these upload operations.
>
> The uploaded disks will still remain unattached though. Or do you plan
> for Engine to create and attach the disks?
>

Right, so when we have that "context" and a VM entity in the databse, we
can attach the disks to the VM when they are created.


>
>
> > 3. To inform the user about the progress of the import process, much like
> > we do today when importing VMs from vSphere to RHV.
>
> How will this be handled? Will Engine report the progress in the Virtual
> Machines view and compute something based on the upload progress?
>

Yes, I don't see why not showing that in the status field (at the VMs tab)
as we do today for VMs that are being imported.
The engine would need to know the estimated actual sizes of the disks in
order to compute it though.


>
> Tomas
>
> > 4. To perform validations on the (partial) VM configuration, e.g.,
> > verifying that no VM with the same name exists/verifying there is enough
> > space (optionally mapping different disks to different storage devices)
> and
> > so on, before uploading the disks.
> >
> >
> > The gaps I see:
> > 1. We don't have a command for step #1 yet but that's something we can
> > provide relatively quickly. We need it also to support uploading an OVA
> via
> > oVirt's webadmin.
> > 2. We have a command for step #3, but it is not exposed via the API.
>
>
> --
> Tomáš Golembiovský 
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Unremovable disks created through the API

[Arik Hadas]

On Wed, Mar 7, 2018 at 12:05 PM, Richard W.M. Jones 
wrote:

> On Wed, Mar 07, 2018 at 10:42:31AM +0200, Arik Hadas wrote:
> > (Moving to devel-list)
> > BTW, I think that the import process should include a preliminary phase
> > where ovirt-engine is informed that the import process starts.
> >
> > Currently, IIUC, the new process is designed to be:
> > 1. virt-v2v uploads the disks
> > 2. virt-v2v calls an API with the OVF configuration so ovirt-engine will
> > add the VM and attach the uploaded disks to that VM
>
> Yes, this is how it happens now, see:
>
> https://www.redhat.com/archives/libguestfs/2018-March/msg00021.html
>
> > IMHO, the process should be comprised of:
> > 1. virt-v2v calls an API with the (probably partial since the OS and
> other
> > things are unknown at that point) OVF configuration
>
> Almost nothing is known at this point, I'm not sure what we could
> provide.  Perhaps just number and virtual size of disks.  It doesn't
> sound like it would be OVF, but something else.
>

Interesting, that contradicts my intuition - I would imagine that most of
the things are actually known (the things that appear in the top-level part
of the domain xml: memory size, memory size, num of CPUs, name,.. ) and
only things that depend on the content of the disks or things that depend
on installations during the conversion are unknown.
But anyway, it is enough IMO to send the name, memory, CPU and size of the
disks to present something useful to the user and make the necessary
validations at that point.


>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~
> rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-df lists disk usage of guests without needing to install any
> software inside the virtual machine.  Supports Linux and Windows.
> http://people.redhat.com/~rjones/virt-df/
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Unremovable disks created through the API

[Arik Hadas]

On Tue, Mar 6, 2018 at 11:19 PM, Richard W.M. Jones 
wrote:

> On Tue, Mar 06, 2018 at 11:14:40PM +0200, Arik Hadas wrote:
> > On Tue, Mar 6, 2018 at 9:18 PM, Richard W.M. Jones 
> > wrote:
> >
> > >
> > > I've been playing with disk uploads through the API.  As a result
> > > I now have lots of disks in the states "Paused by System" and
> > > "Paused by User".  They are not attached to any VM, and I'm logged
> > > in as admin@internal, but there seems to be no way to use them.
> > > Even worse I've now run out of space so can't do anything else.
> > >
> > > How can I remove them?
> >
> >
> > > Screenshot: http://oirase.annexia.org/tmp/ovirt.png
> >
> >
> > Hi Richard,
> >
> > Selecting Upload->Cancel at that tab will remove such a disk.
> > Note that it may take a minute or two.
>
> Yes, that works, thanks.
>

(Moving to devel-list)
BTW, I think that the import process should include a preliminary phase
where ovirt-engine is informed that the import process starts.

Currently, IIUC, the new process is designed to be:
1. virt-v2v uploads the disks
2. virt-v2v calls an API with the OVF configuration so ovirt-engine will
add the VM and attach the uploaded disks to that VM

IMHO, the process should be comprised of:
1. virt-v2v calls an API with the (probably partial since the OS and other
things are unknown at that point) OVF configuration
2. virt-v2v uploads the disks
3. virt-v2v provides the up-to-date configuration

Step #1 will enable ovirt-engine:
1. Most importantly, to cleanup uploaded disks in case of an error during
the import process. Otherwise, we require the client to clean them up,
which can be challenging (e.g., if the virt-v2v process crashes).
2. To inform the user that the process has started - so he won't get
surprised by seeing disks being uploaded suddenly. That will give a context
to these upload operations.
3. To inform the user about the progress of the import process, much like
we do today when importing VMs from vSphere to RHV.
4. To perform validations on the (partial) VM configuration, e.g.,
verifying that no VM with the same name exists/verifying there is enough
space (optionally mapping different disks to different storage devices) and
so on, before uploading the disks.

The gaps I see:
1. We don't have a command for step #1 yet but that's something we can
provide relatively quickly. We need it also to support uploading an OVA via
oVirt's webadmin.
2. We have a command for step #3, but it is not exposed via the API.


>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~
> rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> Fedora Windows cross-compiler. Compile Windows programs, test, and
> build Windows installers. Over 100 libraries supported.
> http://fedoraproject.org/wiki/MinGW
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Fix for: "The CPU type of the cluster is unknown"

[Arik Hadas]

Hi all,

Last week we merged a patch that may reset the cluster's CPU and therefore
could cause the following error to happen when trying to run a VM:
"The CPU type of the cluster is unknown. ..."

A fix is now posted. In the meantime, this can be resolved by refreshing
the capabilities of one of your hosts.

Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Subject: [ OST Failure Report ] [ oVirt Master ] [ Jan 15th 2018 ] [ 006_migrations.migrate_vm ]

[Arik Hadas]

On Fri, Jan 19, 2018 at 12:46 PM, Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
>
> On 18 Jan 2018, at 17:36, Arik Hadas  wrote:
>
>
>
> On Wed, Jan 17, 2018 at 9:41 PM, Milan Zamazal 
> wrote:
>
>> Dafna Ron  writes:
>>
>> > We had a failure in test 006_migrations.migrate_vm
>> > <http://jenkins.ovirt.org/job/ovirt-master_change-queue-test
>> er/4842/testReport/junit/%28root%29/006_migrations/migrate_vm/>.
>> >
>> > the migration failed with reason "VMExists"
>>
>> There are two migrations in 006_migrations.migrate_vm.  The first one
>> succeeded, but if I'm looking correctly into the logs, Engine didn't
>> send Destroy to the source host after the migration had finished.  Then
>> the second migration gets rejected by Vdsm, because Vdsm still keeps the
>> former Vm object instance in Down status.
>>
>> Since the test succeeds most of the time, it looks like some timing
>> issue or border case.  Arik, is it a known problem?  If not, would you
>> like to look into the logs, whether you can see what's happening?
>
>
> Your analysis is correct. That's a nice one actually!
>
> The statistics monitoring cycles of both hosts host-0 and host-1 were
> scheduled in a way that they are executed almost at the same time [1].
>
> Now, at 6:46:34 the VM was migrated from host-1 to host-0.
> At 6:46:42 the migration succeeded - we got events from both hosts, but
> only processed the one from the destination so the VM switched to Up.
> The next statistics monitoring cycle was triggered at 6:46:44 - again, the
> report of that VM from the source host was skipped because we processed the
> one from the destination.
> At 6:46:59, in the next statistics monitoring cycle, it happened again -
> the report of the VM from the source host was skipped.
> The next migration was triggered at 6:47:05 - the engine didn't manage to
> process any report from the source host, so the VM remained Down there.
>
> The probability of this to happen is extremely low.
>
>
> Why wasn't the migration rerun?
>

Good question, because a migration to a particular host (MigrateVmToServer)
was requested.
In this particular case, it seems that there are only two hosts defined so
changing it to MigrateVm wouldn't make any difference though.


>
> However, I think we can make a little tweak to the monitoring code to
> avoid this:
> "If we get the VM as Down on an unexpected host (that is, not the host we
> expect the VM to run on), do not lock the VM"
> It should be safe since we don't update anything in this scenario.
>
> [1] For instance:
> 2018-01-15 06:46:44,905-05 ... GetAllVmStatsVDSCommand ...
> VdsIdVDSCommandParametersBase:{hostId='873a4d36-55fe-4be1-
> acb7-8de9c9123eb2'})
> 2018-01-15 06:46:44,932-05 ... GetAllVmStatsVDSCommand ...
> VdsIdVDSCommandParametersBase:{hostId='31f09289-ec6c-42ff-
> a745-e82e8ac8e6b9'})
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Subject: [ OST Failure Report ] [ oVirt Master ] [ Jan 15th 2018 ] [ 006_migrations.migrate_vm ]

[Arik Hadas]

On Wed, Jan 17, 2018 at 9:41 PM, Milan Zamazal  wrote:

> Dafna Ron  writes:
>
> > We had a failure in test 006_migrations.migrate_vm
> >  tester/4842/testReport/junit/%28root%29/006_migrations/migrate_vm/>.
> >
> > the migration failed with reason "VMExists"
>
> There are two migrations in 006_migrations.migrate_vm.  The first one
> succeeded, but if I'm looking correctly into the logs, Engine didn't
> send Destroy to the source host after the migration had finished.  Then
> the second migration gets rejected by Vdsm, because Vdsm still keeps the
> former Vm object instance in Down status.
>
> Since the test succeeds most of the time, it looks like some timing
> issue or border case.  Arik, is it a known problem?  If not, would you
> like to look into the logs, whether you can see what's happening?


Your analysis is correct. That's a nice one actually!

The statistics monitoring cycles of both hosts host-0 and host-1 were
scheduled in a way that they are executed almost at the same time [1].

Now, at 6:46:34 the VM was migrated from host-1 to host-0.
At 6:46:42 the migration succeeded - we got events from both hosts, but
only processed the one from the destination so the VM switched to Up.
The next statistics monitoring cycle was triggered at 6:46:44 - again, the
report of that VM from the source host was skipped because we processed the
one from the destination.
At 6:46:59, in the next statistics monitoring cycle, it happened again -
the report of the VM from the source host was skipped.
The next migration was triggered at 6:47:05 - the engine didn't manage to
process any report from the source host, so the VM remained Down there.

The probability of this to happen is extremely low.
However, I think we can make a little tweak to the monitoring code to avoid
this:
"If we get the VM as Down on an unexpected host (that is, not the host we
expect the VM to run on), do not lock the VM"
It should be safe since we don't update anything in this scenario.

[1] For instance:

2018-01-15 06:46:44,905-05 ... GetAllVmStatsVDSCommand ...
VdsIdVDSCommandParametersBase:{hostId='873a4d36-55fe-4be1-acb7-8de9c9123eb2'})

2018-01-15 06:46:44,932-05 ... GetAllVmStatsVDSCommand ...
VdsIdVDSCommandParametersBase:{hostId='31f09289-ec6c-42ff-a745-e82e8ac8e6b9'})
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Memory size (1024MB) cannot exceed maximum memory size (0MB).] - when trying to create an instance type

[Arik Hadas]

On Mon, Dec 25, 2017 at 8:29 PM, Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
>
> > On 25 Dec 2017, at 18:41, Juan Hernández  wrote:
> >
> > On 12/25/2017 05:46 PM, Yaniv Kaul wrote:
> >> While trying to add an instance type, I fail with the error:
> >> Operation Failed". Fault detail is "[Cannot add Template. Memory size
> >> (1024MB) cannot exceed maximum memory size (0MB).]
> >> The code is taken from the example in the SDK, so I'm not sure what I'm
> >> doing wrong here.
> >> Code:
> >> instance_types_service.add(
> >> types.InstanceType(
> >> name='myinstancetype',
> >> description='My instance type',
> >> memory=1 * 2**30,
> >> high_availability=types.HighAvailability(
> >> enabled=True,
> >> ),
> >> cpu=types.Cpu(
> >> topology=types.CpuTopology(
> >> cores=2,
> >> sockets=2,
> >> ),
> >> ),
> >> ),
> >> )
> >> engine.log:
> >> 2017-12-25 10:58:19,825-05 INFO
> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-13)
> >> [265ff704-d89f-471b-8207-fc0e1b8816fd] Lock Acquired to object
> >> 'EngineLock:{exclusiveLocks='[myinstancetype=TEMPLATE_NAME,
> >> 703e1265-e160-4a76-82e6-06974156b7b9=TEMPLATE]', sharedLocks='[]'}'
> >> 2017-12-25 10:58:19,831-05 WARN
> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-13)
> >> [265ff704-d89f-471b-8207-fc0e1b8816fd] Validation of action
> 'AddVmTemplate'
> >> failed for user admin@internal-authz. Reasons:
> >> VAR__ACTION__ADD,VAR__TYPE__VM_TEMPLATE,ACTION_TYPE_
> FAILED_MAX_MEMORY_CANNOT_BE_SMALLER_THAN_MEMORY_SIZE,$maxMemory
> >> 0,$memory 1024
> >> 2017-12-25 10:58:19,832-05 INFO
> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-13)
> >> [265ff704-d89f-471b-8207-fc0e1b8816fd] Lock freed to object
> >> 'EngineLock:{exclusiveLocks='[myinstancetype=TEMPLATE_NAME,
> >> 703e1265-e160-4a76-82e6-06974156b7b9=TEMPLATE]', sharedLocks='[]'}'
> >> 2017-12-25 10:58:19,839-05 DEBUG
> >> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
> >> (default task-13) [265ff704-d89f-471b-8207-fc0e1b8816fd] method:
> runAction,
> >> params: [AddVmTemplate,
> >> AddVmTemplateParameters:{commandId='179df9ed-209c-4882-
> a19a-b76a4fe1adb8',
> >> user='null', commandType='Unknown'}], timeElapsed: 33ms
> >> 2017-12-25 10:58:19,846-05 ERROR
> >> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
> (default
> >> task-13) [] Operation Failed: [Cannot add Template. Memory size (1024MB)
> >> cannot exceed maximum memory size (0MB).]
> >> TIA,
> >> Y.
> >
> > I think this is related to the new `memory_policy.max` attribute that
> was introduced in 4.1. I think that for virtual machines it has a default
> value so that it isn't necessary to explicitly provide it. It may not have
> a default value fro instance types. Can you try adding this to the request
> to create the instance type?
> >
> >  memory_policy=types.MemoryPolicy(
> >max=1 * 2**30
> >  )
> >
> > Then try again. If it works I think that we need to fix the engine so
> that it assigns a default value, like it does for virtual machines.
>
> The default for VMs comes from the Blank template. There’s no template for
> instance types, so it always needs to be provided
>

That's true in general, but not for max-memory. Let the blank template be
defined with memory=1024mb and max-memory=4096mb. When a request to add a
VM based on the blank template with memory=8192mb arrives with no
max-memory specified, we cannot take the value of max-memory from the
template since then the operation will fail (as the memory exceeds max
memory)
When adding a VM from rest-api and max-memory is not specified then we set
max-memory = memory * 4 [1, 2].
We can do the same for templates/instance types..

[1]
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/restapi/jaxrs/src/main/java/org/ovirt/engine/api/restapi/resource/BackendVmsResource.java#L163
[2] and that logic is problematic since the result may exceed the max
memory limitation defined for the OS in os-info and then the operation will
fail..


>
>
> ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> >
> >
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Migration failed

[Arik Hadas]

On Tue, Dec 19, 2017 at 12:20 AM, Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
> > On 18 Dec 2017, at 13:21, Milan Zamazal  wrote:
> >
> > Yedidyah Bar David  writes:
> >
> >> On Mon, Dec 18, 2017 at 10:17 AM, Code Review  wrote:
> >>> Jenkins CI posted comments on this change.
> >>>
> >>
> >>> View Change
> >>>
> >>> Patch set 3:Continuous-Integration -1
> >>>
> >>> Build Failed
> >>>
> >>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_
> check-patch-el7-x86_64/2882/
> >>> : FAILURE
> >>
> >> Console output of above job says:
> >>
> >> 08:13:34   # migrate_vm:
> >> 08:16:37 * Collect artifacts:
> >> 08:16:40 * Collect artifacts: Success (in 0:00:03)
> >> 08:16:40   # migrate_vm: Success (in 0:03:06)
> >> 08:16:40   # Results located at
> >> /dev/shm/ost/deployment-basic-suite-master/default/006_
> migrations.py.junit.xml
> >> 08:16:40 @ Run test: 006_migrations.py: Success (in 0:03:50)
> >> 08:16:40 Error occured, aborting
> >>
> >> The file 006_migrations.py.junit.xml [1] says:
> >>
> >> 
> >
> > Reading the logs, I can see the VM migrates normally and seems to be
> > reported to Engine correctly.  When Engine receives end-of-migration
> > event, it sends Destroy to the source (which is correct), calls dumpxmls
> > on the destination in the meantime (looks fine to me) and then calls
>
> looks like a race between getallvmstats reporting VM as Down (statusTime:
> 4296271980) being processed, while there is a Down/MigrationSucceeded event
> arriving (with notify_time 4296273170) at about the same time
> Unfortunately the vdsm.log is not in DEBUG level so there’s very little
> information as to why and what exactly did it send out.
> @infra - can you enable debug log level for vdsm by default?


> It does look like a race to me - does it reproduce?


> > Destroy on the destination, which is weird and I don't understand why
> > the Destroy is invoked.
> >
> > Arik, would you like to take a look?  Maybe I overlooked something or
> > maybe there's a bug.  The logs are at
> > http://jenkins.ovirt.org/job/ovirt-system-tests_master_
> check-patch-el7-x86_64/2882/artifact/exported-artifacts/
> basic-suite-master__logs/test_logs/basic-suite-master/post-
> 006_migrations.py/
> > and the interesting things happen around 2017-12-18 03:13:43,758-05.
>

So it looks like that:
1. the engine polls the VMs from the source host
2. right after #1 we get the down event with proper exit reason (=
migration succeeded) but the engine doesn't process it since the VM is
being locked by the monitoring as part of processing that polling (to
prevent two analysis of the same VM simultaneously).
3. the result of the polling is a VM in status Down and must probably
exit_status=Normal
4. the engine decides to abort the migration and thus the monitoring thread
of the source host destroys the VM on the destination host.

Unfortunately we don't have the exit_reason that is returned by the polling.
However, the only option I can think of is that it is different than
MigrationSucceeded, because otherwise we would have hand-over the VM to the
destination host rather than aborting the migration [1].
That part of the code recently changed as part of [2] - we used to
hand-over the VM when we get from the source host:
status = Down + exit_status = Normal
And in the database: previous_status = MigrationFrom
But after that change we require:
status = Down + exit_status = Normal ** + exit_reason = MigrationSucceeded
**
And in the database: previous_status = MigrationFrom

Long story short, is it possible that VDSM had set the status of the VM to
Down and exit_status to Normal but the exit_reason was not updated (yet?)
to MigrationSucceeded?

[1]
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/monitoring/VmAnalyzer.java#L291
[2] https://gerrit.ovirt.org/#/c/84387/


> >
> >> Can someone please have a look? Thanks.
> >>
> >> As a side note, if indeed this is the cause for the failure for the
> >> job, it's misleading to say "migrate_vm: Success".
> >>
> >> [1]
> >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_
> check-patch-el7-x86_64/2882/artifact/exported-artifacts/
> basic-suite-master__logs/006_migrations.py.junit.xml
> >>
> >>>
> >>> To view, visit change 85177. To unsubscribe, visit settings.
> >>>
> >>> Gerrit-Project: ovirt-system-tests
> >>> Gerrit-Branch: master
> >>> Gerrit-MessageType: comment
> >>> Gerrit-Change-Id: I7eb386744a2a2faf0acd734e0ba44be22dd590b5
> >>> Gerrit-Change-Number: 85177
> >>> Gerrit-PatchSet: 3
> >>> Gerrit-Owner: Yedidyah Bar David 
> >>> Gerrit-Reviewer: Dafna Ron 
> >>> Gerrit-Reviewer: Eyal Edri 
> >>> Gerrit-Reviewer: Jenkins CI
> >>> Gerrit-Reviewer: Sandro Bonazzola 
> >>> Gerrit-Reviewer: Yedidyah Bar David 
> >>> Gerrit-Comment-Date: Mon, 18 Dec 2017 08:17:11 +
> >>> Gerrit-HasComments: No
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > 

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ 05-12-2017 ] [ 006_migrations.migrate_vm ]

[Arik Hadas]

On Mon, Dec 11, 2017 at 6:44 PM, Milan Zamazal  wrote:

> Michal Skrivanek  writes:
>
> > Milan,
> >
> > log [1], VM b3962e5c-08e3-444e-910e-ea675fa1a5c7
> > migration away finished at 2017-12-05 06:26:24,515-0500
> > incoming migration of the same VM back, at 2017-12-05 06:26:46,614-0500
> >
> > seems to me the migration away didn’t really properly clean up the VM.
> Milane,
> > can you check that if logs matches? Perhaps orphaned libvirt’s xml?
>
> It seems that, for unknown reasons, Engine didn’t call Destroy on the
> source after the first migration.  So while the VM was no longer running
> there, it was still present in Vdsm (in Down status) and thus the
> following create call was rejected in API.
>
> There are weird messages in engine.log during the first migration such
> as the following ones, I don’t know whether they are harmless or not:
>
>   2017-12-05 06:26:19,799-05 INFO  [org.ovirt.engine.core.
> vdsbroker.monitoring.VmAnalyzer] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-89)
> [] VM 'b3962e5c-08e3-444e-910e-ea675fa1a5c7'(vm0) was unexpectedly
> detected as 'MigratingTo' on VDS '56bf5bdd-8a95-4b7a-9c72-
> 7206d6b59e38'(lago-basic-suite-master-host-1) (expected on
> '104bd4da-24bb-4368-a5ca-21a465aca70e')
>   2017-12-05 06:26:19,799-05 INFO  [org.ovirt.engine.core.
> vdsbroker.monitoring.VmAnalyzer] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-89)
> [] VM 'b3962e5c-08e3-444e-910e-ea675fa1a5c7' is migrating to VDS
> '56bf5bdd-8a95-4b7a-9c72-7206d6b59e38'(lago-basic-suite-master-host-1)
> ignoring it in the refresh until migration is done
>

These logs are completely harmless - they only indicate that the VM's
run_on_vds still points to the source host and the hand-over was not done
yet.


>
> The migration completion event was successfully obtained by Engine:
>
>   2017-12-05 06:26:24,542-05 INFO  [org.ovirt.engine.core.dal.
> dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-2) []
> EVENT_ID: VM_MIGRATION_DONE(63), Migration completed (VM: vm0, Source:
> lago-basic-suite-master-host-0, Destination: lago-basic-suite-master-host-1,
> Duration: 6 seconds, Total: 6 seconds, Actual downtime: 73ms)
>
> > Otherwise it would need reproduction in CI and some more logs….
> >
> > [1]
> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-
> tester/4278/artifact/exported-artifacts/basic-suit-master-
> el7/test_logs/basic-suite-master/post-006_migrations.py/
> lago-basic-suite-master-host-0/_var_log/vdsm/vdsm.log
> >  tester/4278/artifact/exported-artifacts/basic-suit-master-
> el7/test_logs/basic-suite-master/post-006_migrations.py/
> lago-basic-suite-master-host-0/_var_log/vdsm/vdsm.log>
> >
> >> On 5 Dec 2017, at 13:26, Dafna Ron  wrote:
> >>
> >> Hi,
> >> We had a failure for test 006_migrations.migrate_vm on master.
> >>
> >> There was a libvirt disruption in the migration src  and after that
> vdsm reported the migration as failed because the vm already exists which
> makes me suspect a split bran case.
> >> The patch reported has nothing to do with this issue and I think we
> simply stumbled on a race condition which can cause a split brain.
> >> Please note that there are several metrics related issues reported in
> vdsm logs as well.
> >>
> >> Link and headline of suspected patches:
> >>
> >> Not related
> >>
> >> Link to Job:
> >>
> >>
> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4278/ <
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4278/>
> >>
> >> Link to all logs:
> >>
> >>
> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-
> tester/4278/artifact/exported-artifacts/basic-suit-master-
> el7/test_logs/basic-suite-master/post-006_migrations.py/ <
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-
> tester/4278/artifact/exported-artifacts/basic-suit-master-
> el7/test_logs/basic-suite-master/post-006_migrations.py/>
> >>
> >> (Relevant) error snippet from the log:
> >> 
> >>
> >> Engine:
> >> 2017-12-05 06:26:48,546-05 ERROR [org.ovirt.engine.core.dal.
> dbbroker.auditloghandling.AuditLogDirector] 
> (EE-ManagedThreadFactory-engine-Thread-385)
> [] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed  (VM:
> vm0, Source: lago-
> >> basic-suite-master-host-1, Destination: lago-basic-suite-master-host-
> 0).
> >>
> >>
> >> dst:
> >>
> >> 2017-12-05 06:26:46,615-0500 WARN  (jsonrpc/6) [vds] vm
> b3962e5c-08e3-444e-910e-ea675fa1a5c7 already exists (API:179)
> >> 2017-12-05 06:26:46,615-0500 ERROR (jsonrpc/6) [api] FINISH create
> error=Virtual machine already exists (api:124)
> >> Traceback (most recent call last):
> >>   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line
> 117, in method
> >> ret = func(*args, **kwargs)
> >>   File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 180, in
> create
> >> raise exception.VMExists()
> >> VMExists: Virtual machine already exists
> >> 2017-12-05 06:26:46,620-0500 INFO  (json

Re: [ovirt-devel] [ovirt-users] [ANN] oVirt 4.2.0 First Candidate Release is now available for testing

[Arik Hadas]

On Mon, Dec 11, 2017 at 10:47 AM, Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
> On 11 Dec 2017, at 08:52, Sandro Bonazzola  wrote:
>
>
>
> 2017-12-08 21:49 GMT+01:00 Yaniv Kaul :
>
>>
>>
>> On Fri, Dec 8, 2017 at 5:44 PM, Michal Skrivanek <
>> michal.skriva...@redhat.com> wrote:
>>
>>> I believe we should respin RC2 due to
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1522901
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1522878
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1523399
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1523415
>>
>>
>> I tend to agree, but not all patches are in?
>> Y.
>>
>
> Today we still have 3 approved blockers still on post at
> https://bugzilla.redhat.com/buglist.cgi?quicksearch=
> flag%3Ablocker%2B%20target_milestone%3Aovirt-4.2.0%
> 20status%3Anew%2Cassigned%2Cpost
> ID▲
> oVirt
> Team
> 
> Product
> 
> Component▲
> 
> Assignee
> 
> Status▼
> 
> Summary
> 

Re: [ovirt-devel] oVirt 4.2.0 blockers review - Day 3

[Arik Hadas]

On Thu, Nov 30, 2017 at 11:10 AM, Martin Sivak  wrote:

>
>>>
>>> Bug ID Product Assignee Status Summary
>>> 1518887 
>>> ovirt-hosted-engine-ha b...@ovirt.org NEW ovirt-ha-agent fails parsing
>>> the OVF_STORE due to a change in OVF namespace URI
>>>
>>
>> I'm in favor of reverting the virt change personally.
>>
>
> Unless something else depends on it, the commit message said vdsm needs
> this.
>

vdsm needs it because we would like vdsm to be able to parse OVA files that
are generated by us and OVA files that  are generated by others (and are
VMware-compatbile) with the existing code in vdsm. The existing code is
tailored to VMware-compatible OVA files that are generated by others, in
which the uri doesn't include that slash at the end.

It would be best to adjust ovirt-ha-agent to parse the right uri.
However, it that's too complicated, an alternative solution is to keep
writing the previous uri to ovirt's OVFs, i.e., those in OVF_STORE and in
snapshot's configuration. That would be a pitty since we want to minimize
the differences between the OVFs we generate, but it would be better than
reverting the change..


>
>
>>
>> 1516113 
>>> cockpit-ovirt phbai...@redhat.com POST Deploy the HostedEngine failed
>>> with the default CPU type
>>>
>>
>> Would be happy if the remaining patch could get reviewed quickly.
>>
>
>
>
>
>>
>> 1518693 
>>> ovirt-engine akrej...@redhat.com POST Quota is needed to copy template
>>> disk
>>>
>>
>> This is only via REST and the default quota can be used as a workaround -
>> why is this a blocker?
>>
>
> Automation added it because Raz marked it as Regression. But the change
> was intentional.
>
>
>
>> 1517810 
>>> ovirt-engine stira...@redhat.com Adding additional ha-host fails.
>>>
>>
> This one has a verified engine fix now, we just need to merge it and
> update Node 0 setup (also verified).
>
>
> Martin
>
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] oVirt 4.2.0 blockers review

[Arik Hadas]

On Tue, Nov 28, 2017 at 10:29 PM, Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
> > On 28 Nov 2017, at 15:17, Dan Kenigsberg  wrote:
> >
> >
> >
> > On Tue, Nov 28, 2017 at 9:54 PM, Michal Skrivanek <
> michal.skriva...@redhat.com> wrote:
> >
> >> On 28 Nov 2017, at 06:36, Dan Kenigsberg  wrote:
> >>
> >> On Tue, Nov 28, 2017 at 12:58 PM, Sandro Bonazzola 
> wrote:
> >> Hi,
> >> I'm waiting for last blockers to be fixed for starting a 4.2.0 RC build.
> >> Assignee are in the TO list of this email.
> >> So far we are down to 7 bugs: https://bugzilla.redhat.com/
> buglist.cgi?quicksearch=flag%3Ablocker%2B%20target_
> milestone%3Aovirt-4.2.0%20status%3Anew%2Cassigned%2Cpost
> >>
> >> Please review them and provide an ETA for the fix. If the bug is marked
> as blocker by mistake, please remove the blocker flag and / or postpone the
> bug to a later release.
> >>
> >> Bug ID   Product AssigneeStatus  Summary
> >> 1516113  cockpit-ovirt   phbai...@redhat.com POSTDeploy
> the HostedEngine failed with the default CPU type
> >> 1509629  ovirt-engineah...@redhat.comASSIGNED
> Cold merge failed to remove all volumes
> >> 1507277  ovirt-engineera...@redhat.com   POST[RFE][DR]
> - Vnic Profiles mapping in VMs register from data storage domain should be
> supported also for templates
> >>
> >> Patches are in initial stage of review. Yaniv Lavi is adamant that this
> is indeed a 4.2.0 blocker, so it would cause at least a day or two of delay.
> >>
> >> 1506677  ovirt-enginedchap...@redhat.com POSTHotplug
> fail when attaching a disk with cow format on glusterfs
> >> 1488338  ovirt-enginemlipc...@redhat.com NEW SPM host
> is not moving to Non-Operational status when blocking its access to storage
> domain.
> >> 1512534  ovirt-hosted-engine-ha  pklic...@redhat.com ASSIGNED
>   SHE deployment takes too much time and looks like stuck.
> >> 1496719  vdsmedwa...@redhat.com  POSTPort mirroring is
> not set after VM migration
> >>
> >> We're trying since morning to verify if this has been fixed as a side
> effect by Milan, currently blocked by environmental hurdles (storage
> server; odd SELinux problem in Engine). An answer is still expected today.
> >
> > different fix than the proper hotplug/unplug xml way?
> >
> > No, same one.
>
> ok. it is going to take some time, few days i suppose
>
> >
> > I believe we can also challenge the blocker status since the impact is
> on migrations with hotplugged NICs using mirroring
> >
> >
> > Correct. However, we don't even have an answer. We still did not manager
> to reach a clear verification.
>
> So what I’m saying is to not block on it even if it reproduces, so if it’s
> not even sure it reproduces we indeed should not block GA on it?
>
> > Last news I've heard was that Engine was sending both xml and conf on
> start VM. This sounds more disturbing.
>
> that’s unlikely. Who’s saying that and where? conf is not being sent since
> Nov 8th [1] in RunVM flow.  You still get conf on suspend and resume flow,
> 4.1 compatibility and such, there were some minor issues here and there,
> and the other patches in that bug are needed.
>

False alarm - the engine was not installed properly.


>
> Thanks,
> michal
>
> [1] https://gerrit.ovirt.org/#/c/83407/
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Subject: [ OST Failure Report ] [ oVirt master ] [ 07-07-2017 ] [ 004_basic_sanity.vm_run ]

[Arik Hadas]

On Fri, Jul 7, 2017 at 1:48 PM, Dafna Ron  wrote:

> Hi Arik,
>
> seems there is a second issue in the xml with a required device not sent:
>
> Here is the xml sent:
> http://pastebin.test.redhat.com/501086
>
> This is the error:
>
> 2017-07-07 06:06:36,386-0400 ERROR (vm/300ac8d4) [virt.vm]
> (vmId='300ac8d4-2157-46f6-bc24-291d3da9a83d') The vm start process failed
> (vm:789)
> Traceback (most recent call last):
>   File "/usr/share/vdsm/virt/vm.py", line 723, in _startUnderlyingVm
> self._run()
>   File "/usr/share/vdsm/virt/vm.py", line 2341, in _run
> self._connection.createXML(domxml, flags),
>   File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
> 125, in wrapper
> ret = f(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 646, in
> wrapper
> return func(inst, *args, **kwargs)
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3782, in
> createXML
> if ret is None:raise libvirtError('virDomainCreateXML() failed',
> conn=self)
> libvirtError: Unable to get device ID '(null)': Bad address
>
> Here is some other useful info on the device teardown which can help
> suggest what was the device:
>
> http://pastebin.test.redhat.com/501087
>
> Here is a grep on the vm id from logs: http://pastebin.test.redhat.
> com/501085
> here is a link to full logs:
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
> master/7501/artifact/
>
> Can you please have a look?
>

Reverted https://gerrit.ovirt.org/#/c/78955/ for now.


>
> Thanks,
> Dafna
>
>
>
>
> On 07/07/2017 10:03 AM, Dafna Ron wrote:
>
> Thank you :)
> I will keep an eye on the tests
>
>
> On 07/07/2017 09:43 AM, Arik Hadas wrote:
>
>
>
> בתאריך 6 ביולי 2017 11:13 PM,‏ "Arik Hadas"  כתב:
>
>
>
> On Thu, Jul 6, 2017 at 8:40 PM, Dafna Ron  wrote:
>
>> *Hi, *
>>
>> * vm failed to run with libvrt unsupported configuration (see error
>> below) since the patch is related to 4.2 xml configuration and the vm
>> failed to run on unsupported configuration I suspect its related - can you
>> please have a closer look? *
>>
>
> Right, good analysis.
> Will be easy to fix, just need to copy one of VDSM little hacks in:
> https://github.com/oVirt/vdsm/blob/master/lib/vdsm/virt/vmde
> vices/graphics.py#L87
> To:
> https://github.com/oVirt/ovirt-engine/blob/master/backend/
> manager/modules/vdsbroker/src/main/java/org/ovirt/engine/
> core/vdsbroker/builder/vminfo/LibvirtVmXmlBuilder.java#L1152
> I'll do that tomorrow.
> Not sure why "ci please build" tests passed on that one though.
>
>
> Fix is posted:
> https://gerrit.ovirt.org/#/c/79125
>
>
>
>>
>> * here is a grep from logs on vm id:
>> http://pastebin.test.redhat.com/500889
>> <http://pastebin.test.redhat.com/500889> Test failed:
>> 004_basic_sanity.vm_run Link to suspected patches:
>> https://gerrit.ovirt.org/#/c/78955/ <https://gerrit.ovirt.org/#/c/78955/>
>> Link to Job:
>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/
>> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/>
>> Link to all logs:
>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/
>> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/>
>> Error snippet from the log:  *
>>
>> *engine log: *
>>
>>
>>
>>
>> *2017-07-06 11:26:34,473-04 INFO
>> [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-31)
>> [a7b073ec-04ba-434a-95b4-395957cae6dc] Lock freed to object
>> 'EngineLock:{exclusiveLocks='[d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0=VM]',
>> sharedLocks=''}' {"params": {"d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0":
>> {"status": "Down", "timeOffset": "0", "exitReason": 1, "exitMessage":
>> "unsupported configuration: unknown spice channel name smain", "exitCode":
>> 1}, "notify_time": 4295594320}, "jsonrpc": "2.0", "method":
>> "|virt|VM_status|d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0"}^@ *
>>
>> vdsm log:
>>
>> 2017-07-06 11:26:36,866-0400 ERROR (vm/d3b1b67d) [virt.vm]
>> (vmId='d3b1b67d-d

Re: [ovirt-devel] Subject: [ OST Failure Report ] [ oVirt $VER ] [ $DATE ] [ TEST NAME ]

[Arik Hadas]

בתאריך 6 ביולי 2017 11:13 PM,‏ "Arik Hadas"  כתב:



On Thu, Jul 6, 2017 at 8:40 PM, Dafna Ron  wrote:

> *Hi, *
>
> * vm failed to run with libvrt unsupported configuration (see error below)
> since the patch is related to 4.2 xml configuration and the vm failed to
> run on unsupported configuration I suspect its related - can you please
> have a closer look? *
>

Right, good analysis.
Will be easy to fix, just need to copy one of VDSM little hacks in:
https://github.com/oVirt/vdsm/blob/master/lib/vdsm/virt/
vmdevices/graphics.py#L87
To:
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/
vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/builder/vminfo/
LibvirtVmXmlBuilder.java#L1152
I'll do that tomorrow.
Not sure why "ci please build" tests passed on that one though.


Fix is posted:
https://gerrit.ovirt.org/#/c/79125



>
> * here is a grep from logs on vm id:
> http://pastebin.test.redhat.com/500889
> <http://pastebin.test.redhat.com/500889> Test failed:
> 004_basic_sanity.vm_run Link to suspected patches:
> https://gerrit.ovirt.org/#/c/78955/ <https://gerrit.ovirt.org/#/c/78955/>
> Link to Job:
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/
> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/>
> Link to all logs:
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/
> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/>
> Error snippet from the log:  *
>
> *engine log: *
>
>
>
>
> *2017-07-06 11:26:34,473-04 INFO
> [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-31)
> [a7b073ec-04ba-434a-95b4-395957cae6dc] Lock freed to object
> 'EngineLock:{exclusiveLocks='[d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0=VM]',
> sharedLocks=''}' {"params": {"d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0":
> {"status": "Down", "timeOffset": "0", "exitReason": 1, "exitMessage":
> "unsupported configuration: unknown spice channel name smain", "exitCode":
> 1}, "notify_time": 4295594320}, "jsonrpc": "2.0", "method":
> "|virt|VM_status|d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0"}^@ *
>
> vdsm log:
>
> 2017-07-06 11:26:36,866-0400 ERROR (vm/d3b1b67d) [virt.vm]
> (vmId='d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0') The vm start process failed
> (vm:789)
> Traceback (most recent call last):
>   File "/usr/share/vdsm/virt/vm.py", line 723, in _startUnderlyingVm
> self._run()
>   File "/usr/share/vdsm/virt/vm.py", line 2328, in _run
> self._connection.createXML(domxml, flags),
>   File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
> 125, in wrapper
> ret = f(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 646, in
> wrapper
> return func(inst, *args, **kwargs)
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3782, in
> createXML
> if ret is None:raise libvirtError('virDomainCreateXML() failed',
> conn=self)
> libvirtError: unsupported configuration: unknown spice channel name smain
>
>
> **
>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Subject: [ OST Failure Report ] [ oVirt $VER ] [ $DATE ] [ TEST NAME ]

[Arik Hadas]

On Thu, Jul 6, 2017 at 8:40 PM, Dafna Ron  wrote:

> *Hi, *
>
> * vm failed to run with libvrt unsupported configuration (see error below)
> since the patch is related to 4.2 xml configuration and the vm failed to
> run on unsupported configuration I suspect its related - can you please
> have a closer look? *
>

Right, good analysis.
Will be easy to fix, just need to copy one of VDSM little hacks in:
https://github.com/oVirt/vdsm/blob/master/lib/vdsm/virt/vmdevices/graphics.py#L87
To:
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/builder/vminfo/LibvirtVmXmlBuilder.java#L1152
I'll do that tomorrow.
Not sure why "ci please build" tests passed on that one though.


>
> * here is a grep from logs on vm id:
> http://pastebin.test.redhat.com/500889
>  Test failed:
> 004_basic_sanity.vm_run Link to suspected patches:
> https://gerrit.ovirt.org/#/c/78955/ 
> Link to Job:
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/
> 
> Link to all logs:
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/7491/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/
> 
> Error snippet from the log:  *
>
> *engine log: *
>
>
>
>
> *2017-07-06 11:26:34,473-04 INFO
> [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-31)
> [a7b073ec-04ba-434a-95b4-395957cae6dc] Lock freed to object
> 'EngineLock:{exclusiveLocks='[d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0=VM]',
> sharedLocks=''}' {"params": {"d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0":
> {"status": "Down", "timeOffset": "0", "exitReason": 1, "exitMessage":
> "unsupported configuration: unknown spice channel name smain", "exitCode":
> 1}, "notify_time": 4295594320}, "jsonrpc": "2.0", "method":
> "|virt|VM_status|d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0"}^@ *
>
> vdsm log:
>
> 2017-07-06 11:26:36,866-0400 ERROR (vm/d3b1b67d) [virt.vm]
> (vmId='d3b1b67d-d2fd-4ed7-86d1-795ba2f10bc0') The vm start process failed
> (vm:789)
> Traceback (most recent call last):
>   File "/usr/share/vdsm/virt/vm.py", line 723, in _startUnderlyingVm
> self._run()
>   File "/usr/share/vdsm/virt/vm.py", line 2328, in _run
> self._connection.createXML(domxml, flags),
>   File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
> 125, in wrapper
> ret = f(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 646, in
> wrapper
> return func(inst, *args, **kwargs)
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3782, in
> createXML
> if ret is None:raise libvirtError('virDomainCreateXML() failed',
> conn=self)
> libvirtError: unsupported configuration: unknown spice channel name smain
>
>
> **
>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Chaning the statistics monitoring interval to 30s

[Arik Hadas]

On Wed, Jul 5, 2017 at 9:36 PM, Shirly Radco  wrote:

>
>
> --
>
> SHIRLY RADCO
>
> BI SOFTWARE ENGINEER,
>
> Red Hat Israel <https://www.redhat.com/>
>
> sra...@redhat.com
>  <https://red.ht/sig>
>  <https://redhat.com/summit>
>
>
> On Wed, Jul 5, 2017 at 6:35 PM, Arik Hadas  wrote:
>
>>
>>
>> On Wed, Jul 5, 2017 at 5:57 PM, Roy Golan  wrote:
>>
>>> Hi all,
>>>
>>> I would like to get feedback on $subject and see if I'm missing
>>> something. The impact of this is simply less resource consumption and by
>>> that we can support even greater number of hosts [1] and vms in the system.
>>>
>>
>>> If you think more relaxed statistics collection will affect a core flow
>>> let me know - as far as I see I didn't spot anything critical.
>>>
>>
>>> The overhead of a cycle per host something like that: 2 roundtrips per
>>> host in a cycle, (vm + host stats) and tons of memory allocation for char[]
>>> -> json-> maps of maps -> VM/Vds statistics -> Maps -> serialiazing to DB.
>>>
>>> To minimize the effect of this change we can leave a call to 'list' verb
>>> to at least detect vms existence in the same rate as today.
>>>
>>
>> +1
>>
>>
>>>
>>> Pros
>>> - Engine has rore resources to support more hosts/vms/other activities
>>> of the engine
>>> - Vdsm will have more resources as well (need to tweak vdsm to collect
>>> in the same
>>> frequency)
>>> - less DB writes and reads, approx half of what the system will do in
>>> the in its lifefpan (cause this is what is mainly does all the time)
>>>
>>> Cons
>>> - DWH/Dashboard will have less entries, I'm not sure what is graphical
>>> affect given our hourly resolution (cmiiw here)
>>>
>>
>> What's the frequency of the queries done by DWH/Dashboard? Do they count
>> on the _update_date column of the queried data?
>>
>
> Current frequency is 20 seconds.
> The configurations are queried based on the _update_date, but  statistics
> are queried every interval.
>
> The affect will be less accuracy in the hourly calculations.
>

Ack. So if the proposed change is done, it would probably make sense to
increase the inverval of those queries to be higher than 30 sec, or at
least taking into consideration the _update_date of vm_statistics as well.


>
>
>> I'm asking because if they query the database every minute and say "the
>> time now is 10:30 and the queried data is ..." then there should not be
>> less entries.
>>
>>
>>>
>>>
>>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1430876
>>>
>>
>>>
>>> ___
>>> Devel mailing list
>>> Devel@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>> ___
>> Devel mailing list
>> Devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Chaning the statistics monitoring interval to 30s

[Arik Hadas]

On Wed, Jul 5, 2017 at 5:57 PM, Roy Golan  wrote:

> Hi all,
>
> I would like to get feedback on $subject and see if I'm missing something.
> The impact of this is simply less resource consumption and by that we can
> support even greater number of hosts [1] and vms in the system.
>

> If you think more relaxed statistics collection will affect a core flow
> let me know - as far as I see I didn't spot anything critical.
>

> The overhead of a cycle per host something like that: 2 roundtrips per
> host in a cycle, (vm + host stats) and tons of memory allocation for char[]
> -> json-> maps of maps -> VM/Vds statistics -> Maps -> serialiazing to DB.
>
> To minimize the effect of this change we can leave a call to 'list' verb
> to at least detect vms existence in the same rate as today.
>

+1


>
> Pros
> - Engine has rore resources to support more hosts/vms/other activities of
> the engine
> - Vdsm will have more resources as well (need to tweak vdsm to collect in
> the same
> frequency)
> - less DB writes and reads, approx half of what the system will do in the
> in its lifefpan (cause this is what is mainly does all the time)
>
> Cons
> - DWH/Dashboard will have less entries, I'm not sure what is graphical
> affect given our hourly resolution (cmiiw here)
>

What's the frequency of the queries done by DWH/Dashboard? Do they count on
the _update_date column of the queried data?
I'm asking because if they query the database every minute and say "the
time now is 10:30 and the queried data is ..." then there should not be
less entries.


>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1430876
>

>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Removal of Export Storage Domain and virt-v2v

[Arik Hadas]

On Wed, Jun 14, 2017 at 2:04 PM, Richard W.M. Jones 
wrote:

> As you may know virt-v2v can use the Export Storage Domain (ESD) to
> upload converted virtual machines to oVirt.  It was brought to my
> attention yesterday that the ESD feature is being dropped, so this
> will no longer work at some point in the future.  (BTW I would
> appreciate notice if you're going to drop major features that we rely on.)
>

It caught me by surprise as well while discussing [1].
Allon, has it been shared publicly? if not, can we publicly discuss the
high level design and goals of this effort?


>
> Although virt-v2v can still work via the GUI, this isn't really
> suitable for bulk, scripted upload of hundreds or thousands of VMs.
>

We now enable triggering that process also via REST-API (this is mostly
intended for importing from VMware using ManageIQ, which is expected to be
available soon).


>
> The ESD method was never very good.  It was sort of an undocumented
> back door into oVirt, and it was slow, and still required manual
> intervention (after virt-v2v had done its job, you still needed to go
> through the GUI and import the guests into the Data Domain).
>
> What we really need is a fully scripted method to upload VMs --
> metadata and disk images -- to oVirt.  Maybe one exists already?  If
> not, what's the best way to do this?
>
>
So besides using the REST-API mentioned above, we may be able to deliver
some of the stuff described in [1] before dropping the export domain. With
that, we can think of generating an oVirt-compatible OVA as an output of
virt-v2v that can be consumed in a similar way to how VMs in the export
domain used to be consumed.

Other than that, there is a rumor about introucing a backup-data-domain.
Hopefully, that plan will also be shared publicly and we'll know more
details about it. If it will be close in concept to the export domain, we
may be able to replace the export domain with that domain in virt-v2v.

[1]
http://www.ovirt.org/develop/release-management/features/virt/enhance-import-export-with-ova/


> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~
> rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Feature: enhanced OVA support

[Arik Hadas]

On Tue, May 16, 2017 at 5:56 PM, Adam Litke  wrote:

> Great feature!  I am glad to see you plan to use the existing imageio
> framework for transferring data.
>

Yep, and that's the part we need the feedback on the most.


>
> Will you allow export of VMs from a particular snapshot?  I guess that's
> how you'll have to do it if you want to support export of running VMs.
>

Exactly, that's how we intend to support exporting VMs with no downtime.


>
> I think you should definitely have a comment in the ovf to indicate that
> an OVA was generated by a oVirt.  People will try to use this new feature
> to import random OVAs from who knows where.  I'd also recommend adding a
> version to this comment:  or
> perhaps even a schema version in case you need to deal with compatibility
> issues in the future.
>
>
So yeah, there will be some sort of marking - either by a field or a
comment. We'll see about that.
As for the schema version, it already exists in OVFs generated by oVirt
today exactly for that purpose.


> I agree with Yaniv Kaul that we should offer to sparsify the VM to
> optimize it for export.  We should also return compressed data.  When
> exporting, does it make sense to cache the stored OVA file in some sort of
> ephemeral storage (host local is fine, storage domain may be better) in
> order to allow the client to resume or restart an interrupted download
> without having to start from scratch?
>
>
Well, on the one hand it makes sense and I would expect such a mechanism to
be used for image-download as well. On the other hand, we don't support
this concept of pausing ongoing operations or resuming interrupted
opeations on the engine side, so in the context of ova-download (where the
streaming process is comprised of several steps orchestrated by the engine)
supporting this may require too much effort.


>
> On Sun, May 14, 2017 at 9:56 AM, Arik Hadas  wrote:
>
>> Hi everyone,
>>
>> We would like to share our plan for extending the currently provided
>> support for OVA files with:
>> 1. Support for uploading OVA.
>> 2. Support for exporting a VM/template as OVA.
>> 3. Support for importing OVA that was generated by oVirt (today, we only
>> support those that are VMware-compatible).
>> 4. Support for downloading OVA.
>>
>> This can be found on the feature page
>> <http://www.ovirt.org/develop/release-management/features/virt/enhance-import-export-with-ova/>
>> .
>>
>> Your feedback and cooperation will be highly appreciated.
>>
>> Thanks,
>> Arik
>>
>>
>> ___
>> Users mailing list
>> us...@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>
>
> --
> Adam Litke
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 14/05/2017 ] [ test-repo_ovirt_experimental_master ]

[Arik Hadas]

On Mon, May 15, 2017 at 12:40 PM, Piotr Kliczewski <
piotr.kliczew...@gmail.com> wrote:

> I agree with Nir that the above issue is not related to the job failure.
>
> I see that there is a timeout for:
>
> 2017-05-14 04:10:04,215-04 DEBUG
> [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient]
> (org.ovirt.thread.pool-7-thread-12) [] Message sent: SEND
> destination:jms.topic.vdsm_requests
> content-length:377
> reply-to:jms.topic.vdsm_responses
>
>  Host.setupNetworks, params: {networks={Migration_Net={vlan=200,
> ipv6autoconf=false, nic=eth0, bridged=false, dhcpv6=false, mtu=1500,
> ipv6addr=2001:0db8:85a3:::574c:14ea:0a02/64, switch=legacy}},
> bondings={}, options={connectivityTimeout=120,
> connectivityCheck=true}}>
>
>
> and the timeout occurred at:
>
> 2017-05-14 04:10:07,535-04 DEBUG
> [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker]
> (ResponseWorker) [] Message received:
> {"jsonrpc":"2.0","error":{"code":"lago-basic-suite-
> master-host1:286640862","message":"Vds
> timeout occured"},"id":null}
>
> I see in supervdsm log:
>
> MainProcess|jsonrpc/1::INFO::2017-05-14
> 04:10:05,566::netconfpersistence::52::root::(setNetwork) Adding
> network Migration_Net({u'ipv6autoconf': False, 'nameservers': [],
> u'nic': u'eth0', u'vlan': 200, u'mtu': 1500, u'switch': u'legacy',
> u'dhcpv6': False, u'bridged': False, u'ipv6addr':
> u'2001:0db8:85a3:::574c:14ea:0a02/64', 'defaultRoute': False,
> 'bootproto': 'none'})
> ...
> MainProcess|jsonrpc/1::WARNING::2017-05-14
> 04:10:05,580::ifcfg::247::root::(_addSourceRoute) Invalid input for
> source routing: name=eth0.200, addr=None, netmask=None, gateway=None
> netlink/events::DEBUG::2017-05-14
> 04:10:05,581::concurrent::184::root::(run) START thread
>  (func= method Monitor._scan of  at 0x7f2c8c01ebd0>>, args=(), kwargs={})
> MainProcess|jsonrpc/1::WARNING::2017-05-14
> 04:10:05,581::ifcfg::894::root::(_ignore_missing_device) eth0.200
> didn't exist (yet) and so IPv6 wasn't enabled.
>
> My question is why only 3 seconds to timeout setupNetworks.
>
>
>
> I found one more exception not related to the above issue which we should
> fix:
>

Ack. This one is easy.


>
> 2017-05-14 04:08:08,924-04 DEBUG
> [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl]
> (DefaultQuartzScheduler10) [58fd90c1] Exception:
> java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedMethodAccessor178.invoke(Unknown Source)
> [:1.8.0_131]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> [rt.jar:1.8.0_131]
> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_131]
> at org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(
> JobWrapper.java:83)
> [scheduler.jar:]
> at org.ovirt.engine.core.utils.timer.JobWrapper.execute(
> JobWrapper.java:55)
> [scheduler.jar:]
> at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [rt.jar:1.8.0_131]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [rt.jar:1.8.0_131]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> [rt.jar:1.8.0_131]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> [rt.jar:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_131]
> Caused by: java.lang.NullPointerException
> at org.ovirt.engine.core.vdsbroker.monitoring.VmStatsRefresher.lambda$
> processDevices$0(VmStatsRefresher.java:44)
> [vdsbroker.jar:]
> at java.util.stream.ReferencePipeline$2$1.accept(
> ReferencePipeline.java:174)
> [rt.jar:1.8.0_131]
> at java.util.stream.ReferencePipeline$3$1.accept(
> ReferencePipeline.java:193)
> [rt.jar:1.8.0_131]
> at java.util.stream.ReferencePipeline$2$1.accept(
> ReferencePipeline.java:175)
> [rt.jar:1.8.0_131]
> at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.
> java:1374)
> [rt.jar:1.8.0_131]
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> [rt.jar:1.8.0_131]
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(
> AbstractPipeline.java:471)
> [rt.jar:1.8.0_131]
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(
> ForEachOps.java:151)
> [rt.jar:1.8.0_131]
> at java.util.stream.ForEachOps$ForEachOp$OfRef.
> evaluateSequential(ForEachOps.java:174)
> [rt.jar:1.8.0_131]
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> [rt.jar:1.8.0_131]
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> [rt.jar:1.8.0_131]
> at org.ovirt.engine.core.vdsbroker.monitoring.VmStatsRefresher.
> processDevices(VmStatsRefresher.java:46)
> [vdsbroker.jar:]
> at org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher.poll(
> PollVmStatsRefresher.java:44)
> [vdsbroker.jar:]
> ... 11 more
>
> Thanks,
> Piotr
>
>
> On Sun, May 14, 2017 at 6:12 PM, Nir Soffer  wrote:
> > On Sun, May 14, 2017 at 5:14 PM Shlomo Ben David 
> > wr

Re: [ovirt-devel] [ovirt-users] Feature: enhanced OVA support

[Arik Hadas]

On Mon, May 15, 2017 at 8:05 PM, Richard W.M. Jones 
wrote:

> On Sun, May 14, 2017 at 04:56:56PM +0300, Arik Hadas wrote:
> > Hi everyone,
> >
> > We would like to share our plan for extending the currently provided
> > support for OVA files with:
> > 1. Support for uploading OVA.
> > 2. Support for exporting a VM/template as OVA.
> > 3. Support for importing OVA that was generated by oVirt (today, we only
> > support those that are VMware-compatible).
> > 4. Support for downloading OVA.
> >
> > This can be found on the feature page
> > <http://www.ovirt.org/develop/release-management/features/
> virt/enhance-import-export-with-ova/>
> > .
> >
> > Your feedback and cooperation will be highly appreciated.
>
> The plan as stated seems fine, but I have some reservations which I
> don't think are answered by the page:
>
> (1) How will oVirt know the difference between an OVA generated
> by oVirt and one generated by VMware (or indeed other sources)?
> A VMware OVF has an XML comment:
>
> 
>
> but not any official metadata that I could see.


So that's something that we have not decided on yet.
Indeed, we need some indication of the system that generated the OVA and it
makes sense to have it inside the OVF. I thought about a field that is part
of the VM configuration, like the "origin" field of VMs in ovirt-engine.
Having a comment like you mentioned is also an option.


>
> (By the way, I don't think importing via virt-v2v vs directly will be
> any quicker.  The v2v conversion / device installation takes only a
> fraction of the time.  Most of the time is consumed doing the format
> conversion from VMDK to qcow2.  However you are correct that when you
> know that the source is oVirt/KVM, you should not run virt-v2v.)
>

Note that the disks within the OVA will be of type qcow2. So not only that
no v2v conversion / device installation is needed, but also no format
conversion will be needed on the import and upload flows.


>
> (2) I think you're going to have a lot of fun generating OVAs which
> work on VMware.  As Yaniv says, the devices aren't the same so you'd
> be having to do some virt-v2v -like driver installation / registry
> modification.  Plus the OVF file is essentially a VMware data dump
> encoded as XML.  OVF isn't a real standard.  I bet there are a million
> strange corner cases.  Even writing VMDK files is full of pitfalls.
>
> VMware has a reasonable V2V import tool (actually their P2V tooling is
> very decent).  Of course it's proprietary, but then so is their
> hypervisor.  Maybe oVirt can drive their tools?
>

No no, that fun is not part of the plan :)
The OVAs we'll generate are supposed to contain:
1. OVF - it should be similar to the one virt-v2v generates for oVirt (that
is similar to the one we use internally in oVirt for snapshots and for
backup within storage domains, i.e., OVF-stores). We will definitely need
some extensions, like an indication that the OVA was generated by oVirt. We
may make some tweaks here and there, like removing network interfaces from
the list of resources. But we already are generally aligned with what the
specification says about OVFs.

2. qcow2 disks - thus, no conversion (device installation) and no format
conversion will be needed (we may consider to convert them to raw later on,
since they are expected to be the base volumes of the disks, not sure it
worth the effort though).

I will add this to the feature page.

We are aware to the fact that with this design, the OVAs could not be
directly consumed by others, like VMware. But this could make it easier for
them to make the needed conversion - they won't need to query the VM
configuration from oVirt and won't need to lookup the disks inside oVirt's
storage domains. Anyway, we assume that this conversion is done by other
tools.


> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~
> rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> Fedora Windows cross-compiler. Compile Windows programs, test, and
> build Windows installers. Over 100 libraries supported.
> http://fedoraproject.org/wiki/MinGW
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Feature: enhanced OVA support

[Arik Hadas]

On Mon, May 15, 2017 at 6:11 PM, Derek Atkins  wrote:

> Hi,
>
> Arik Hadas  writes:
>
> > Hi everyone,
> >
> > We would like to share our plan for extending the currently provided
> support
> > for OVA files with:
> > 1. Support for uploading OVA.
> > 2. Support for exporting a VM/template as OVA.
> > 3. Support for importing OVA that was generated by oVirt (today, we only
> > support those that are VMware-compatible).
> > 4. Support for downloading OVA.
> >
> > This can be found on the feature page.
> >
> > Your feedback and cooperation will be highly appreciated.
>
> Sounds like a great plan.
>
> What's the chance that the export would enable an OVA that could be
> loaded back into VMware?
>

I'm afraid that would be too costly.
As far as I know, there are pretty good conversion tools for converting VMs
from KVM to VMware today [1] (which, btw, I consider to be a strength as
Eduardo mentioned before. we certainly not in favor of vendor lock-in). It
would only make sense for them to support converting the OVA we produce,
much like we extended virt-v2v for converting VMware's OVA not long ago -
and I'm pretty sure they won't object to do this, they have all the
building blocks - getting a VM as a single archive that complies with the
OVA standard should make it easy for them).

[1] https://www.youtube.com/watch?v=DJedam7TJWo (the relevant part starts
at 14:40)


>
> > Thanks,
> > Arik
>
> -derek
>
> --
>Derek Atkins 617-623-3745
>de...@ihtfp.com www.ihtfp.com
>Computer and Internet Security Consultant
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Feature: enhanced OVA support

[Arik Hadas]

Hi everyone,

We would like to share our plan for extending the currently provided
support for OVA files with:
1. Support for uploading OVA.
2. Support for exporting a VM/template as OVA.
3. Support for importing OVA that was generated by oVirt (today, we only
support those that are VMware-compatible).
4. Support for downloading OVA.

This can be found on the feature page

.

Your feedback and cooperation will be highly appreciated.

Thanks,
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ACTION SUGGESTED] can't start VMs on master? run engine-setup

[Arik Hadas]

On Mon, Apr 24, 2017 at 7:37 PM, Greg Sheremeta  wrote:

> Hi,
>
> I noticed today that I couldn't start a VM on master. Turns out I needed
> to run engine-setup to pull in some database changes (I assume). For what
> it's worth, the stacktrace [1] didn't say anything about missing columns
> (as it usually will for things like this). It just said it couldn't create
> a new RunVm because of NPE.
>
>
I didn't manage to reproduce it with the latest master.
There were some changes in RunVmCommand#init though, can you please debug
the back-end and see where exactly does it fail?


> Best wishes,
> Greg
>
> [1] https://paste.fedoraproject.org/paste/TYNKZh1tR0f2XU3ifb
> oHCF5M1UNdIGYhyRLivL9gydE=
>
> --
> Greg Sheremeta, MBA
> Sr. Software Engineer
> Red Hat, Inc.
> gsher...@redhat.com
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] oVirt in the new programming-journal

[Arik Hadas]

Hi everyone,

I would like to share a paper that has just been published in the new
programming-journal (that is called "The Art, Science, and Engineering of
Programming") named "Language Oriented Modularity: From Theory to Practice"
[1].

The evaluation of the proposed approach for language-oriented modularity
is done in part with oVirt (specifically, with the engine) and also
the difficulty/inability to reuse a DSAL that was designed for one
application in a different application is demonstrated with oVirt and
muCommander [2].

I hope you will find it interesting :)

[1] http://programming-journal.org/2017/1/10/
[2] https://github.com/mucommander

Regards.
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] System tests failure (wrong test code?)

[Arik Hadas]

On Sun, Mar 19, 2017 at 2:09 PM, Yaniv Kaul  wrote:

>
>
> On Sun, Mar 19, 2017 at 12:14 PM, Arik Hadas  wrote:
>
>>
>>
>> On Sat, Mar 18, 2017 at 12:57 PM, Yaniv Kaul  wrote:
>>
>>>
>>>
>>> On Sat, Mar 18, 2017 at 12:05 AM, Michal Skrivanek 
>>> wrote:
>>>
>>>>
>>>> On 17 Mar 2017, at 20:49, Yaniv Kaul  wrote:
>>>>
>>>>
>>>>
>>>> On Thu, Mar 16, 2017 at 8:16 PM, Michal Skrivanek 
>>>> wrote:
>>>>
>>>>>
>>>>> On 16 Mar 2017, at 15:18, Yaniv Kaul  wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Mar 16, 2017 at 10:46 AM, Nir Soffer  wr
>>>>> ote:
>>>>>
>>>>>> If found this error in system tests - looks like wrong assert - code
>>>>>> should check
>>>>>> if disk is not None before checking state.
>>>>>>
>>>>>> I'm not sure who is the owner of this test, so posting here.
>>>>>>
>>>>>
>>>>> Theoretically, perhaps.
>>>>> Practically, it worked (until yesterday?) and now I'm also seeing this
>>>>> failure - it's not a coincidence.
>>>>>
>>>>> However, looking at my failure[1], I'm seeing other nasty stuff, which
>>>>> may explain the later on failures
>>>>>
>>>>> For example, new NPE I have not seen in the past:
>>>>> 2017-03-16 09:57:57,581-04 INFO  
>>>>> [org.ovirt.engine.core.bll.AddVmTemplateCommand]
>>>>> (DefaultQuartzScheduler1) [5d94233] Ending command
>>>>> 'org.ovirt.engine.core.bll.AddVmTemplateCommand' successfully.
>>>>>
>>>>>
>>>>> Is this IST or which TZ?
>>>>>
>>>>
>>>> Interesting question - I guess that's why we need the offset from UTC
>>>> or use UTC in the logs.
>>>>
>>>>
>>>>> Likely https://gerrit.ovirt.org/#/c/69323 merged 10:33:20 CET today
>>>>>
>>>>
>>>> That's the cause or the fix?
>>>>
>>>>
>>>> oh, sorry, the cause.
>>>> we were not able to find anyone to confirm and resolve it today
>>>> unfortunately
>>>>
>>>
>> A fix is posted: https://gerrit.ovirt.org/#/c/74276/
>> The cause is explained in the commit message.
>>
>
Merged.


>
> Thanks.
>
>
>> In our defense I'll say that somehow the system tests passed when we ran
>> them ([1]) and that the command for adding a template became way too
>> complicated than it is supposed to be with the introduction of import from
>> glance and instance-types.
>>
>
> Do we have intentions to simplify it?
>

Well, we had such intentions but these things are always the first
candidates to postpone. We should.


>
>
>>
>> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_manual/129/
>>
>>
>>>
>>> A secondary bug is that it fails every 10 seconds or so, continuously.
>>> Y.
>>>
>>
>> Right, because it fails in the end-action phase of the command and we
>> have a retry mechanism that attempts to end the command (unless stated
>> otherwise) periodically.
>>
>
> But the end user experience is an event every few seconds (10?) that we
> fail an operation.
> When does it give up?
>

For the most part, chances are extremely low for a command to fail in its
end-action phase. Generally, commands either fail in their validation phase
or execution phase or when their async tasks/jobs fail. In those case, we
don't have/need a mechanism for periodic retry. On the other hand, the
end-action phase is relatively short and simple and is really not expected
to fail (unless there is a bug, like in this case), but if it does fail -
we have to retry and the user will probably experience more acute issues
than a flood of audit messages (e.g., inability to connect to the
database). AFAIK there is no limit to the number of retry attempts in this
case.


> Y.
>
>
>>
>>
>>>
>>>
>>>
>>>>
>>>> Y.
>>>>
>>>>
>>>>>
>>>>> 2017-03-16 09:57:57,591-04 INFO  
>>>>> [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand]
>>>>> (DefaultQuartzScheduler1) [5d94233] START, SetVmStatusVDSCommand(
>>>>> SetVmStatusVDSCommandParameters:{runAsync='

Re: [ovirt-devel] System tests failure (wrong test code?)

[Arik Hadas]

On Sat, Mar 18, 2017 at 12:57 PM, Yaniv Kaul  wrote:

>
>
> On Sat, Mar 18, 2017 at 12:05 AM, Michal Skrivanek 
> wrote:
>
>>
>> On 17 Mar 2017, at 20:49, Yaniv Kaul  wrote:
>>
>>
>>
>> On Thu, Mar 16, 2017 at 8:16 PM, Michal Skrivanek 
>> wrote:
>>
>>>
>>> On 16 Mar 2017, at 15:18, Yaniv Kaul  wrote:
>>>
>>>
>>>
>>> On Thu, Mar 16, 2017 at 10:46 AM, Nir Soffer  wrote:
>>>
 If found this error in system tests - looks like wrong assert - code
 should check
 if disk is not None before checking state.

 I'm not sure who is the owner of this test, so posting here.

>>>
>>> Theoretically, perhaps.
>>> Practically, it worked (until yesterday?) and now I'm also seeing this
>>> failure - it's not a coincidence.
>>>
>>> However, looking at my failure[1], I'm seeing other nasty stuff, which
>>> may explain the later on failures
>>>
>>> For example, new NPE I have not seen in the past:
>>> 2017-03-16 09:57:57,581-04 INFO  
>>> [org.ovirt.engine.core.bll.AddVmTemplateCommand]
>>> (DefaultQuartzScheduler1) [5d94233] Ending command
>>> 'org.ovirt.engine.core.bll.AddVmTemplateCommand' successfully.
>>>
>>>
>>> Is this IST or which TZ?
>>>
>>
>> Interesting question - I guess that's why we need the offset from UTC or
>> use UTC in the logs.
>>
>>
>>> Likely https://gerrit.ovirt.org/#/c/69323 merged 10:33:20 CET today
>>>
>>
>> That's the cause or the fix?
>>
>>
>> oh, sorry, the cause.
>> we were not able to find anyone to confirm and resolve it today
>> unfortunately
>>
>
A fix is posted: https://gerrit.ovirt.org/#/c/74276/
The cause is explained in the commit message.
In our defense I'll say that somehow the system tests passed when we ran
them ([1]) and that the command for adding a template became way too
complicated than it is supposed to be with the introduction of import from
glance and instance-types.

[1] http://jenkins.ovirt.org/job/ovirt-system-tests_manual/129/


>
> A secondary bug is that it fails every 10 seconds or so, continuously.
> Y.
>

Right, because it fails in the end-action phase of the command and we have
a retry mechanism that attempts to end the command (unless stated
otherwise) periodically.


>
>
>
>>
>> Y.
>>
>>
>>>
>>> 2017-03-16 09:57:57,591-04 INFO  
>>> [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand]
>>> (DefaultQuartzScheduler1) [5d94233] START, SetVmStatusVDSCommand(
>>> SetVmStatusVDSCommandParameters:{runAsync='true',
>>> vmId='----', status='Down',
>>> exitStatus='Normal'}), log id: 30ee3299
>>> 2017-03-16 09:57:57,593-04 DEBUG [org.ovirt.engine.core.dal.dbb
>>> roker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>> (DefaultQuartzScheduler1) [5d94233] Compiled stored procedure. Call string
>>> is [{call getvmdynamicbyvmguid(?)}]
>>> 2017-03-16 09:57:57,594-04 DEBUG [org.ovirt.engine.core.dal.dbb
>>> roker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
>>> (DefaultQuartzScheduler1) [5d94233] SqlCall for procedure
>>> [GetVmDynamicByVmGuid] compiled
>>> 2017-03-16 09:57:57,595-04 ERROR 
>>> [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand]
>>> (DefaultQuartzScheduler1) [5d94233] Command 'SetVmStatusVDSCommand(
>>> SetVmStatusVDSCommandParameters:{runAsync='true',
>>> vmId='----', status='Down',
>>> exitStatus='Normal'})' execution failed: null
>>> 2017-03-16 09:57:57,595-04 DEBUG 
>>> [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand]
>>> (DefaultQuartzScheduler1) [5d94233] Exception:
>>> java.lang.NullPointerException
>>> at org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand.execut
>>> eVDSCommand(SetVmStatusVDSCommand.java:33) [vdsbroker.jar:]
>>> at 
>>> org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:73)
>>> [vdsbroker.jar:]
>>> at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)
>>> [dal.jar:]
>>> at org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandE
>>> xecutor.execute(DefaultVdsCommandExecutor.java:14) [vdsbroker.jar:]
>>> at 
>>> org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:404)
>>> [vdsbroker.jar:]
>>> at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsComman
>>> d(VDSBrokerFrontendImpl.java:33) [bll.jar:]
>>> at org.ovirt.engine.core.bll.VmHandler.unLockVm(VmHandler.java:377)
>>> [bll.jar:]
>>>
>>>
>>>
>>> Y.
>>>
>>> [1]  http://jenkins.ovirt.org/job/ovirt-system-tests_master_chec
>>> k-patch-el7-x86_64/326/artifact/exported-artifacts/basic_sui
>>> te_master__logs/test_logs/basic-suite-master/post-004_basic_
>>> sanity.py/lago-basic-suite-master-engine/_var_log/ovirt-
>>> engine/engine.log
>>>
>>>
 *08:28:05*   # snapshots_merge: *08:28:31* Unhandled exception in 
  at 0x276a5f0>*08:28:31* Traceback (most recent call 
 last):*08:28:31*   File 
 "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in 
 assert_equals_within*08:28:31* res = func()*08:28:31*   File 
 "/home/jenkins/workspace/ovirt-system-tests_manual

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 31.01.2017 ] [add_secondary_storage_domains]

[Arik Hadas]

Fixed

On Tue, Jan 31, 2017 at 6:34 PM, Arik Hadas  wrote:

> Working on a fix
>
> On Tue, Jan 31, 2017 at 5:42 PM, Yaniv Kaul  wrote:
>
>>
>>
>> On Tue, Jan 31, 2017 at 4:43 PM, Evgheni Dereveanchin <
>> edere...@redhat.com> wrote:
>>
>>> Test failed: add_secondary_storage_domains
>>> Links to suspected patches:
>>>  https://gerrit.ovirt.org/71376
>>>  https://gerrit.ovirt.org/71427
>>>  https://gerrit.ovirt.org/71345
>>>
>>> Link to Job:
>>>  http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/5040/
>>>
>>> Link to all logs:
>>>  http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_m
>>> aster/5040/artifact/exported-artifacts/basic-suit-master-el7
>>> /test_logs/basic-suite-master/post-002_bootstrap.py/
>>
>>
>> I think the issue is actually:
>>
>> 2017-01-31 05:46:51,412-05 ERROR 
>> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (DefaultQuartzScheduler6) 
>> [17e8e3e] Command 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: 
>> null
>> 2017-01-31 05:46:51,412-05 ERROR 
>> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (DefaultQuartzScheduler6) 
>> [17e8e3e] Exception: java.lang.NullPointerException
>>  at 
>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.addManagedDevice(VmDeviceUtils.java:1553)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.addManagedDevice(VmDeviceUtils.java:1509)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.addCdDevice(VmDeviceUtils.java:183)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1390)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1444)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.AddVmTemplateCommand.lambda$executeCommand$1(AddVmTemplateCommand.java:356)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202)
>>  [utils.jar:]
>>  at 
>> org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand(AddVmTemplateCommand.java:338)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1251)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1391)
>>  [bll.jar:]
>>  at 
>> org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2055)
>>  [bll.jar:]
>>
>> ...
>>
>> 2017-01-31 05:46:51,430-05 ERROR 
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
>> (DefaultQuartzScheduler6) [17e8e3e] EVENT_ID: 
>> USER_ADD_VM_TEMPLATE_FAILURE(36), Correlation ID: 17e8e3e, Job ID: 
>> 6d664704-97ff-4ff3-8aaa-63389dc51663, Call Stack: null, Custom Event ID: -1, 
>> Message: Failed creating Template CirrOS_0.3.4_for_x86_64_glance_template.
>> 2017-01-31 05:46:51,431-05 INFO  
>> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (DefaultQuartzScheduler6) 
>> [17e8e3e] Lock freed to object 'EngineLock:{exclusiveLocks='null', 
>> sharedLocks='[----=> ACTION_TYPE_FAILED_TEMPLATE_IS_BEING_CREATED>]'}'
>> 2017-01-31 05:46:51,435-05 DEBUG 
>> [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExceptionTranslator]
>>  (DefaultQuartzScheduler6) [17e8e3e] Translating SQLException with SQL state 
>> '23503', error code '0', message [ERROR: insert or update on table 
>> "disk_vm_element" violates foreign key constraint 
>> "fk_disk_vm_element_vm_static"
>>   Detail: Key (vm_id)=(7dd54891-477c-4e64-ad47-02afc12ab32d) is not present 
>> in table "vm_static".
>>   Where: SQL statement "INSERT INTO disk_vm_element (
>> disk_id,
>> vm_id,
>> is_boot,
>> pass_discard,
>> disk_interface,
>> is_using_scsi_reservation)
>> VALUES (
>> v_disk_id,
>> v_vm_id,
>> v_is_boot,
>> v_pass_discard,
>> v_disk_interface,
>> v_is_using_scsi_reservation)"
>> PL/pgSQL function insertdiskvmelement(uuid,uuid,boolean,boolean,character 
>> varying,boolean) line 3 at SQL statement]; SQL was [{call 
>> insertdiskvmelement(?, ?, ?, ?, ?, ?)}] for task [CallableStatementCa

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 31.01.2017 ] [add_secondary_storage_domains]

[Arik Hadas]

Working on a fix

On Tue, Jan 31, 2017 at 5:42 PM, Yaniv Kaul  wrote:

>
>
> On Tue, Jan 31, 2017 at 4:43 PM, Evgheni Dereveanchin  > wrote:
>
>> Test failed: add_secondary_storage_domains
>> Links to suspected patches:
>>  https://gerrit.ovirt.org/71376
>>  https://gerrit.ovirt.org/71427
>>  https://gerrit.ovirt.org/71345
>>
>> Link to Job:
>>  http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/5040/
>>
>> Link to all logs:
>>  http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_m
>> aster/5040/artifact/exported-artifacts/basic-suit-master-el7
>> /test_logs/basic-suite-master/post-002_bootstrap.py/
>
>
> I think the issue is actually:
>
> 2017-01-31 05:46:51,412-05 ERROR 
> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (DefaultQuartzScheduler6) 
> [17e8e3e] Command 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: 
> null
> 2017-01-31 05:46:51,412-05 ERROR 
> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (DefaultQuartzScheduler6) 
> [17e8e3e] Exception: java.lang.NullPointerException
>   at 
> org.ovirt.engine.core.bll.utils.VmDeviceUtils.addManagedDevice(VmDeviceUtils.java:1553)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.utils.VmDeviceUtils.addManagedDevice(VmDeviceUtils.java:1509)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.utils.VmDeviceUtils.addCdDevice(VmDeviceUtils.java:183)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1390)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1444)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.AddVmTemplateCommand.lambda$executeCommand$1(AddVmTemplateCommand.java:356)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202)
>  [utils.jar:]
>   at 
> org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand(AddVmTemplateCommand.java:338)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1251)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1391)
>  [bll.jar:]
>   at 
> org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2055) 
> [bll.jar:]
>
> ...
>
> 2017-01-31 05:46:51,430-05 ERROR 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
> (DefaultQuartzScheduler6) [17e8e3e] EVENT_ID: 
> USER_ADD_VM_TEMPLATE_FAILURE(36), Correlation ID: 17e8e3e, Job ID: 
> 6d664704-97ff-4ff3-8aaa-63389dc51663, Call Stack: null, Custom Event ID: -1, 
> Message: Failed creating Template CirrOS_0.3.4_for_x86_64_glance_template.
> 2017-01-31 05:46:51,431-05 INFO  
> [org.ovirt.engine.core.bll.AddVmTemplateCommand] (DefaultQuartzScheduler6) 
> [17e8e3e] Lock freed to object 'EngineLock:{exclusiveLocks='null', 
> sharedLocks='[----= ACTION_TYPE_FAILED_TEMPLATE_IS_BEING_CREATED>]'}'
> 2017-01-31 05:46:51,435-05 DEBUG 
> [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExceptionTranslator] 
> (DefaultQuartzScheduler6) [17e8e3e] Translating SQLException with SQL state 
> '23503', error code '0', message [ERROR: insert or update on table 
> "disk_vm_element" violates foreign key constraint 
> "fk_disk_vm_element_vm_static"
>   Detail: Key (vm_id)=(7dd54891-477c-4e64-ad47-02afc12ab32d) is not present 
> in table "vm_static".
>   Where: SQL statement "INSERT INTO disk_vm_element (
> disk_id,
> vm_id,
> is_boot,
> pass_discard,
> disk_interface,
> is_using_scsi_reservation)
> VALUES (
> v_disk_id,
> v_vm_id,
> v_is_boot,
> v_pass_discard,
> v_disk_interface,
> v_is_using_scsi_reservation)"
> PL/pgSQL function insertdiskvmelement(uuid,uuid,boolean,boolean,character 
> varying,boolean) line 3 at SQL statement]; SQL was [{call 
> insertdiskvmelement(?, ?, ?, ?, ?, ?)}] for task [CallableStatementCallback]
> 2017-01-31 05:46:51,436-05 INFO  
> [org.ovirt.engine.core.utils.transaction.TransactionSupport] 
> (DefaultQuartzScheduler6) [17e8e3e] transaction rolled back
>
> Y.
>
>
>>
>>
>> Error snippet from log:
>> 2017-01-31 05:54:53,862-0500 ERROR (jsonrpc/5) [storage.TaskManager.Task]
>> (Task='ad8f48fc-de09-4c26-b052-7a036d7651ad') Unexpected error (task:871)
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line
>> 878, in _run
>> return fn(*args, **kargs)
>>   File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 52, in
>> wrapper
>> res = f(*args, **kwargs)
>>   File "/usr/share/vdsm/storage/hsm.py", line 2215, in getAllTasksInfo
>> allTasksInfo = self._pool.getAllTasksInfo()
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
>> line 77, in wrapper
>> raise SecureError("Secured object is not in safe state")
>> SecureError: Se

[ovirt-devel] Update on Domain XML created by the engine

[Arik Hadas]

Hi All,

Yesterday we merged the class LibvirtVmXmlBuilder in ovirt-engine.
It does not include the conversion of all the previously supported VM
properties (e.g., payload devices, OVS stuff, cinder disks - any help would
be appreciated btw) or covers all flows (e.g., run once) yet and there is
surely a place for improvement, but in its current state
LibvirtVmXmlBuilder is able to generate a libvirt's domxml on the engine
side for many typical VMs in our development environment.

If you happen to start working on a change that involves changing something
in the domxml, please do it by modifying this class. Our plan is to
deprecate the VM info map/dictionary soon.

Let me try to motivate you by example:

Let's say that you want to add support for a property of a disk called
'discard'.
In 4.1:
You had to patch VmInfoBuilderImpl to add another element to the VM info
map and then modify VDSM to use it when encoding the Disk device into XML.
Now:
You need to patch LibvirtVmXmlBuilder so the 'discard' property would be
set on the disk device element in the generated XML.

In order to test such a change you can:
(1) apply [1] to send the XML generated by the engine to VDSM
(2) apply [2] to send the XML generated by the engine to libvirt

[1] https://gerrit.ovirt.org/#/c/71088/
[2] https://gerrit.ovirt.org/#/c/65182/

Regards,
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Master is a bit broken - add template fails

[Arik Hadas]

Pushed a fix:
https://gerrit.ovirt.org/#/c/68677/

On Sun, Dec 18, 2016 at 12:08 PM, Eyal Edri  wrote:

> This has been broken from last week, I suggest we revert it ASAP, its also
> been communicated on the devel list from Thursday,
> So I think enough time was given for any possible fix to be merged.
>
> On Sun, Dec 18, 2016 at 12:05 PM, Yaniv Kaul  wrote:
>
>>
>>
>> On Sun, Dec 18, 2016 at 11:55 AM, Jakub Niedermertl 
>> wrote:
>>
>>> It looks like it comes from
>>> https://gerrit.ovirt.org/#/c/68400/5/backend/manager/modules
>>> /bll/src/main/java/org/ovirt/engine/core/bll/utils/VmDeviceU
>>> tils.java@801.
>>>
>>
>> That was my suspicion as well. It has to be fixed promptly or reverted.
>> I'm in favor of reverting, of course.
>> Y.
>>
>>
>>>
>>> On Sat, Dec 17, 2016 at 3:51 PM, Yaniv Kaul  wrote:
>>> > 2016-12-17 09:33:01,766-05 DEBUG
>>> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$
>>> PostgresSimpleJdbcCall]
>>> > (DefaultQuartzScheduler10) [5524d71b] SqlCall for procedure
>>> > [GetVmDeviceByVmId] compiled
>>> > 2016-12-17 09:33:01,771-05 DEBUG
>>> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$
>>> PostgresSimpleJdbcCall]
>>> > (DefaultQuartzScheduler10) [5524d71b] Compiled stored procedure. Call
>>> string
>>> > is [{call getvmde
>>> > vicebyvmidtypeanddevice(?, ?, ?, ?, ?)}]
>>> > 2016-12-17 09:33:01,771-05 DEBUG
>>> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$
>>> PostgresSimpleJdbcCall]
>>> > (DefaultQuartzScheduler10) [5524d71b] SqlCall for procedure
>>> > [GetVmDeviceByVmIdTypeAndDevice] c
>>> > ompiled
>>> > 2016-12-17 09:33:01,841-05 INFO
>>> > [org.ovirt.engine.core.utils.transaction.TransactionSupport]
>>> > (DefaultQuartzScheduler10) [5524d71b] transaction rolled back
>>> > 2016-12-17 09:33:01,842-05 ERROR
>>> > [org.ovirt.engine.core.bll.AddVmTemplateCommand]
>>> (DefaultQuartzScheduler10)
>>> > [5524d71b] Command 'org.ovirt.engine.core.bll.AddVmTemplateCommand'
>>> failed:
>>> > null
>>> > 2016-12-17 09:33:01,842-05 ERROR
>>> > [org.ovirt.engine.core.bll.AddVmTemplateCommand]
>>> (DefaultQuartzScheduler10)
>>> > [5524d71b] Exception: java.lang.NullPointerException
>>> > at
>>> > org.ovirt.engine.core.bll.utils.VmDeviceUtils.getUsbControll
>>> erModel(VmDeviceUtils.java:803)
>>> > [bll.jar:]
>>> > at
>>> > org.ovirt.engine.core.bll.utils.VmDeviceUtils.updateNormalUs
>>> b(VmDeviceUtils.java:786)
>>> > [bll.jar:]
>>> >
>>> >
>>> > ...
>>> >
>>> > 2016-12-17 09:33:01,848-05 DEBUG
>>> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$
>>> PostgresSimpleJdbcCall]
>>> > (DefaultQuartzScheduler10) [5524d71b] Compiled stored procedure. Call
>>> string
>>> > is [{call get_entity_snapshot_by_command_id(?)}]
>>> > 2016-12-17 09:33:01,848-05 DEBUG
>>> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$
>>> PostgresSimpleJdbcCall]
>>> > (DefaultQuartzScheduler10) [5524d71b] SqlCall for procedure
>>> > [get_entity_snapshot_by_command_id] compiled
>>> > 2016-12-17 09:33:01,851-05 INFO
>>> > [org.ovirt.engine.core.bll.AddVmTemplateCommand]
>>> (DefaultQuartzScheduler10)
>>> > [5524d71b] Command [id=67fa2717-f180-4f6a-83c2-f0b8cb9dd082]:
>>> Compensating
>>> > NEW_ENTITY_ID of org.ovirt.engine.core.common.b
>>> usinessentities.VmTemplate;
>>> > snapshot: f908f5f4-b097-4256-90e0-9ef77ccc98c9.
>>> > 2016-12-17 09:33:01,869-05 ERROR
>>> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>> > (DefaultQuartzScheduler10) [5524d71b] Correlation ID: 5524d71b, Job ID:
>>> > ecda173b-b3c1-4b70-8fd8-2be55953187e, Call Stack: null, Custom Event
>>> ID: -1,
>>> > Message: Failed creating Template CirrOS_0.3.4_for_x86_64_glance
>>> _template.
>>> > 2016-12-17 09:33:01,874-05 DEBUG
>>> > [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExc
>>> eptionTranslator]
>>> > (DefaultQuartzScheduler10) [5524d71b] Translating SQLException with SQL
>>> > state '23503', error code '0', message [ERROR: insert or update on
>>> table
>>> > "disk_vm_element" violates foreign key constraint
>>> > "fk_disk_vm_element_vm_static"
>>> >   Detail: Key (vm_id)=(f908f5f4-b097-4256-90e0-9ef77ccc98c9) is not
>>> present
>>> > in table "vm_static".
>>> >   Where: SQL statement "INSERT INTO disk_vm_element (
>>> > disk_id,
>>> > vm_id,
>>> > is_boot,
>>> > pass_discard,
>>> > disk_interface,
>>> > is_using_scsi_reservation)
>>> > VALUES (
>>> > v_disk_id,
>>> > v_vm_id,
>>> > v_is_boot,
>>> > v_pass_discard,
>>> > v_disk_interface,
>>> > v_is_using_scsi_reservation)"
>>> > PL/pgSQL function insertdiskvmelement(uuid,uuid,
>>> boolean,boolean,character
>>> > varying,boolean) line 3 at SQL statement]; SQL was [{call
>>> > insertdiskvmelement(?, ?, ?, ?, ?, ?)}] for task
>>> [CallableStatementCallback]
>>> > 2016-12-17 09:33:01,877-05 INFO
>>> > [org.ovirt.engine.core.utils.transaction.TransactionSupport]
>>> > (DefaultQuartzSche

Re: [ovirt-devel] [VDSM] Correct implementation of virt-sysprep job

[Arik Hadas]

Adam,
Just out of curiosity: when you write "v2v has promised" - what exactly do
you mean? the tool? Richard Jones (the maintainer of virt-v2v)? Shahar and
I that implemented the integration with virt-v2v? I'm not aware of such a
promise by any of these options :)

Anyway, let's say that you were given such a promise by someone and thus
consider that mechanism to be deprecated - it doesn't really matter.
The current implementation doesn't well fit to this flow (it requires
per-volume job, it creates leases that are not needed for template's disks,
...) and with the "next-gen API" with proper support for virt flows not
even being discussed with us (and iiuc also not with the infra team) yet, I
don't understand what do you suggest except for some strong, though
irrelevant, statements.
I suggest loud and clear to reuse (not to add dependencies, not to enhance,
..) an existing mechanism for a very similar flow of virt-v2v that works
well and simple.

Do you "promise" to implement your "next gen API" for 4.1 as an alternative?


On Tue, Dec 6, 2016 at 5:04 PM, Adam Litke  wrote:

> On 05/12/16 11:17 +0200, Arik Hadas wrote:
>
>>
>>
>> On Mon, Dec 5, 2016 at 10:05 AM, Nir Soffer  wrote:
>>
>>On Sun, Dec 4, 2016 at 8:50 PM, Shmuel Melamud 
>> wrote:
>>>
>>> Hi!
>>>
>>> I'm currently working on integration of virt-sysprep into oVirt.
>>>
>>> Usually, if user creates a template from a regular VM, and then
>> creates
>>new VMs from this template, these new VMs inherit all configuration of
>> the
>>original VM, including SSH keys, UDEV rules, MAC addresses, system ID,
>>hostname etc. It is unfortunate, because you cannot have two network
>>devices with the same MAC address in the same network, for example.
>>>
>>> To avoid this, user must clean all machine-specific configuration
>> from
>>the original VM before creating a template from it. You can do this
>>manually, but there is virt-sysprep utility that does this
>> automatically.
>>>
>>> Ideally, virt-sysprep should be seamlessly integrated into template
>>creation process. But the first step is to create a simple button: user
>>selects a VM, clicks the button and oVirt executes virt-sysprep on the
>> VM.
>>>
>>> virt-sysprep works directly on VM's filesystem. It accepts list of
>> all
>>disks of the VM as parameters:
>>>
>>> virt-sysprep -a disk1.img -a disk2.img -a disk3.img
>>>
>>> The architecture is as follows: command on the Engine side runs a
>> job on
>>VDSM side and tracks its success/failure. The job on VDSM side runs
>>virt-sysprep.
>>>
>>> The question is how to implement the job correctly?
>>>
>>> I thought about using storage jobs, but they are designed to work
>> only
>>with a single volume, correct?
>>
>>New storage verbs are volume based. This make it easy to manage
>>them on the engine side, and will allow parallelizing volume operations
>>on single or multiple hosts.
>>
>>A storage volume job is using sanlock lease on the modified volume
>>and volume generation number. If a host running pending jobs becomes
>>non-responsive and cannot be fenced, we can detect the state of
>>the job, fence the job, and start the job on another host.
>>
>>In the SPM task, if a host becomes non-responsive and cannot be
>>fenced, the whole setup is stuck, there is no way to perform any
>>storage operation.
>>  > Is is possible to use them with operation that is performed on
>> multiple
>>volumes?
>>> Or, alternatively, is it possible to use some kind of 'VM jobs' -
>> that
>>work on VM at whole?
>>
>>We can do:
>>
>>1. Add jobs with multiple volumes leases - can make error handling very
>>complex. How do tell a job state if you have multiple leases? which
>>volume generation you use?
>>
>>2. Use volume job using one of the volumes (the boot volume?). This
>> does
>>not protect the other volumes from modification but engine is
>>responsible
>>for this.
>>
>>3. Use new "vm jobs", using a vm lease (should be available this week
>>on master).
>>This protects a vm during sysprep from starting the vm.
>>We still need a generation to detect the job state, I thin

Re: [ovirt-devel] [VDSM] Correct implementation of virt-sysprep job

[Arik Hadas]

On Mon, Dec 5, 2016 at 10:05 AM, Nir Soffer  wrote:

> On Sun, Dec 4, 2016 at 8:50 PM, Shmuel Melamud 
> wrote:
> >
> > Hi!
> >
> > I'm currently working on integration of virt-sysprep into oVirt.
> >
> > Usually, if user creates a template from a regular VM, and then creates
> new VMs from this template, these new VMs inherit all configuration of the
> original VM, including SSH keys, UDEV rules, MAC addresses, system ID,
> hostname etc. It is unfortunate, because you cannot have two network
> devices with the same MAC address in the same network, for example.
> >
> > To avoid this, user must clean all machine-specific configuration from
> the original VM before creating a template from it. You can do this
> manually, but there is virt-sysprep utility that does this automatically.
> >
> > Ideally, virt-sysprep should be seamlessly integrated into template
> creation process. But the first step is to create a simple button: user
> selects a VM, clicks the button and oVirt executes virt-sysprep on the VM.
> >
> > virt-sysprep works directly on VM's filesystem. It accepts list of all
> disks of the VM as parameters:
> >
> > virt-sysprep -a disk1.img -a disk2.img -a disk3.img
> >
> > The architecture is as follows: command on the Engine side runs a job on
> VDSM side and tracks its success/failure. The job on VDSM side runs
> virt-sysprep.
> >
> > The question is how to implement the job correctly?
> >
> > I thought about using storage jobs, but they are designed to work only
> with a single volume, correct?
>
> New storage verbs are volume based. This make it easy to manage
> them on the engine side, and will allow parallelizing volume operations
> on single or multiple hosts.
>
> A storage volume job is using sanlock lease on the modified volume
> and volume generation number. If a host running pending jobs becomes
> non-responsive and cannot be fenced, we can detect the state of
> the job, fence the job, and start the job on another host.
>
> In the SPM task, if a host becomes non-responsive and cannot be
> fenced, the whole setup is stuck, there is no way to perform any
> storage operation.
>
> > Is is possible to use them with operation that is performed on multiple
> volumes?
> > Or, alternatively, is it possible to use some kind of 'VM jobs' - that
> work on VM at whole?
>
> We can do:
>
> 1. Add jobs with multiple volumes leases - can make error handling very
> complex. How do tell a job state if you have multiple leases? which
> volume generation you use?
>
> 2. Use volume job using one of the volumes (the boot volume?). This does
> not protect the other volumes from modification but engine is
> responsible
> for this.
>
> 3. Use new "vm jobs", using a vm lease (should be available this week
> on master).
> This protects a vm during sysprep from starting the vm.
> We still need a generation to detect the job state, I think we can
> use the sanlock
> lease generation for this.
>
> I like the last option since sysprep is much like running a vm.
>
> > How v2v solves this problem?
>
> It does not.
>
> v2v predates storage volume jobs. It does not use volume leases and
> generation
> and does have any way to recover if a host running v2v becomes
> non-responsive
> and cannot be fenced.
>
> It also does not use the jobs framework and does not use a thread pool for
> v2v jobs, so it has no limit on the number of storage operations on a host.
>
>
Right, but let's be fair and present the benefits of v2v-jobs as well:
1. it is the simplest "infrastructure" in terms of LOC
2. it is the most efficient mechanism in terms of interactions between the
engine and VDSM (it doesn't require new verbs/call, the data is attached to
VdsStats; probably the easiest mechanism to convert to events)
3. it is the most efficient implementation in terms of interaction with the
database (no date is persisted into the database, no polling is done)

Currently we have 3 mechanisms to report jobs:
1. VM jobs - that is currently used for live-merge. This requires the VM
entity to exist in VDSM, thus not suitable for virt-sysprep.
2. storage jobs - complicated infrastructure, targeted for recovering from
failures to maintain storage consistency. Many of the things this
infrastructure knows to handle is irrelevant for virt-sysprep flow, and the
fact that virt-sysprep is invoked on VM rather than particular disk makes
it less suitable.
3. V2V jobs - no mechanism is provided to resume failed jobs, no leases, etc

I have some arguments for using V2V-like jobs [1]:
1. creating template from vm is rarely done - if host goes unresponsive or
any other failure is detected we can just remove the template and report
the error
2. the phase of virt-sysprep is, unlike typical storage operation, short -
reducing the risk of failures during the process
3. during the operation the VM is down - by locking the VM/template and its
disks on the engine side, we render leases-like mechanism redundant
4. in the worst case - the disk 

Re: [ovirt-devel] Experimental Flow for Master Fails to Run a VM

[Arik Hadas]

Yaniv will try to lower the cluster level used in the system-tests to 4.0 -
this is supposed to solve the issue.
If it won't help (we will know it in about an hour), we'll add a db-script
that changes the rng device of the blank template only.

On Sun, Dec 4, 2016 at 3:34 PM, Eyal Edri  wrote:

> FYI,
>
> I opened a bug [1] to track this issue since I don't see any attempts to
> resolve the issue on the thread, hopefully a bug will get more attention.
> Opened on VDSM since we see the libvirt error there, feel free to move
> product/team.
>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1401303
>
> On Sun, Dec 4, 2016 at 1:23 PM, Eyal Edri  wrote:
>
>> Not sure if relevant, but Juan posted a fix for SDK4 last time it
>> happened ( but different failure on log-collector ):
>>
>> https://gerrit.ovirt.org/#/c/67213/
>>
>> * Added `urandom` to the `RngSource` enumerated type.
>>
>> On Sun, Dec 4, 2016 at 9:17 AM, Eyal Edri  wrote:
>>
>>> And its still failing from Friday,
>>> Since we don't have official Centos 7.3 repos yet ( hopefully we'll have
>>> it this week, but as of this moment its not published yet ) , we have to
>>> either revert the offending patch
>>> or send a quick fix.
>>>
>>> Right now all experimental flows for master are not working and nightly
>>> rpms are not refreshed with new RPMs.
>>>
>>>
>>>
>>> On Fri, Dec 2, 2016 at 9:41 PM, Yaniv Kaul  wrote:
>>>


 On Dec 2, 2016 2:11 PM, "Anton Marchukov"  wrote:

 Hello Martin.

 Do by outdated you mean the old libvirt? If so that is that livirt
 available in CentOS 7.2? There is no 7.3 yet.


 Right, this is the issue.
 Y.


 Anton.

 On Fri, Dec 2, 2016 at 1:07 PM, Martin Polednik 
 wrote:

> On 02/12/16 10:55 +0100, Anton Marchukov wrote:
>
>> Hello All.
>>
>> Engine log can be viewed here:
>>
>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma
>> ster/3838/artifact/exported-artifacts/basic_suite_master.sh-
>> el7/exported-artifacts/test_logs/basic-suite-master/post-004
>> _basic_sanity.py/lago-basic-suite-master-engine/_var_log_ovi
>> rt-engine/engine.log
>>
>> I see the following exception there:
>>
>> 2016-12-02 04:29:24,030-05 DEBUG
>> [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker]
>> (ResponseWorker) [83b6b5d] Message received: {"jsonrpc": "2.0", "id":
>> "ec254aad-441b-47e7-a644-aebddcc1d62c", "result": true}
>> 2016-12-02 04:29:24,030-05 ERROR
>> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker)
>> [83b6b5d] Not able to update response for
>> "ec254aad-441b-47e7-a644-aebddcc1d62c"
>> 2016-12-02 04:29:24,041-05 DEBUG
>> [org.ovirt.engine.core.utils.timer.FixedDelayJobListener]
>> (DefaultQuartzScheduler3) [47a31d72] Rescheduling
>> DEFAULT.org.ovirt.engine.core.bll.gluster.GlusterSyncJob.ref
>> reshLightWeightData#-9223372036854775775
>> as there is no unfired trigger.
>> 2016-12-02 04:29:24,024-05 DEBUG
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>> task-12) [d932871a-af4f-4fc9-9ee5-f7a0126a7b85] Exception:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
>> VDSGenericException: VDSNetworkException: Timeout during xml-rpc call
>> at org.ovirt.engine.core.vdsbroke
>> r.vdsbroker.FutureVDSCommand.get(FutureVDSCommand.java:73)
>> [vdsbroker.jar:]
>>
>> ...
>>
>> 2016-12-02 04:29:24,042-05 ERROR
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>> task-12) [d932871a-af4f-4fc9-9ee5-f7a0126a7b85] Timeout waiting for
>> VDSM response: Internal timeout occured
>> 2016-12-02 04:29:24,044-05 DEBUG
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
>> (default task-12) [d932871a-af4f-4fc9-9ee5-f7a0126a7b85] START,
>> GetCapabilitiesVDSCommand(HostName = lago-basic-suite-master-host0,
>> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
>> hostId='5eb7019e-28a3-4f93-9188-685b6c64a2f5',
>> vds='Host[lago-basic-suite-master-host0,5eb7019e-28a3-4f93-9
>> 188-685b6c64a2f5]'}),
>> log id: 58f448b8
>> 2016-12-02 04:29:24,044-05 DEBUG
>> [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (default
>> task-12) [d932871a-af4f-4fc9-9ee5-f7a0126a7b85] SEND
>> destination:jms.topic.vdsm_requests
>> reply-to:jms.topic.vdsm_responses
>> content-length:105
>>
>>
>> Please note that this runs on localhost with local bridge. So it is
>> not
>> likely to be network itself.
>>
>
> The main issue I see is that the VM run command has actually failed
> due to libvirt no accepting /dev/urandom as RNG source[1]. This was
> done as engine patch and according to git log, posted around Mon Nov
> 28. Also adding Jakub - this should either not happen from engine's
> point of view or t

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> On Tue, Nov 29, 2016 at 4:30 PM, Arik Hadas  wrote:
> >
> >
> > - Original Message -
> >> On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas  wrote:
> >> >
> >> >
> >> > - Original Message -
> >> >> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
> >> >> > Hi All,
> >> >> >
> >> >> > We are working on something that is expected to have a big impact,
> >> >> > hence
> >> >> > this heads-up.
> >> >> > First, we want you to be aware of this change and provide your
> >> >> > feedback
> >> >> > to
> >> >> > make it as good as possible.
> >> >> > Second, until the proposed mechanism is fully merged there will be a
> >> >> > chase
> >> >> > to cover all features unless new features are also implemented with
> >> >> > the
> >> >> > new mechanism. So please, if you are working on something that
> >> >> > adds/changes something in the Libvirt's domain xml, do it with this
> >> >> > new
> >> >> > mechanism as well (first version would be merged soon).
> >> >> >
> >> >> > * Goal
> >> >> > Creating Libvirt XML in the engine rather than in VDSM.
> >> >> > ** Today's flow
> >> >> > Engine: VM business entity -> VM properties map
> >> >> > VDSM:   VM properties map  -> Libvirt XML
> >> >> > ** Desired flow
> >> >> > Engine: VM business entity -> Libvirt XML
> >> >> >
> >> >> > * Potential Benefits
> >> >> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> >> >> > mistakes in the process.
> >> >> > 2. Reduce the amount of code in VDSM.
> >> >> > 3. Make VM related changes easier - today many of these changes need
> >> >> > to
> >> >> > be
> >> >> > reviewed in 2 projects, this will eliminate the one that tends to
> >> >> > take
> >> >> > longer.
> >> >> > 4. Prevent shortcuts in the form of VDSM-only changes that should be
> >> >> > better
> >> >> > reflected in the engine.
> >> >> > 5. Not to re-generate the XML on each rerun attempt of VM
> >> >> > run/migration.
> >> >> > 6. Future - not to re-generate the XML on each attempt to auto-start
> >> >> > HA
> >> >> > VM
> >> >> > when using vm-leases (need to make sure we're using the up-to-date VM
> >> >> > configuration though).
> >> >> > 7. We already found improvements and cleanups that could be made
> >> >> > while
> >> >> > touching this area (e.g., remove the boot order from devices in the
> >> >> > database).
> >> >> >
> >> >> > * Challenges
> >> >> > 1. Not to move host-specific information to the engine. For example,
> >> >> > path
> >> >> > to storage domain or sockets of channels.
> >> >> >The solution is to use place-holders that will be replaced by
> >> >> >VDSM.
> >> >> > 2. Backward compatibility.
> >> >> > 3. The more challenging part is the other direction - that will be
> >> >> > the
> >> >> > next
> >> >> > phase.
> >> >> >
> >> >> > * Status
> >> >> > As a first step, we began with producing the Libvirt XML in the
> >> >> > engine
> >> >> > by
> >> >> > converting the VM properties map to XML in the engine [1]
> >> >> > And using the XML that is received as an input in VDSM [2]
> >> >> >
> >> >> >
> >> >> > [1] https://gerrit.ovirt.org/#/c/64473/
> >> >> > [2] https://gerrit.ovirt.org/#/c/65182/
> >> >>
> >> >> I should start by saying that I love libvirt's domxml standard. Unlike
> >> >> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
> >> >> you are suggesting a positive step.
> >> >>
> >> >> However, Engine is much more complex than Vdsm. It is also 

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas  wrote:
> >
> >
> > - Original Message -
> >> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
> >> > Hi All,
> >> >
> >> > We are working on something that is expected to have a big impact, hence
> >> > this heads-up.
> >> > First, we want you to be aware of this change and provide your feedback
> >> > to
> >> > make it as good as possible.
> >> > Second, until the proposed mechanism is fully merged there will be a
> >> > chase
> >> > to cover all features unless new features are also implemented with the
> >> > new mechanism. So please, if you are working on something that
> >> > adds/changes something in the Libvirt's domain xml, do it with this new
> >> > mechanism as well (first version would be merged soon).
> >> >
> >> > * Goal
> >> > Creating Libvirt XML in the engine rather than in VDSM.
> >> > ** Today's flow
> >> > Engine: VM business entity -> VM properties map
> >> > VDSM:   VM properties map  -> Libvirt XML
> >> > ** Desired flow
> >> > Engine: VM business entity -> Libvirt XML
> >> >
> >> > * Potential Benefits
> >> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> >> > mistakes in the process.
> >> > 2. Reduce the amount of code in VDSM.
> >> > 3. Make VM related changes easier - today many of these changes need to
> >> > be
> >> > reviewed in 2 projects, this will eliminate the one that tends to take
> >> > longer.
> >> > 4. Prevent shortcuts in the form of VDSM-only changes that should be
> >> > better
> >> > reflected in the engine.
> >> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> >> > 6. Future - not to re-generate the XML on each attempt to auto-start HA
> >> > VM
> >> > when using vm-leases (need to make sure we're using the up-to-date VM
> >> > configuration though).
> >> > 7. We already found improvements and cleanups that could be made while
> >> > touching this area (e.g., remove the boot order from devices in the
> >> > database).
> >> >
> >> > * Challenges
> >> > 1. Not to move host-specific information to the engine. For example,
> >> > path
> >> > to storage domain or sockets of channels.
> >> >The solution is to use place-holders that will be replaced by VDSM.
> >> > 2. Backward compatibility.
> >> > 3. The more challenging part is the other direction - that will be the
> >> > next
> >> > phase.
> >> >
> >> > * Status
> >> > As a first step, we began with producing the Libvirt XML in the engine
> >> > by
> >> > converting the VM properties map to XML in the engine [1]
> >> > And using the XML that is received as an input in VDSM [2]
> >> >
> >> >
> >> > [1] https://gerrit.ovirt.org/#/c/64473/
> >> > [2] https://gerrit.ovirt.org/#/c/65182/
> >>
> >> I should start by saying that I love libvirt's domxml standard. Unlike
> >> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
> >> you are suggesting a positive step.
> >>
> >> However, Engine is much more complex than Vdsm. It is also our
> >> single-point-of-failure, and where CPU is the most scarce. I am worried
> >> that in the foreseeable future it would only make Engine bigger, without
> >> reducing the size and complexity of Vdsm.
> >>
> >> Before taking this move, we must map what Vdsm does, because that logic
> >> would have to be copied into Engine. Few things pop up to mind:
> >>
> >> - pci addresses. would Vdsm report back the libvirt-assigned addresses
> >>   in XML format, or would it keep parsing them?
> >
> > Ideally, VDSM will report back the devices in XML format.
> > The engine will then add the unmanaged devices and update the pci
> > addresses.
> > Need to put some more thoughts into this, though.
> >
> >>
> >> - hot plug. Device xml should be generated by Engine, much like in the
> >>   vm cteate flow
> >
> > Good point, I didn't think of hot plugs - right, they could be changed as
> > well later on.
> >
> >>
> >>

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
> > Hi All,
> > 
> > We are working on something that is expected to have a big impact, hence
> > this heads-up.
> > First, we want you to be aware of this change and provide your feedback to
> > make it as good as possible.
> > Second, until the proposed mechanism is fully merged there will be a chase
> > to cover all features unless new features are also implemented with the
> > new mechanism. So please, if you are working on something that
> > adds/changes something in the Libvirt's domain xml, do it with this new
> > mechanism as well (first version would be merged soon).
> > 
> > * Goal
> > Creating Libvirt XML in the engine rather than in VDSM.
> > ** Today's flow
> > Engine: VM business entity -> VM properties map
> > VDSM:   VM properties map  -> Libvirt XML
> > ** Desired flow
> > Engine: VM business entity -> Libvirt XML
> > 
> > * Potential Benefits
> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> > mistakes in the process.
> > 2. Reduce the amount of code in VDSM.
> > 3. Make VM related changes easier - today many of these changes need to be
> > reviewed in 2 projects, this will eliminate the one that tends to take
> > longer.
> > 4. Prevent shortcuts in the form of VDSM-only changes that should be better
> > reflected in the engine.
> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> > 6. Future - not to re-generate the XML on each attempt to auto-start HA VM
> > when using vm-leases (need to make sure we're using the up-to-date VM
> > configuration though).
> > 7. We already found improvements and cleanups that could be made while
> > touching this area (e.g., remove the boot order from devices in the
> > database).
> > 
> > * Challenges
> > 1. Not to move host-specific information to the engine. For example, path
> > to storage domain or sockets of channels.
> >The solution is to use place-holders that will be replaced by VDSM.
> > 2. Backward compatibility.
> > 3. The more challenging part is the other direction - that will be the next
> > phase.
> > 
> > * Status
> > As a first step, we began with producing the Libvirt XML in the engine by
> > converting the VM properties map to XML in the engine [1]
> > And using the XML that is received as an input in VDSM [2]
> > 
> > 
> > [1] https://gerrit.ovirt.org/#/c/64473/
> > [2] https://gerrit.ovirt.org/#/c/65182/
> 
> I should start by saying that I love libvirt's domxml standard. Unlike
> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
> you are suggesting a positive step.
> 
> However, Engine is much more complex than Vdsm. It is also our
> single-point-of-failure, and where CPU is the most scarce. I am worried
> that in the foreseeable future it would only make Engine bigger, without
> reducing the size and complexity of Vdsm.
> 
> Before taking this move, we must map what Vdsm does, because that logic
> would have to be copied into Engine. Few things pop up to mind:
> 
> - pci addresses. would Vdsm report back the libvirt-assigned addresses
>   in XML format, or would it keep parsing them?

Ideally, VDSM will report back the devices in XML format.
The engine will then add the unmanaged devices and update the pci addresses.
Need to put some more thoughts into this, though.

> 
> - hot plug. Device xml should be generated by Engine, much like in the
>   vm cteate flow

Good point, I didn't think of hot plugs - right, they could be changed as well 
later on.

> 
> - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
>   that is connected no-where. Engine would need to care about what was
>   up until now a vdsm-side implementation detail.

Right, I almost finished to copy the creation of the network interfaces to the 
engine.
This knowledge that you refer to will only be in the module that creates the 
XML, it doesn't seem to be much of an issue.

> 
> - storage path. this was mentioned above, but actually, the paths are
>   the same on all hosts. We inteded to have an abstraction layer there,
>   but we never ever used it. All volumes sit under
>   /rhev/data-center/poolID/domainID/imageID/volumeID
>   Basically, Engine can hard-code this in the domxml, and no one would
>   notice.

But I see that LUN and cinder disks are represented differently (not as PDIV) - 
I'll check this.

> 
> - OvS. Recently, we have changed how VMs can be connected to their
>   network. It is possible (albeit n

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> On 23 November 2016 at 19:27, Yaniv Dary  wrote:
> 
> > Does this mean we will be able to move the hooks to the engine side at
> > some point? That would be much better than needing it on VDSM.
> >
> > Yaniv Dary
> > Technical Product Manager
> > Red Hat Israel Ltd.
> > 34 Jerusalem Road
> > Building A, 4th floor
> > Ra'anana, Israel 4350109
> >
> > Tel : +972 (9) 7692306
> > 8272306
> > Email: yd...@redhat.com
> > IRC : ydary
> >
> >
> > On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas  wrote:
> >
> >>
> >>
> >> - Original Message -
> >> > Hi,
> >> >
> >> > Arik there is also custom metadata section for MOM and QoS in the
> >> > libvirt domain XML and those values are now created as XML and later
> >> > manipulated using libvirt API. VDSM will still need to know at least
> >> > some basic stuff about the XML to be able to process it correctly (the
> >> > metadata descriptor and the structure for example).
> >>
> >> Besides the parsing of the devices that we would love to get rid of, I
> >> believe the parsing of Libvirt XML done by VDSM would remain as is.
> >>
> >> >
> >> > Will the engine assume anything about the XML post-migration (for
> >> > example to speed-up restarts for highly available VMs)? Because the
> >> > hooks can change it significantly. Start hooks will be used anyway,
> >> > but migration hook changes might not be reflected during the restart.
> >> >
> >>
> >> Not sure that I fully understand.
> >> You have a VM with configuration conf1 running on host1. While being
> >> migrated to host2 the configuration is changed to conf2.
> >> Do you suggest that the next time the VM restarts it would be created
> >> based on conf2?
> >>
> >> Changes in the devices would be reflected on the next run (assuming the
> >> engine got the updated devices) - as today.
> >> Other than the devices, no configuration that is used on VM creation
> >> (static data) is read back by the engine so if it is changed it won't be
> >> reflected.
> >> And at the moment we have no plan to change the content of the data that
> >> is passed between the engine and VDSM, only its representation.
> >>
> >> Btw, I'm taking back my statement about migration:
> >> "5. Not to re-generate the XML on each rerun attempt of VM run/migration."
> >> VM migration flow is irrelevant since the engine doesn't pass the VM
> >> configuration. It will remain as-is.
> >>
> >> >
> >> > Martin
> >> >
> >> >
> >> >
> >> > On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas  wrote:
> >> > > Hi All,
> >> > >
> >> > > We are working on something that is expected to have a big impact,
> >> hence
> >> > > this heads-up.
> >> > > First, we want you to be aware of this change and provide your
> >> feedback to
> >> > > make it as good as possible.
> >> > > Second, until the proposed mechanism is fully merged there will be a
> >> chase
> >> > > to cover all features unless new features are also implemented with
> >> the
> >> > > new mechanism. So please, if you are working on something that
> >> > > adds/changes something in the Libvirt's domain xml, do it with this
> >> new
> >> > > mechanism as well (first version would be merged soon).
> >> > >
> >> > > * Goal
> >> > > Creating Libvirt XML in the engine rather than in VDSM.
> >> > > ** Today's flow
> >> > > Engine: VM business entity -> VM properties map
> >> > > VDSM:   VM properties map  -> Libvirt XML
> >> > > ** Desired flow
> >> > > Engine: VM business entity -> Libvirt XML
> >> > >
> >> > > * Potential Benefits
> >> > > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> >> > > mistakes in the process.
> >> > > 2. Reduce the amount of code in VDSM.
> >> > > 3. Make VM related changes easier - today many of these changes need
> >> to be
> >> > > reviewed in 2 projects, this will eliminate the one that tends to take
> >> > > longer.
> >> > > 4. P

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> Does this mean we will be able to move the hooks to the engine side at some
> point? That would be much better than needing it on VDSM.

Well, it sounds feasible to provide a hooking mechanism that enables to alter 
the domain XML in the engine, at some point.
I'm not sure though that it could actually replace the hooks in VDSM since:
1. The XML won't be complete (host-specific configuration will be missing).
2. If the user wants to do some operation on the host with the hook, he'll need 
to send a request from the engine to the host, which is more complicated.

> 
> Yaniv Dary
> Technical Product Manager
> Red Hat Israel Ltd.
> 34 Jerusalem Road
> Building A, 4th floor
> Ra'anana, Israel 4350109
> 
> Tel : +972 (9) 7692306
> 8272306
> Email: yd...@redhat.com
> IRC : ydary
> 
> 
> On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas  wrote:
> 
> >
> >
> > - Original Message -
> > > Hi,
> > >
> > > Arik there is also custom metadata section for MOM and QoS in the
> > > libvirt domain XML and those values are now created as XML and later
> > > manipulated using libvirt API. VDSM will still need to know at least
> > > some basic stuff about the XML to be able to process it correctly (the
> > > metadata descriptor and the structure for example).
> >
> > Besides the parsing of the devices that we would love to get rid of, I
> > believe the parsing of Libvirt XML done by VDSM would remain as is.
> >
> > >
> > > Will the engine assume anything about the XML post-migration (for
> > > example to speed-up restarts for highly available VMs)? Because the
> > > hooks can change it significantly. Start hooks will be used anyway,
> > > but migration hook changes might not be reflected during the restart.
> > >
> >
> > Not sure that I fully understand.
> > You have a VM with configuration conf1 running on host1. While being
> > migrated to host2 the configuration is changed to conf2.
> > Do you suggest that the next time the VM restarts it would be created
> > based on conf2?
> >
> > Changes in the devices would be reflected on the next run (assuming the
> > engine got the updated devices) - as today.
> > Other than the devices, no configuration that is used on VM creation
> > (static data) is read back by the engine so if it is changed it won't be
> > reflected.
> > And at the moment we have no plan to change the content of the data that
> > is passed between the engine and VDSM, only its representation.
> >
> > Btw, I'm taking back my statement about migration:
> > "5. Not to re-generate the XML on each rerun attempt of VM run/migration."
> > VM migration flow is irrelevant since the engine doesn't pass the VM
> > configuration. It will remain as-is.
> >
> > >
> > > Martin
> > >
> > >
> > >
> > > On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas  wrote:
> > > > Hi All,
> > > >
> > > > We are working on something that is expected to have a big impact,
> > hence
> > > > this heads-up.
> > > > First, we want you to be aware of this change and provide your
> > feedback to
> > > > make it as good as possible.
> > > > Second, until the proposed mechanism is fully merged there will be a
> > chase
> > > > to cover all features unless new features are also implemented with the
> > > > new mechanism. So please, if you are working on something that
> > > > adds/changes something in the Libvirt's domain xml, do it with this new
> > > > mechanism as well (first version would be merged soon).
> > > >
> > > > * Goal
> > > > Creating Libvirt XML in the engine rather than in VDSM.
> > > > ** Today's flow
> > > > Engine: VM business entity -> VM properties map
> > > > VDSM:   VM properties map  -> Libvirt XML
> > > > ** Desired flow
> > > > Engine: VM business entity -> Libvirt XML
> > > >
> > > > * Potential Benefits
> > > > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> > > > mistakes in the process.
> > > > 2. Reduce the amount of code in VDSM.
> > > > 3. Make VM related changes easier - today many of these changes need
> > to be
> > > > reviewed in 2 projects, this will eliminate the one that tends to take
> > > > longer.
> > > > 4. Prevent shortcuts in the form of VDSM-only chan

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> Hi,
> 
> Arik there is also custom metadata section for MOM and QoS in the
> libvirt domain XML and those values are now created as XML and later
> manipulated using libvirt API. VDSM will still need to know at least
> some basic stuff about the XML to be able to process it correctly (the
> metadata descriptor and the structure for example).

Besides the parsing of the devices that we would love to get rid of, I believe 
the parsing of Libvirt XML done by VDSM would remain as is.

> 
> Will the engine assume anything about the XML post-migration (for
> example to speed-up restarts for highly available VMs)? Because the
> hooks can change it significantly. Start hooks will be used anyway,
> but migration hook changes might not be reflected during the restart.
> 

Not sure that I fully understand.
You have a VM with configuration conf1 running on host1. While being migrated 
to host2 the configuration is changed to conf2.
Do you suggest that the next time the VM restarts it would be created based on 
conf2?

Changes in the devices would be reflected on the next run (assuming the engine 
got the updated devices) - as today.
Other than the devices, no configuration that is used on VM creation (static 
data) is read back by the engine so if it is changed it won't be reflected.
And at the moment we have no plan to change the content of the data that is 
passed between the engine and VDSM, only its representation.

Btw, I'm taking back my statement about migration:
"5. Not to re-generate the XML on each rerun attempt of VM run/migration."
VM migration flow is irrelevant since the engine doesn't pass the VM 
configuration. It will remain as-is.

> 
> Martin
> 
> 
> 
> On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas  wrote:
> > Hi All,
> >
> > We are working on something that is expected to have a big impact, hence
> > this heads-up.
> > First, we want you to be aware of this change and provide your feedback to
> > make it as good as possible.
> > Second, until the proposed mechanism is fully merged there will be a chase
> > to cover all features unless new features are also implemented with the
> > new mechanism. So please, if you are working on something that
> > adds/changes something in the Libvirt's domain xml, do it with this new
> > mechanism as well (first version would be merged soon).
> >
> > * Goal
> > Creating Libvirt XML in the engine rather than in VDSM.
> > ** Today's flow
> > Engine: VM business entity -> VM properties map
> > VDSM:   VM properties map  -> Libvirt XML
> > ** Desired flow
> > Engine: VM business entity -> Libvirt XML
> >
> > * Potential Benefits
> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> > mistakes in the process.
> > 2. Reduce the amount of code in VDSM.
> > 3. Make VM related changes easier - today many of these changes need to be
> > reviewed in 2 projects, this will eliminate the one that tends to take
> > longer.
> > 4. Prevent shortcuts in the form of VDSM-only changes that should be better
> > reflected in the engine.
> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> > 6. Future - not to re-generate the XML on each attempt to auto-start HA VM
> > when using vm-leases (need to make sure we're using the up-to-date VM
> > configuration though).
> > 7. We already found improvements and cleanups that could be made while
> > touching this area (e.g., remove the boot order from devices in the
> > database).
> >
> > * Challenges
> > 1. Not to move host-specific information to the engine. For example, path
> > to storage domain or sockets of channels.
> >The solution is to use place-holders that will be replaced by VDSM.
> > 2. Backward compatibility.
> > 3. The more challenging part is the other direction - that will be the next
> > phase.
> >
> > * Status
> > As a first step, we began with producing the Libvirt XML in the engine by
> > converting the VM properties map to XML in the engine [1]
> > And using the XML that is received as an input in VDSM [2]
> >
> >
> > [1] https://gerrit.ovirt.org/#/c/64473/
> > [2] https://gerrit.ovirt.org/#/c/65182/
> >
> > Regards,
> > Arik
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> Hi,
> 
> very first question which comes to mind
> is: will vdsm hooks which alter the libvirt xml
> still work?
> 
> I believe this is a much used feature
> and if it would not work anymore out of the box
> this would be a huge pain in upgrading existing systems.

Yes, the hooks will still work. Please look at my response to didi regarding 
VDSM hooks.

> 
> furthermore I don't really see the benfits when you generate
> the xml in engine:
> 
> you still need to alter it in vdsm as you write, to fill in host
> specific things like storagepaths anyway, meaning you have to maintain
> twice the codebase _and_ coordinate between the two of them, so nothing
> goes wrong if one project changes something in the xml.

The removal of the code that is being executed before the hooks are called 
which creates the XML from the VM properties map would justify the effort.
Once VDSM will support only the new API (of getting Libvirt XML as an input) 
this code could be removed.

> 
> but this are just my 2 cent :-)
> 
> 
> --
> Mit freundlichen Grüßen / Regards
> 
> Sven Kieske
> 
> Systemadministrator
> Mittwald CM Service GmbH & Co. KG
> Königsberger Straße 6
> 32339 Espelkamp
> T: +495772 293100
> F: +495772 29
> https://www.mittwald.de
> Geschäftsführer: Robert Meyer
> St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
> Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
> 
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

- Original Message -
> On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas  wrote:
> > Hi All,
> >
> > We are working on something that is expected to have a big impact, hence
> > this heads-up.
> > First, we want you to be aware of this change and provide your feedback to
> > make it as good as possible.
> > Second, until the proposed mechanism is fully merged there will be a chase
> > to cover all features unless new features are also implemented with the
> > new mechanism. So please, if you are working on something that
> > adds/changes something in the Libvirt's domain xml, do it with this new
> > mechanism as well (first version would be merged soon).
> >
> > * Goal
> > Creating Libvirt XML in the engine rather than in VDSM.
> > ** Today's flow
> > Engine: VM business entity -> VM properties map
> > VDSM:   VM properties map  -> Libvirt XML
> > ** Desired flow
> > Engine: VM business entity -> Libvirt XML
> >
> > * Potential Benefits
> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> > mistakes in the process.
> > 2. Reduce the amount of code in VDSM.
> > 3. Make VM related changes easier - today many of these changes need to be
> > reviewed in 2 projects, this will eliminate the one that tends to take
> > longer.
> > 4. Prevent shortcuts in the form of VDSM-only changes that should be better
> > reflected in the engine.
> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> > 6. Future - not to re-generate the XML on each attempt to auto-start HA VM
> > when using vm-leases (need to make sure we're using the up-to-date VM
> > configuration though).
> > 7. We already found improvements and cleanups that could be made while
> > touching this area (e.g., remove the boot order from devices in the
> > database).
> >
> > * Challenges
> > 1. Not to move host-specific information to the engine. For example, path
> > to storage domain or sockets of channels.
> >The solution is to use place-holders that will be replaced by VDSM.
> > 2. Backward compatibility.
> > 3. The more challenging part is the other direction - that will be the next
> > phase.
> >
> > * Status
> > As a first step, we began with producing the Libvirt XML in the engine by
> > converting the VM properties map to XML in the engine [1]
> > And using the XML that is received as an input in VDSM [2]
> 
> Thanks for the heads up!
> 
> How does this, if at all, affect:
> 1. vdsm hooks

VDSM hooks will remain unaffected - they will get the same XML.
The only difference is that this XML will be one that was produced by the 
engine after the replacement of its place-holders rather than one that is 
produced by VDSM from the properties map.

> 2. The interaction of engine, hosted-engine HA agent, engine vm
> configuration saved in shared storage, etc.? Will this require a
> change in ha-agent and the saved configuration?

In the foreseeable future VDSM should support creating the VM from VM 
properties map as well - so legacy code will keep working, including the 
creation of the hosted-engine.
However, at some point the creation of the hosted-engine would also need to 
change I assume.
The rest, other than the creation flow, will also remain unaffected.

> 
> Best,
> 
> >
> >
> > [1] https://gerrit.ovirt.org/#/c/64473/
> > [2] https://gerrit.ovirt.org/#/c/65182/
> >
> > Regards,
> > Arik
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> 
> --
> Didi
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ ERROR ] schema.sh: Please fix numbering to interval 04010491 to 04010500 and run the upgrade script.

[Arik Hadas]

it should be fixed now (by https://gerrit.ovirt.org/#/c/67210/)

- Original Message -
> engine master CI is failing on $subject, please fix or revert as soon as
> possible, thanks.
> 
> --
> Sandro Bonazzola
> Better technology. Faster innovation. Powered by community collaboration.
> See how it works at redhat.com
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

Hi All,

We are working on something that is expected to have a big impact, hence this 
heads-up.
First, we want you to be aware of this change and provide your feedback to make 
it as good as possible.
Second, until the proposed mechanism is fully merged there will be a chase to 
cover all features unless new features are also implemented with the new 
mechanism. So please, if you are working on something that adds/changes 
something in the Libvirt's domain xml, do it with this new mechanism as well 
(first version would be merged soon).

* Goal
Creating Libvirt XML in the engine rather than in VDSM.
** Today's flow
Engine: VM business entity -> VM properties map
VDSM:   VM properties map  -> Libvirt XML
** Desired flow
Engine: VM business entity -> Libvirt XML

* Potential Benefits
1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes 
in the process.
2. Reduce the amount of code in VDSM.
3. Make VM related changes easier - today many of these changes need to be 
reviewed in 2 projects, this will eliminate the one that tends to take longer.
4. Prevent shortcuts in the form of VDSM-only changes that should be better 
reflected in the engine.
5. Not to re-generate the XML on each rerun attempt of VM run/migration.
6. Future - not to re-generate the XML on each attempt to auto-start HA VM when 
using vm-leases (need to make sure we're using the up-to-date VM configuration 
though).
7. We already found improvements and cleanups that could be made while touching 
this area (e.g., remove the boot order from devices in the database).

* Challenges
1. Not to move host-specific information to the engine. For example, path to 
storage domain or sockets of channels.
   The solution is to use place-holders that will be replaced by VDSM.
2. Backward compatibility.
3. The more challenging part is the other direction - that will be the next 
phase.

* Status
As a first step, we began with producing the Libvirt XML in the engine by 
converting the VM properties map to XML in the engine [1]
And using the XML that is received as an input in VDSM [2]


[1] https://gerrit.ovirt.org/#/c/64473/
[2] https://gerrit.ovirt.org/#/c/65182/

Regards,
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Post: External DSL for API Specification in oVirt

[Arik Hadas]

- Original Message -
> Hi Arik,
> 
> nice article. I do agree that external DSLs are useful, but I have

Thanks!

> (long-standing) objection to inventing new ones, especially in an area
> of SDKs. I still feel we should have adopted something that is (at
> least close to) industry standard - Swagger [1] or maybe Blueprint
> [2]. We would get all the tooling and SDK generators "for free".
> Inventing a new DSL that only we know about means we have to maintain
> all the tools ourselves.

Right, I do not claim that the proposed external DSL is the ideal
language, others including existing ones can (maybe even better) fit.
I'm not familiar with Swagger and Blueprint so I cannot address them
specifically, but I guess the general trade-off will hold: by using
languages that intend at being reusable in multiple projects you
can leverage their implementation and tools that are also typically
more mature but would typically lack project-specific things like
the ability to define a field with predefined values as our status of
documentation (and then the question to be asked is if we actually
need such project-specific things).

In a paper that is currently in review I argue that chances that
ordinary DSL would be reusable as third-party language in other
projects are significantly higher than those of the reusability of a
domain-specific aspect language. Of course, if that is the case then
a third-party DSL should be considered in light of this trade-off.

> 
> I agree with the general points though, it is very nice when the
> boilerplate is inferred from the sources. It reduces overhead and the
> chance of (coding) bugs.
> 
> On a side note:
> 
> The separation of the API definition jar from the rest of the engine
> source code was a good move. But on the other hand, the repository
> separation of the definition from the engine implementation was not so
> great :( We lost atomicity and easy revert capabilities (for example
> when the API definition version requires two new features, but only
> one is implemented till deadline) and increased the overhead.

> 
> 
> [1] http://swagger.io
> [2] https://apiblueprint.org/
> 
> On Tue, Oct 25, 2016 at 8:07 AM, Arik Hadas  wrote:
> > Hi All,
> >
> > Last month I attended Juan's session on the recent changes in RHV API.
> > I wrote a post on how I believe it can be enhanced by using an external
> > domain-specific language rather than the internal domain-specific language
> > that is used today:
> > http://ahadas.github.io/dsl-for-api-spec-in-ovirt/
> >
> > Your feedback is welcomed,
> > Arik
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Post: External DSL for API Specification in oVirt

[Arik Hadas]

Hi All,

Last month I attended Juan's session on the recent changes in RHV API.
I wrote a post on how I believe it can be enhanced by using an external 
domain-specific language rather than the internal domain-specific language that 
is used today:
http://ahadas.github.io/dsl-for-api-spec-in-ovirt/

Your feedback is welcomed,
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Proposing Martin Sivak as a Backend Maintainer for SLA & Scheduling

[Arik Hadas]

- Original Message -
> 
> 
> On Mon, Sep 19, 2016 at 3:07 PM, Roy Golan < rgo...@redhat.com > wrote:
> 
> 
> 
> 
> 
> On 19 September 2016 at 15:12, Doron Fediuck < dfedi...@redhat.com > wrote:
> 
> 
> 
> Hi All,
> 
> Martin Sivak has been working on the oVirt engine since June 2013.
> His main focus over the years was around MoM, external scheduler,
> hosted-engine, and
> related engine code.
> In the past year Martin took over many of the oVirt scheduler related work
> and did a lot of improvements there. In total Martin has around 170 engine
> patches already merged.
> Within these you will find improving the scheduler's notifications and error
> handling, rewriting the pending resource mechanism which resolved many
> issues, VM affinity, affinity labels and multiple improvements for our
> scheduling policies and hosted engine.
> I would like to thank Martin for his great contribution and hope for many
> more patches to come :)
> 
> I would also like to propose Martin as an engine maintainer for SLA and
> Scheduling.
> Thanks, Doron
> 
> +1 Actually +2 :)
> 
> +1

+1

> 
> 
> 
> 
> 
> 
> 
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Blog post: Monitoring Improvements in oVirt

[Arik Hadas]

- Original Message -
> Hi Arik,
> 
> Very nice output..

Thank you

> 
> see also our mojo page:
> 
> https://mojo.redhat.com/groups/rhev-m-scalability-performance-team
> 
> can you elaborate how did you rampup 6K vms?

1. Created a cluster with 1 real host, 1 fake-VDSM based host and 1 storage 
domain (so the storage pool will be up).
2. Created a pool with 6000 VMs that is based on template with one network 
interface and no disks.
3. Ran all the VMs on fake-VDSM (they were pinned to that host)

At that point I had a fake-VDSM host that runs 6K VMs.
Then, 1. I put all the things in my master-based setup to maintenance and 
removed all VMs.
  2. Configured 1 cluster with this host - all the 6K VMs were added as 
external VMs.
And on 3.6 I did the same - created a cluster only with that host (and the VMs 
were added as external VMs).

By this, I had the exact same setup on both 3.6 and 4.0 and I didn't have 
anything else that could influence the measurements.

Note that that's why many of the numbers presented in the post should not be 
considered as absolute values but only
as a relative values between 3.6 and 4.0 setups.
My point is that when I say that saving the statistics took 2% of the 
interactions with the database, this '2%'
would probably be different in other environments and lower when the system 
does other things my system did not do.

> 
> also can you elaborate the following:
> 
> - HW definition?

Dell optiplex 780, 16GB of RAM, 4 CPUs with 4 cores-per-CPI
Both the engine and fake VDSM were running on that host.

> 
> - java HEAP Size?

I didn't change it manually - initial and maximum heap size of 3971M

> 
> - any postgres tuning ?

No, I kept the default settings.

> 
> 
> we are about to scale out the latest 4.0 RHEVM
> 
> any relevant changes that we should aware of?

Note that only part of these changes are in 4.0.
What bothers me and worth checking on 4.0 is the memory consumption -
in 4.0 we already cache VM statistics but many of the changes that probably lead
of the overall decrement of the memory consumption are missing there.
So it would be interesting to see what is the implication of storing more data
in memory in 4.0 - if it is problematic we can easily remove this cache as it
is rarely used in 4.0.

> 
> -Eldad
> 
> 
> 
> On 07/24/2016 05:15 PM, Arik Hadas wrote:
> > Hi all,
> >
> > I wrote a blog post that summarizes the recent improvements in the code
> > that monitors virtual machines in oVirt.
> > It describes the changes that were done and shows a comparison of different
> > aspects of oVirt with these changes
> > (based on the master branch) vs oVirt 3.6 without these changes:
> > http://ahadas.github.io/monitoring-improvements-in-ovirt/
> >
> > It is mostly a technical post.
> >
> > However, users might find it interesting as well.
> > Note that the measurements were taken on an environment with 1 host and
> > 6000 running VMs (in a pool).
> > Each VM had a network interface and was diskless.
> > Obviously, this is not a typical setup but it was useful to easily expose
> > some of weak sides of the monitoring.
> >
> > Your feedback is always welcome!
> >
> > Regards,
> > Arik
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Blog post: Monitoring Improvements in oVirt

[Arik Hadas]

Hi all,

I wrote a blog post that summarizes the recent improvements in the code that 
monitors virtual machines in oVirt.
It describes the changes that were done and shows a comparison of different 
aspects of oVirt with these changes
(based on the master branch) vs oVirt 3.6 without these changes:
http://ahadas.github.io/monitoring-improvements-in-ovirt/

It is mostly a technical post.

However, users might find it interesting as well.
Note that the measurements were taken on an environment with 1 host and 6000 
running VMs (in a pool).
Each VM had a network interface and was diskless.
Obviously, this is not a typical setup but it was useful to easily expose some 
of weak sides of the monitoring.

Your feedback is always welcome!

Regards,
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] oVirt-specific aspect languages

[Arik Hadas]

Hi,

Thanks for the feedback!
I commented below.

- Original Message -
> Hi all,
> 
> I agree that it was an interesting article.
> 
> > However, since AspectJ is an extension to Java, most of oVirt
> > developers would need to know additional programming language
> > which puts the cost-effectiveness of this approach into question.
> 
> Actually you only need to learn some very basic stuff about pointcut
> definition. The rest was pure Java for anything I ever needed (before,
> after and around cases).
> 

Well, it depends on both the base code and the crosscutting-concerns you 
develop the aspects for I guess.
>From my experience with AspectJ, the very basic stuff there typically fits 
>only for very simple crosscutting concerns.
For example, in the generated aspects for the languages I developed for the 
crosscutting concerns in oVirt I had to use more than the basic stuff (e.g. 
inter-type declarations).
I'm curios, where did you use AspectJ? For how many and what kind of 
crosscutting concerns did you use it?

> DSLs are a nice thing to have, and we can start by utilizing object
> builders more :) Those are a kind of DSL too if nicely written. See
> the latest email on this list from Roman Mohr that talks about Domain
> Object Builders (or Spring Security Java configuration [2] to see a
> more complicated example of how it is used elsewhere).
> 
> Martin
> 
> [1] http://lists.ovirt.org/pipermail/devel/2016-April/012790.html
> [2]
> http://docs.spring.io/spring-security/site/docs/current/reference/html/jc.html
> 
> On Mon, Apr 11, 2016 at 4:13 PM, Vojtech Szocs  wrote:
> > Hi Arik,
> >
> > thanks for sharing this, the article is very interesting!
> >
> > "
> > However, since AspectJ is an extension to Java, most of oVirt
> > developers would need to know additional programming language
> > which puts the cost-effectiveness of this approach into question.
> > "
> >
> > I always thought that good Java developers should be familiar
> > with AOP concepts anyway.

I think that many are familiar with the general AOP concepts but nevertheless 
decide not to adopt them.
Many think that AOP is useful only for simple logging and tracing or as a 
simplified API for byte-code manipulation (I guess that's why AspectJ is used 
by GWT in our build process, right?) or in its simplified form as spring-aop, 
because of the known drawbacks of AspectJ.
Hopefully the proposed approach would change that :)

> >
> > That said, I really like the combo of AOP with DSL (or ASL)
> > and I agree that it can simplify handling concerns in a way
> > that is much closer to app's domain, rather than being closer
> > to its implementation (as with AOP).
> >
> > Regards,
> > Vojtech
> >
> >
> > - Original Message -
> >> From: "Arik Hadas" 
> >> To: "devel" 
> >> Sent: Monday, April 11, 2016 9:02:39 AM
> >> Subject: [ovirt-devel] oVirt-specific aspect languages
> >>
> >> Hi All,
> >>
> >> Last month I participated in Modularity'16, an academic conference that
> >> focuses on advanced software modularization techniques, where I presented
> >> my
> >> work in the area of domain specific aspect languages that was evaluated on
> >> oVirt.
> >>
> >> Many have asked me to elaborate on this work, so I wrote a post that
> >> explains
> >> the idea and provides references to relevant materials that were presented
> >> in the conference, which I would like to share with you:
> >> http://ahadas.github.io/oVirt-Specifc-Aspect-Languages/
> >>
> >> Regards,
> >> Arik
> >> ___
> >> Devel mailing list
> >> Devel@ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/devel
> >>
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] oVirt-specific aspect languages

[Arik Hadas]

Hi All,

Last month I participated in Modularity'16, an academic conference that focuses 
on advanced software modularization techniques, where I presented my work in 
the area of domain specific aspect languages that was evaluated on oVirt.

Many have asked me to elaborate on this work, so I wrote a post that explains 
the idea and provides references to relevant materials that were presented in 
the conference, which I would like to share with you:
http://ahadas.github.io/oVirt-Specifc-Aspect-Languages/

Regards,
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Access to memory snapshots?

[Arik Hadas]

Hi Matt,

When creating a snapshot with memory, 2 additional images are created:
1. for the VM configuration/metadata
2. for the memory dump

In oVirt 3.6 and below you'll notice a field called memory_vol_handle in the 
snapshots table that points to these images. It is of the follow format:
,

In master branch, you'll find two fields in the snapshots table called 
memory_dump_disk_id and memory_metadata_disk_id that contain the disk IDs.
We plan to remove the memory_vol_handle field from the snapshots table soon, so 
please do not rely on it.

Regards,
Arik

- Original Message -
> Hi everyone!
> I am working on developing a plugin for the cuckoo sandbox (
> http://cuckoosandbox.org ), that will allow users to select oVirt as a
> virtualiztion solution. I have it working, for the most part, but one area
> where I need some guidance is in retrieving memory images. I know that when
> you take a snapshot, you can request that the memory be stored as well.
> 
> My issue is that I don't see a way to retrieve the memory, so that it can be
> feed into analysis tools like Volatility (
> http://www.volatilityfoundation.org/ ). Does anyone have any pointers for
> me?
> 
> Thanks!
> 
> --Matt
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Ovirt API Import OVA

[Arik Hadas]

Actually it depends on the content of the OVA file.
If the OVA file was generated by vSphere (export VM) then since oVirt 3.6 you 
can import it via the webadmin (virtual machines tab -> import -> select OVA as 
the source).
Note that you'll need to copy the OVA to one of your hosts that is installed 
with virt-v2v for that.
This action is missing in the API.

- Original Message -
> Bryan hi,
> 
> There is not an automated way to import an OVA into oVirt currently.
> 
> There might be a possibility to manually import the disks, then import the
> OVF file, but it won't be straightforward.
> 
> In the future, there will be a support to upload an image from the web UI.
> [1]
> 
> Regards,
> 
> Fred
> 
> [1] http://www.ovirt.org/develop/release-management/features/image-upload/
> 
> On Thu, Mar 3, 2016 at 2:33 AM, Bryan Hughes < bhug...@exetergov.com > wrote:
> 
> 
> 
> Hello,
> 
> I was just wondering if there was an automated way to import an OVA into
> ovirt. We are attempting an unattended install and wanted to import an OVA
> into ovirt through the API preferably.
> 
> I could not find anything in the API docs itself but may have missed
> something.
> 
> Thanks,
> Bryan
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Proposing Martin Betak as oVirt UI maintainer

[Arik Hadas]

+1

- Original Message -
> +1, just for even being able to upgrade GWT! :) Not easy stuff.
> 
> Well deserved!
> 
> Best wishes,
> Greg
> 
> 
> On Wed, Feb 3, 2016 at 12:38 PM, Omer Frenkel < d.mosqu...@gmail.com > wrote:
> 
> 
> +1 :)
> 
> On Wed, Feb 3, 2016 at 4:00 PM, Vojtech Szocs < vsz...@redhat.com > wrote:
> > Hello, UI folks!
> > 
> > I'd like to propose Martin Betak as oVirt UI maintainer.
> > 
> > Over time, Martin made some significant UI contributions:
> > 
> > - improving UiCommon by utilizing Java generics [1]
> > 
> > - adding GinUiBinder [2] to allow @Inject'ing GIN-managed
> > objects into GWT widgets created in ui.xml templates
> > 
> > - upgrade GWT version to 2.6.1 [3] for both oVirt webapps
> > 
> > - providing excellent feedback and ideas on oVirtJS project
> > [4] while demonstrating outstanding JavaScript knowledge
> > 
> > [1] https://gerrit.ovirt.org/#/c/32907/
> > [2] https://gerrit.ovirt.org/#/c/34954/
> > [3] https://gerrit.ovirt.org/#/c/32135/
> > [4] https://gerrit.ovirt.org/#/c/49466/
> > 
> > Personal note: Martin is familiar with oVirt GWT UI infra
> > as a whole (GWT-P component model, advanced GWT features,
> > UiCommon integration etc). His grasp on JS and its current
> > eco-system is very solid.
> > 
> > Regards,
> > Vojtech
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> 
> --
> Greg Sheremeta, MBA
> Red Hat, Inc.
> Sr. Software Engineer
> gsher...@redhat.com
> 919-741-4016
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Failure running engine-setup on latest master

[Arik Hadas]

My bad, fix is posted: https://gerrit.ovirt.org/#/c/51420/1

- Original Message -
> Hi,
> 
> I am trying to run engine-setup on latest master and it is failing on the
> database upgrade portion. The relevant part of the log is here. Can anyone
> help me solve this?
> 
> psql:/home/awels/ovirt-**FILTERED**/share/ovirt-
> **FILTERED**/dbscripts/common_sp.sql:1117: NOTICE:  drop cascades to function
> fn_db_get_async_tasks()
> NOTICE:  drop cascades to trigger delete_disk_image_dynamic_for_image on
> table
> images
> psql:/home/awels/ovirt-**FILTERED**/share/ovirt-
> **FILTERED**/dbscripts/upgrade/04_00_0140_convert_memory_snapshots_to_disks.sql:57:
> ERROR:  integer out of range
> FATAL: Cannot execute sql command: --file=/home/awels/ovirt-
> **FILTERED**/share/ovirt-
> **FILTERED**/dbscripts/upgrade/04_00_0140_convert_memory_snapshots_to_disks.sql
> 
> 2016-01-05 09:18:21 DEBUG otopi.context context._executeMethod:156 method
> exception
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146, in
> _executeMethod
> method['method']()
>   File "/home/awels/ovirt-**FILTERED**/share/ovirt-
> **FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-setup/ovirt-
> **FILTERED**/db/schema.py", line 291, in _misc
> o**FILTERED**cons.EngineDBEnv.PGPASS_FILE
>   File "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 946, in
> execute
> command=args[0],
> RuntimeError: Command '/home/awels/ovirt-**FILTERED**/share/ovirt-
> **FILTERED**/dbscripts/schema.sh' failed to execute
> 
> Thanks,
> Alexander
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [DB] Diffrent UUIDs for inserted data per installation

[Arik Hadas]

- Original Message -
> Hi
> 
> I am trying to cleanup all the data insertion to the engine DB and make it
> more general
> The main drive to that is DB version squashing that was done manually and
> therefor was error prone ...
> 
> I know that both storage_pool_id (a.k.a DC) and vds_group_id (a.k.a Cluster)
> needs to get a different UUID per installation.
> But I had found that UUIDs are generated per installation for also :
> 
> table  |   column/s
> 
> 
> [cpu_profiles] : id
> 
> [gluster_services] : id
> 
> [mac_pools] : id
> 
> [permissions] : id, object_id
They are generated for instance-types.
id doesn't have to be different per installation
object_id doesn't have to be different either since it points to id of default 
instance-type that can be static

> 
> [vm_device]: device_id, vm_id
device_id - generated when adding virtio-serial devices. It doesn't have to be 
different per installation
I didn't find where vm_id is generated..

> 
> [vm_static] : vm_guid
Generated when inserting default instance-types. It doesn't have to be 
different per installation

> 
> [vnic_profiles] : id
> 
> 
> Please let me know if any of the above should be generated using the
> uuid_generate_v1() function on each installation or we can have static IDs
> for those.
> 
>  
> 
> Thanks
> Eli Mesika
> 
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] virt-v2v intergration with vdsm

[Arik Hadas]

- Original Message -
> On 10/02/2014 11:23 AM, Arik Hadas wrote:
> >
> >
> > - Original Message -
> >> On 09/30/2014 04:18 PM, Arik Hadas wrote:
> >>>
> >>>
> >>> - Original Message -
> >>>> To expand on the previous email, the web page is quite vague about
> >>>> what precise sources you want to import from.
> >>>>
> >>>> The ones supported by virt-v2v are:
> >>>>
> >>>>- VMware live vCenter: I assume you would want this.  (Note ESXi is
> >>>>  mentioned in the web page, but virt-v2v does not support import
> >>>>  from ESXi directly)
> >>>>
> >>>>- OVA file: As Shahar wrote this bit, I'll assume you'll want to use
> >>>>  it.
> >>>>
> >>>>- RHEL 5 Xen: SSH key management is going to be a problem here.
> >>>>
> >>>>- Local disk files: Seems unlikely you'll want this.
> >>>>
> >>>>- Citrix Xen, Hyper-V, etc.: Requires development work.
> >>>>
> >>>>- Physical machines
> >>>>
> >>>> VMware vCenter
> >>>> --
> >>>>
> >>>> See:
> >>>> http://libguestfs.org/virt-v2v.1.html#input-from-vmware-vcenter-server
> >>>>
> >>>> The user will need to specify:
> >>>>- hostname of VMware vCenter server
> >>>>- optional port number
> >>>>- username
> >>>>- password
> >>>>- name of VMware Datacenter
> >>>>- hostname of VMware ESXi server [may be able to relax this in
> >>>>future]
> >>>>- whether or not to check HTTPS certs
> >>>>- name of the guest
> >>>>
> >>>> As detailed in the link above, there are various virsh commands you
> >>>> can run from VDSM to establish whether conversion is likely to work,
> >>>> and to get a list of guests which you can present to the user.
> >>>>
> >>>> OVA file
> >>>> 
> >>>>
> >>>> The user will need to specify:
> >>>>- OVA file (eg. location)
> >>>>
> >>>> I've no idea how the file will end up being accessible to VDSM.  NFS
> >>>> mount?
> >>>>
> >>>> RHEL 5 Xen
> >>>> --
> >>>>
> >>>> See: http://libguestfs.org/virt-v2v.1.html#input-from-rhel-5-xen
> >>>>
> >>>> I think we're going to have a problem with ssh keys, because this mode
> >>>> currently only works when the virt-v2v client can get passwordless
> >>>> access to the RHEL 5 Xen server using ssh-agent (note that Kerberos
> >>>> does not work).  We might fix this in future, but at the moment you're
> >>>> somehow going to have to ensure that the oVirt node has a private key
> >>>> which is listed in the authorized_keys in the RHEL 5 Xen server.  I
> >>>> simply cannot imagine a user interface that is going to make this
> >>>> usable.
> >>>>
> >>>> I should also say that few people care about RHEL 5 Xen imports.
> >>>> Citrix Xen imports OTOH, certainly, but those were out of scope for
> >>>> current upstream development for RHEL 7.1.
> >>>>
> >>>> Anyway, the user will have to specify:
> >>>>
> >>>>- hostname of RHEL 5 Xen server
> >>>>- username
> >>>>- password
> >>>>- name of the guest
> >>>>
> >>>> There are various virsh commands you can run from VDSM to establish
> >>>> whether conversion is likely to work, and to get a list of guests
> >>>> which you can present to the user.
> >>>>
> >>>> Physical machines
> >>>> -
> >>>>
> >>>> See also: http://libguestfs.org/virt-p2v.1.html
> >>>> [Update from previous email: I have now updated this page to contain
> >>>> the latest documentation.]
> >>>>
> >>>> Virt-v2v can import physical machines, but you have to boot the
> >>>> separate virt-p2v ISO on the physical machine, and generally speaking
> 

Re: [ovirt-devel] virt-v2v intergration with vdsm

[Arik Hadas]

- Original Message -
> On 09/30/2014 04:18 PM, Arik Hadas wrote:
> >
> >
> > - Original Message -
> >> To expand on the previous email, the web page is quite vague about
> >> what precise sources you want to import from.
> >>
> >> The ones supported by virt-v2v are:
> >>
> >>   - VMware live vCenter: I assume you would want this.  (Note ESXi is
> >> mentioned in the web page, but virt-v2v does not support import
> >> from ESXi directly)
> >>
> >>   - OVA file: As Shahar wrote this bit, I'll assume you'll want to use
> >> it.
> >>
> >>   - RHEL 5 Xen: SSH key management is going to be a problem here.
> >>
> >>   - Local disk files: Seems unlikely you'll want this.
> >>
> >>   - Citrix Xen, Hyper-V, etc.: Requires development work.
> >>
> >>   - Physical machines
> >>
> >> VMware vCenter
> >> --
> >>
> >> See:
> >> http://libguestfs.org/virt-v2v.1.html#input-from-vmware-vcenter-server
> >>
> >> The user will need to specify:
> >>   - hostname of VMware vCenter server
> >>   - optional port number
> >>   - username
> >>   - password
> >>   - name of VMware Datacenter
> >>   - hostname of VMware ESXi server [may be able to relax this in future]
> >>   - whether or not to check HTTPS certs
> >>   - name of the guest
> >>
> >> As detailed in the link above, there are various virsh commands you
> >> can run from VDSM to establish whether conversion is likely to work,
> >> and to get a list of guests which you can present to the user.
> >>
> >> OVA file
> >> 
> >>
> >> The user will need to specify:
> >>   - OVA file (eg. location)
> >>
> >> I've no idea how the file will end up being accessible to VDSM.  NFS
> >> mount?
> >>
> >> RHEL 5 Xen
> >> --
> >>
> >> See: http://libguestfs.org/virt-v2v.1.html#input-from-rhel-5-xen
> >>
> >> I think we're going to have a problem with ssh keys, because this mode
> >> currently only works when the virt-v2v client can get passwordless
> >> access to the RHEL 5 Xen server using ssh-agent (note that Kerberos
> >> does not work).  We might fix this in future, but at the moment you're
> >> somehow going to have to ensure that the oVirt node has a private key
> >> which is listed in the authorized_keys in the RHEL 5 Xen server.  I
> >> simply cannot imagine a user interface that is going to make this
> >> usable.
> >>
> >> I should also say that few people care about RHEL 5 Xen imports.
> >> Citrix Xen imports OTOH, certainly, but those were out of scope for
> >> current upstream development for RHEL 7.1.
> >>
> >> Anyway, the user will have to specify:
> >>
> >>   - hostname of RHEL 5 Xen server
> >>   - username
> >>   - password
> >>   - name of the guest
> >>
> >> There are various virsh commands you can run from VDSM to establish
> >> whether conversion is likely to work, and to get a list of guests
> >> which you can present to the user.
> >>
> >> Physical machines
> >> -
> >>
> >> See also: http://libguestfs.org/virt-p2v.1.html
> >> [Update from previous email: I have now updated this page to contain
> >> the latest documentation.]
> >>
> >> Virt-v2v can import physical machines, but you have to boot the
> >> separate virt-p2v ISO on the physical machine, and generally speaking
> >> the process is directed from the physical machine.
> >>
> >> This makes it quite unlike the other input sources.  (Users will still
> >> be able to use the self-boot + '-o rhev' method, ie. import into the
> >> Export Storage Domain, as with RHEL 6).
> >>
> >> If oVirt can (or does?) run a PXE server, then it's possible to serve a
> >> virt-p2v bootable image to physical machines, and it's also possible
> >> to automate it from the PXE server side.
> >> [http://libguestfs.org/virt-p2v.1.html#kernel-command-line-configuration]
> >>
> >> Environment variables
> >> -
> >>
> >> You may also want to set the following environment variables before
> >> running virt-v2v:
> >>
> >> LIBGUESTFS_BACKEND=direct
>

Re: [ovirt-devel] virt-v2v intergration with vdsm

[Arik Hadas]

- Original Message -
> To expand on the previous email, the web page is quite vague about
> what precise sources you want to import from.
> 
> The ones supported by virt-v2v are:
> 
>  - VMware live vCenter: I assume you would want this.  (Note ESXi is
>mentioned in the web page, but virt-v2v does not support import
>from ESXi directly)
> 
>  - OVA file: As Shahar wrote this bit, I'll assume you'll want to use
>it.
> 
>  - RHEL 5 Xen: SSH key management is going to be a problem here.
> 
>  - Local disk files: Seems unlikely you'll want this.
> 
>  - Citrix Xen, Hyper-V, etc.: Requires development work.
> 
>  - Physical machines
> 
> VMware vCenter
> --
> 
> See: http://libguestfs.org/virt-v2v.1.html#input-from-vmware-vcenter-server
> 
> The user will need to specify:
>  - hostname of VMware vCenter server
>  - optional port number
>  - username
>  - password
>  - name of VMware Datacenter
>  - hostname of VMware ESXi server [may be able to relax this in future]
>  - whether or not to check HTTPS certs
>  - name of the guest
> 
> As detailed in the link above, there are various virsh commands you
> can run from VDSM to establish whether conversion is likely to work,
> and to get a list of guests which you can present to the user.
> 
> OVA file
> 
> 
> The user will need to specify:
>  - OVA file (eg. location)
> 
> I've no idea how the file will end up being accessible to VDSM.  NFS mount?
> 
> RHEL 5 Xen
> --
> 
> See: http://libguestfs.org/virt-v2v.1.html#input-from-rhel-5-xen
> 
> I think we're going to have a problem with ssh keys, because this mode
> currently only works when the virt-v2v client can get passwordless
> access to the RHEL 5 Xen server using ssh-agent (note that Kerberos
> does not work).  We might fix this in future, but at the moment you're
> somehow going to have to ensure that the oVirt node has a private key
> which is listed in the authorized_keys in the RHEL 5 Xen server.  I
> simply cannot imagine a user interface that is going to make this
> usable.
> 
> I should also say that few people care about RHEL 5 Xen imports.
> Citrix Xen imports OTOH, certainly, but those were out of scope for
> current upstream development for RHEL 7.1.
> 
> Anyway, the user will have to specify:
> 
>  - hostname of RHEL 5 Xen server
>  - username
>  - password
>  - name of the guest
> 
> There are various virsh commands you can run from VDSM to establish
> whether conversion is likely to work, and to get a list of guests
> which you can present to the user.
> 
> Physical machines
> -
> 
> See also: http://libguestfs.org/virt-p2v.1.html
> [Update from previous email: I have now updated this page to contain
> the latest documentation.]
> 
> Virt-v2v can import physical machines, but you have to boot the
> separate virt-p2v ISO on the physical machine, and generally speaking
> the process is directed from the physical machine.
> 
> This makes it quite unlike the other input sources.  (Users will still
> be able to use the self-boot + '-o rhev' method, ie. import into the
> Export Storage Domain, as with RHEL 6).
> 
> If oVirt can (or does?) run a PXE server, then it's possible to serve a
> virt-p2v bootable image to physical machines, and it's also possible
> to automate it from the PXE server side.
> [http://libguestfs.org/virt-p2v.1.html#kernel-command-line-configuration]
> 
> Environment variables
> -
> 
> You may also want to set the following environment variables before
> running virt-v2v:
> 
> LIBGUESTFS_BACKEND=direct
> 
> This is currently required for VMware and RHEL 5 Xen conversions owing
> to a bug in libvirt (1134592) that was today pushed to RHEL 7.2.  You
> should be able to drop this once libvirt is fixed.
> 
> TMPDIR=...
> 
> You might want to set this to control where large temporary files are
> created during the v2v process.
> 
> Other environment variables that may affect virt-v2v are discussed on
> both of these pages:
> http://libguestfs.org/guestfs.3.html#environment-variables
> http://libguestfs.org/virt-v2v.1.html#environment-variables
> 
> A note on the output mode
> -
> 
> You're going to be using '-o vdsm -os /storage_domain' and the other
> options: '--vdsm-image-uuid UUID', '--vdsm-vol-uuid UUID',
> '--vdsm-vm-uuid UUID' [http://libguestfs.org/virt-v2v.1.html#options]
> 
> Implicit in the contract with '-o vdsm' is that VDSM is responsible
> for creating the output directories, running virt-v2v as user 36:36,
> and deleting the output directories if virt-v2v fails.

VDSM is not only responsible for creating the output directories but also the 
output files (we have to have it that way when the destination storage domain 
resides in block-based device). We'll probably need to use a technique that was 
used elsewhere, where output was streamed into those files. so virt-v2v won't 
create the output files.

> 
> In this output mode virt-v2v doesn't create or clean up a

Re: [ovirt-devel] virt-v2v integration feature

[Arik Hadas]

Hi All,

Better late than never..

Thanks for all the feedback, it was really constructive.
I made major changes in the wiki page to address the comments,
Please take another look:
http://www.ovirt.org/Features/virt-v2v_Integration

Thanks,
Arik

- Original Message -
> Hi All,
> 
> The proposed feature will introduce a new process of import virtual machines
> from external systems using virt-v2v in oVirt.
> I've created a wiki page that contains initial thoughts and design for it:
> http://www.ovirt.org/Features/virt-v2v_Integration
> 
> You are more than welcome to share your thoughts and insights.
> 
> Thanks,
> Arik
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] virt-v2v integration feature

[Arik Hadas]

Hi All,

The proposed feature will introduce a new process of import virtual machines 
from external systems using virt-v2v in oVirt.
I've created a wiki page that contains initial thoughts and design for it:
http://www.ovirt.org/Features/virt-v2v_Integration

You are more than welcome to share your thoughts and insights.

Thanks,
Arik
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] HA not working 3.4.2

[Arik Hadas]

Hi,

- Original Message -
> Hi i was testing HA of ovirt,(i am from Xenserver background and trying to
> use KVM solution in our environment)
> 
> Have 2 hosts with NFS storage and i tested migration between these two hosts.
> 
> On Host2 while VM was running on it, i unplugged Power cable and it took long
> minutes to update that VM is down,okkk now long wait..
> ..
> .
> ...
> 
> VM state changed to unkown and not booted on second Node ie Host1
> 
> its been half an hour and still the VM is not restarted
> 
> What can be the issue?What should i do it make the VM in real HA so that it
> will be restarted on the active node.

We don't restart the VM in that case on purpose, since we really don't know
what's the status of the VM - the host might have been just disconnected from
the network while the VM is still running (and connected to the storage),
so if we'll also run the VM on a different host, we'll get "split-brain".

In order to restart HA VMs which were running on host that went down, you either
needs to have power-management defined for that host (for automatic restart) or
to trigger it manually by selecting "confirm host has been rebooted" when the
host is changed to non-responsive state (and you need to have additional host
which is up of course).

> 
> How HA is enabled
> what i did was while creating vm> Advanced option>High Availability>Highly
> Available
> 
> I am sure this can be a configuration error,Can some one please help me with
> enabling HA.
> 
> Please Help
> 
> Thanks
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel