Re: [ovirt-devel] [Engine] Runtime log controll script

[Martin Betak]

On Tue, Apr 18, 2017 at 9:52 AM, Roy Golan  wrote:

> Ever wanted to raise level of the engine logs and wanted to to that fast
> and runtime?
>
> This is a small wrapper around wildfly mgmt interface, called
> *log-control* to do the trick[1]
>
> Example, debug the db interaction layer:
>
> ./log-control org.ovirt.engine.core.dal debug
>
> It will first try blantly to add the log category and then will set the
> log level according to what you set. It is simple and stupid.
>
> More interesting logger categories:
>
> business logic (commands, queries) - org.ovirt.engine.core.bll
> hosts interaction - org.ovirt.engine.core.vdsbroker
> various utilities - org.ovirt.engine.core.utils
> aaa - org.ovirt.engine.exttool.aaa
>
> General suggestion -
> I think is is it time for *ovirt-engine-contrib* so mini-helpers like
> that can exists and when they are solid can go into mainstream repo, if
> needed in there.
>

Nice job, Roy! +1 to the "ovirt-engine-contrib" idea.


>
> [1] https://gist.github.com/rgolangh/1cb9f9b3b7f7f0a1d16b5a976d90bd55
>
> Thanks,
> Roy
>
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 9/4/2017 ] [ add_secondary_storage_domains ]

[Martin Betak]

Ok, Piotr, your patch appears to also be clean
http://jenkins.ovirt.org/job/ovirt-system-tests_manual/230/ ...

On Mon, Apr 10, 2017 at 4:01 PM, Piotr Kliczewski 
wrote:

>
>
> On Mon, Apr 10, 2017 at 3:39 PM, Martin Betak  wrote:
>
>> I have now enqueued another run of OST for the next VDSM patch in line
>> (that was merged around the time of the first failure)
>> https://gerrit.ovirt.org/#/c/75299/2 to see if it was the culprit.
>>
>
> This patch is xmlrpc related. Can you point me to the issue it caused?
>
>
>> Not sure why we allow only one (*globally for all developers) *concurrent
>> run of the manual OST job... I've created a jira issue for this
>> https://ovirt-jira.atlassian.net/browse/OST-61
>>
>> Martin
>>
>> On Mon, Apr 10, 2017 at 3:29 PM, Martin Betak  wrote:
>>
>>> Actually it appears that this failure is not engine related (tried
>>> reverting my patches from saturday but still received similar errors with
>>> lago in the [ add_secondary_storage_domains] test).
>>> But when running the current engine master with VDSM cutoff before the
>>> time of failure, it passed http://jenkins.ovirt.or
>>> g/job/ovirt-system-tests_manual/229/
>>>
>>> This was the VDSM commit that *worked*
>>> *Commit 5c8aff6177bdaa81ee11c20d417c9bb10e651fb8 by Francesco Romani
>>> <http://jenkins.ovirt.org/user/from...@redhat.com/>*
>>>
>>> On Mon, Apr 10, 2017 at 11:08 AM, Michal Skrivanek <
>>> michal.skriva...@redhat.com> wrote:
>>>
>>>>
>>>> > On 9 Apr 2017, at 09:42, Barak Korren  wrote:
>>>> >
>>>> > On 9 April 2017 at 10:39, Yaniv Kaul  wrote:
>>>> >> 1. The error:
>>>> >> 2017-04-08 14:12:11,376-04 ERROR
>>>> >> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) []
>>>> Not able
>>>> >> to update response for "fc0cef3f-d8fa-4074-b315-e36cc2f63fa1"
>>>> >>
>>>> >> Is still seen, not sure why.
>>>> >>
>>>> >> 2. MOM is not available (unrelated, but still a bug)
>>>> >>
>>>> >> 3. New error I have not seen in the past on host1
>>>> >> (http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_m
>>>> aster/6257/artifact/exported-artifacts/basic-suit-master-el7
>>>> /test_logs/basic-suite-master/post-002_bootstrap.py/lago-bas
>>>> ic-suite-master-host1/_var_log/vdsm/vdsm.log
>>>> >> ) :
>>>> >>
>>>> >> 2017-04-08 14:12:22,208-0400 WARN  (jsonrpc/6) [storage.LVM] lvm pvs
>>>> failed:
>>>> >> 5 [] ['  Failed to find physical volume
>>>> >> "/dev/mapper/360014050eacbb3c8b21428fb6683f074".'] (lvm:325)
>>>> >>
>>>> >> 4. New warning:
>>>> >> 2017-04-08 14:13:17,327-0400 WARN  (check/loop) [storage.asyncutils]
>>>> Call
>>>> >> >>> >> /dev/0eb0f05c-0507-4f02-ba13-b8d865538d7a/metadata running
>>>> >> next_check=4295183.36 at 0x2fb9190>> delayed by 1.43 seconds
>>>> >> (asyncutils:138)
>>>> >>
>>>> >>
>>>> >> Do we know when it began?
>>>> >
>>>> > Yes. I linked to the patch that seems to have started this.
>>>>
>>>> This should be easy to revert
>>>> Working on it
>>>> Then we’ll see what exactly is the culprit
>>>>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Barak Korren
>>>> > bkor...@redhat.com
>>>> > RHCE, RHCi, RHV-DevOps Team
>>>> > https://ifireball.wordpress.com/
>>>> > ___
>>>> > Devel mailing list
>>>> > Devel@ovirt.org
>>>> > http://lists.ovirt.org/mailman/listinfo/devel
>>>> >
>>>> >
>>>>
>>>> ___
>>>> Devel mailing list
>>>> Devel@ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 9/4/2017 ] [ add_secondary_storage_domains ]

[Martin Betak]

I have now enqueued another run of OST for the next VDSM patch in line
(that was merged around the time of the first failure) https://gerrit.ovirt.
org/#/c/75299/2 to see if it was the culprit.
Not sure why we allow only one (*globally for all developers) *concurrent
run of the manual OST job... I've created a jira issue for this
https://ovirt-jira.atlassian.net/browse/OST-61

Martin

On Mon, Apr 10, 2017 at 3:29 PM, Martin Betak  wrote:

> Actually it appears that this failure is not engine related (tried
> reverting my patches from saturday but still received similar errors with
> lago in the [ add_secondary_storage_domains] test).
> But when running the current engine master with VDSM cutoff before the
> time of failure, it passed http://jenkins.ovirt.or
> g/job/ovirt-system-tests_manual/229/
>
> This was the VDSM commit that *worked*
> *Commit 5c8aff6177bdaa81ee11c20d417c9bb10e651fb8 by Francesco Romani
> <http://jenkins.ovirt.org/user/from...@redhat.com/>*
>
> On Mon, Apr 10, 2017 at 11:08 AM, Michal Skrivanek <
> michal.skriva...@redhat.com> wrote:
>
>>
>> > On 9 Apr 2017, at 09:42, Barak Korren  wrote:
>> >
>> > On 9 April 2017 at 10:39, Yaniv Kaul  wrote:
>> >> 1. The error:
>> >> 2017-04-08 14:12:11,376-04 ERROR
>> >> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not
>> able
>> >> to update response for "fc0cef3f-d8fa-4074-b315-e36cc2f63fa1"
>> >>
>> >> Is still seen, not sure why.
>> >>
>> >> 2. MOM is not available (unrelated, but still a bug)
>> >>
>> >> 3. New error I have not seen in the past on host1
>> >> (http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_m
>> aster/6257/artifact/exported-artifacts/basic-suit-master-el7
>> /test_logs/basic-suite-master/post-002_bootstrap.py/lago-bas
>> ic-suite-master-host1/_var_log/vdsm/vdsm.log
>> >> ) :
>> >>
>> >> 2017-04-08 14:12:22,208-0400 WARN  (jsonrpc/6) [storage.LVM] lvm pvs
>> failed:
>> >> 5 [] ['  Failed to find physical volume
>> >> "/dev/mapper/360014050eacbb3c8b21428fb6683f074".'] (lvm:325)
>> >>
>> >> 4. New warning:
>> >> 2017-04-08 14:13:17,327-0400 WARN  (check/loop) [storage.asyncutils]
>> Call
>> >> > >> /dev/0eb0f05c-0507-4f02-ba13-b8d865538d7a/metadata running
>> >> next_check=4295183.36 at 0x2fb9190>> delayed by 1.43 seconds
>> >> (asyncutils:138)
>> >>
>> >>
>> >> Do we know when it began?
>> >
>> > Yes. I linked to the patch that seems to have started this.
>>
>> This should be easy to revert
>> Working on it
>> Then we’ll see what exactly is the culprit
>>
>> >
>> >
>> >
>> > --
>> > Barak Korren
>> > bkor...@redhat.com
>> > RHCE, RHCi, RHV-DevOps Team
>> > https://ifireball.wordpress.com/
>> > ___
>> > Devel mailing list
>> > Devel@ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>> >
>> >
>>
>> ___
>> Devel mailing list
>> Devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 9/4/2017 ] [ add_secondary_storage_domains ]

[Martin Betak]

Actually it appears that this failure is not engine related (tried
reverting my patches from saturday but still received similar errors with
lago in the [ add_secondary_storage_domains] test).
But when running the current engine master with VDSM cutoff before the time
of failure, it passed
http://jenkins.ovirt.org/job/ovirt-system-tests_manual/229/

This was the VDSM commit that *worked*
*Commit 5c8aff6177bdaa81ee11c20d417c9bb10e651fb8 by Francesco Romani
*

On Mon, Apr 10, 2017 at 11:08 AM, Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

>
> > On 9 Apr 2017, at 09:42, Barak Korren  wrote:
> >
> > On 9 April 2017 at 10:39, Yaniv Kaul  wrote:
> >> 1. The error:
> >> 2017-04-08 14:12:11,376-04 ERROR
> >> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not
> able
> >> to update response for "fc0cef3f-d8fa-4074-b315-e36cc2f63fa1"
> >>
> >> Is still seen, not sure why.
> >>
> >> 2. MOM is not available (unrelated, but still a bug)
> >>
> >> 3. New error I have not seen in the past on host1
> >> (http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
> master/6257/artifact/exported-artifacts/basic-suit-master-
> el7/test_logs/basic-suite-master/post-002_bootstrap.py/
> lago-basic-suite-master-host1/_var_log/vdsm/vdsm.log
> >> ) :
> >>
> >> 2017-04-08 14:12:22,208-0400 WARN  (jsonrpc/6) [storage.LVM] lvm pvs
> failed:
> >> 5 [] ['  Failed to find physical volume
> >> "/dev/mapper/360014050eacbb3c8b21428fb6683f074".'] (lvm:325)
> >>
> >> 4. New warning:
> >> 2017-04-08 14:13:17,327-0400 WARN  (check/loop) [storage.asyncutils]
> Call
> >>  >> /dev/0eb0f05c-0507-4f02-ba13-b8d865538d7a/metadata running
> >> next_check=4295183.36 at 0x2fb9190>> delayed by 1.43 seconds
> >> (asyncutils:138)
> >>
> >>
> >> Do we know when it began?
> >
> > Yes. I linked to the patch that seems to have started this.
>
> This should be easy to revert
> Working on it
> Then we’ll see what exactly is the culprit
>
> >
> >
> >
> > --
> > Barak Korren
> > bkor...@redhat.com
> > RHCE, RHCi, RHV-DevOps Team
> > https://ifireball.wordpress.com/
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> >
> >
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] REST API data aggregation

[Martin Betak]

On Mon, Mar 27, 2017 at 4:49 PM, Greg Sheremeta  wrote:

> On Mon, Mar 27, 2017 at 8:59 AM, Tomas Jelinek 
> wrote:
>
>>
>>
>> On Mon, Mar 27, 2017 at 1:32 PM, Juan Hernández 
>> wrote:
>>
>>> On 03/27/2017 01:03 PM, Tomas Jelinek wrote:
>>> >
>>> >
>>> > On Mon, Mar 27, 2017 at 11:21 AM, Juan Hernández >> > > wrote:
>>> >
>>> > Top posting, sorry.
>>> >
>>> > There are a few things I'd like to clarify, regarding this subject:
>>> >
>>> > 1. Data aggregation, as requested now by Tomas, and by other
>>> people in
>>> > the past.
>>> >
>>> > We used to have that 'detail' parameter, to aggregate certain very
>>> > specific types of data, in particular to aggregate VM disks and
>>> NICs. We
>>> > removed that in version 4 of the API because the implementation was
>>> > extremely inefficient, from the engine point of view. An innocent
>>> > request like this:
>>> >
>>> >   GET /ovirt-engine/api/vms?detail=+disks,+nics
>>> >
>>> > Would generate, with the implementation we used to have, 1 query
>>> for the
>>> > VMs and then as many queries for disks and NICs as VMs in the
>>> system. In
>>> > our scale test environments, for example, with approx 4000 VMs and
>>> 1
>>> > disks, that would take more than 20 hours to execute.
>>> >
>>> > In addition, we didn't have in the past any mechanism to make this
>>> > available in a generic one, because there was no knowledge in the
>>> API of
>>> > what are 'details'.
>>> >
>>> > In version 4 of the API we introduced a formal (kind of)
>>> specification
>>> > of the API (a.k.a. the model), and int includes knowledge about
>>> what are
>>> > 'links'. For example, the specification of the VM type contains
>>> this:
>>> >
>>> >   @Link DiskAttachment[] diskAttachments();
>>> >   @Link Nic[] nics();
>>> >
>>> > With this information we are now in a position where we can
>>> implement
>>> > this in a generic way.
>>> >
>>> > We intend to implement this using a mechanism similar to the
>>> existing
>>> > 'detail' parameter:
>>> >
>>> >   GET /ovirt-engine/api/vms/123?follow=disk_attachments,nics
>>> >
>>> > The naive implementation of this is to let the API call itself. For
>>> > example, when the user requests to follow the 'disk_attachments'
>>> detail
>>> > the API can just call itself to get that:
>>> >
>>> >   GET /ovirt-engine/api/vms/123/disk_attachments
>>> >
>>> > However, we can't use that naive approach, if we do we end with the
>>> > 1+C*N query problem described before. We need to use specific
>>> > implementations for certain frequent use cases, like
>>> VMs+disks+nics, and
>>> > that needs work in the API and in the backend.
>>> >
>>> > Tomas, if you want to help moving this forward, please open a RFE
>>> and
>>> > makes sure it gets attention.
>>>
>>
>> ok, opened: https://bugzilla.redhat.com/show_bug.cgi?id=1436206
>> Will try to get it done soon.
>>
>
> Forgive me if this is radical, but has anyone thought of / discussed using
> a NoSQL alternative to our very normalized SQL db as a way to avoid the
> problem of aggregating details? Using mongodb as an example, embed some of
> the smaller objects, and there's no cost of aggregation there. IIRC, Doctor
> REST uses mongo under the hood.
>
> https://docs.mongodb.com/manual/tutorial/model-embedded-one-to-many-
> relationships-between-documents/
>

Greg, I really appreciate your enthusiasm for NoSQL technologies but we
have to distinguish between the functional requirements for cache of
schemaless frontend-optimized projections (as in the Dr. Rest case) and the
main application data store, supporting transactional business logic. I
don't believe the current architecture could survive the weak
eventual-consistency guarantees of not fully transactional DB (such as
Postgres is) :-)

Best regards,

Martin


>
>
> Greg
>
>
>>
>>
>>> >
>>> >
>>> > This sounds pretty good! I will open, but since we are talking already
>>> > here I'll just use the opportunity to clarify the topic more and than
>>> > I'll open the BZ.
>>> >
>>> > What I can imagine is the GetAllVmsQuery will accept in params also the
>>> > list of details it should provide. Than, the GetAllVmsQuery will
>>> > implement the efficient way of retrieving this info as well.
>>> >
>>> > So, from the API perspective, it will be about taking the
>>> > ?follow= part and passing it to the backend query params.
>>> >
>>> > What you think?
>>> >
>>>
>>> Exactly, that is the point! The API by itself can't optimize database
>>> queries, all it can do is call the backend. It is the backend that has
>>> the opportunity and possibility to send optimized queries to the
>>> database.
>>>
>>> For other less common things we can use the naive approach, and
>>> implement the aggregation in the API itself. But for common use cases,
>>> like VM+disks+nics, we need to do it in an efficient way.
>>>
>>> >

Re: [ovirt-devel] proposing Filip Krepinsky as moVirt maintainer

[Martin Betak]

Since his joining the project, moVirt has seen a *massive* spike in
activity.
In his many patches implementing features, bug fixes and major refactorings
he has shown great understanding of the project,
the Android operating system and oVirt in general.

With the above in mind this gives a *strong +2* from my side!

Please keep up the good work, Filip :-)

Best regards,

Martin

On Tue, Feb 28, 2017 at 1:34 PM, Tomas Jelinek  wrote:

> Hi All,
>
> it is my pleasure to propose Filip Krepinsky (suomiy) as the new
> maintainer of the moVirt project!
>
> Filip has joined the moVirt project 11th November 2015 by this commit:
> https://github.com/matobet/moVirt/commit/d37e71a436e38b6342903b991a30a8
> f358655ffb
>
> and since than with his 140 commits became the contributor with second
> most patches (from 13 contributors in total).
> He have also contributed to the aSPICE project and was active in finding
> issues in the oVirt core (around dashboard).
>
> His patches are well crafted and I'm absolutely sure he will do great as a
> maintainer.
>
> have a nice day,
> Tomas
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [Call for Vote] moVirt as a Full oVirt Project

[Martin Betak]

+1 Albeit obviously biased :-), I think given the continuously growing user 
base and stability, it is well deserved.

- Original Message -
> From: "Brian Proffitt" 
> To: "devel" , bo...@ovirt.org
> Cc: "Michal Skrivanek" 
> Sent: Monday, November 21, 2016 6:07:11 PM
> Subject: [ovirt-devel] [Call for Vote] moVirt as a Full oVirt Project
> 
> All:
> 
> The moVirt Project was initially accepted as an oVirt incubator project in
> February 2015. It has been a successful subproject for quite some time and
> it is well due for being accepted as a full oVirt project. I believe it is
> appropriate to post a Call for Vote on the Devel and Board lists.
> 
> http://www.ovirt.org/develop/projects/project-movirt/
> 
> A “healthy” project, as determined by the oVirt Board, can be found at
> http://www.ovirt.org/develop/projects/adding-a-new-project/
> 
> Voting will be open until 1200 UTC Nov. 30, 2016. A net total of +7 votes
> should be received to formalize this project as an full oVirt project.
> Please use the following vote process:
> 
> +1
> Yes, agree, or the action should be performed. On some issues, this vote must
> only be given after the voter has tested the action on their own system(s).
> 
> ±0
> Abstain, no opinion, or I am happy to let the other group members decide this
> issue. An abstention may have detrimental affects if too many people
> abstain.
> 
> -1
> No, I veto this action. All vetos must include an explanation of why the veto
> is appropriate. A veto with no explanation is void.
> 
> Thank you!
> 
> Brian Proffitt
> Principal Community Analyst
> Open Source and Standards
> @TheTechScribe
> 574.383.9BKP
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Proposing Martin Sivak as a Backend Maintainer for SLA & Scheduling

[Martin Betak]

- Original Message -
> From: "Arik Hadas" 
> To: "Doron Fediuck" 
> Cc: "devel" 
> Sent: Monday, September 19, 2016 3:52:53 PM
> Subject: Re: [ovirt-devel] Proposing Martin Sivak as a Backend Maintainer for 
> SLA & Scheduling
> 
> 
> 
> - Original Message -
> > 
> > 
> > On Mon, Sep 19, 2016 at 3:07 PM, Roy Golan < rgo...@redhat.com > wrote:
> > 
> > 
> > 
> > 
> > 
> > On 19 September 2016 at 15:12, Doron Fediuck < dfedi...@redhat.com > wrote:
> > 
> > 
> > 
> > Hi All,
> > 
> > Martin Sivak has been working on the oVirt engine since June 2013.
> > His main focus over the years was around MoM, external scheduler,
> > hosted-engine, and
> > related engine code.
> > In the past year Martin took over many of the oVirt scheduler related work
> > and did a lot of improvements there. In total Martin has around 170 engine
> > patches already merged.
> > Within these you will find improving the scheduler's notifications and
> > error
> > handling, rewriting the pending resource mechanism which resolved many
> > issues, VM affinity, affinity labels and multiple improvements for our
> > scheduling policies and hosted engine.
> > I would like to thank Martin for his great contribution and hope for many
> > more patches to come :)
> > 
> > I would also like to propose Martin as an engine maintainer for SLA and
> > Scheduling.
> > Thanks, Doron
> > 
> > +1 Actually +2 :)
> > 
> > +1
> 
> +1
> 

+1
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> > 
> > 
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> > 
> > 
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Best (and up-to-date) practices for developers?

[Martin Betak]

Yeah definitely +1.

Even after being with the project for some time, I often wonder about things
like in what order should fields in class be based on their function (as in
logger, static sonstants, @Injected dependencies, actual instance fields...)
so such document could answer questions like this (that are not encoded in 
existing checkstyle rules) or provide general high-level guidance throughout
the code base - at least for things that don't change that much. For example
I believe that the process to invoke a command (or a query) from the Backend 
via the VdcActionType, CommandFactory .etc hasn't changed that much, 
and probably won't change in the foreseeable future :-)

Martin B.

- Original Message -
> From: "Phillip Bailey" 
> To: "Martin Sivak" 
> Cc: "engine-de...@ovirt.org" 
> Sent: Monday, September 19, 2016 1:54:23 PM
> Subject: Re: [ovirt-devel] Best (and up-to-date) practices for developers?
> 
> +1 for the idea if it doesn't already exist. It would ease development for
> those already working on the project, but it would also make it easier for
> others who would like to contribute to it, but aren't comfortable with
> navigating such a large code base.
> 
> -Phillip Bailey
> 
> On Mon, Sep 19, 2016 at 3:56 AM, Martin Sivak < msi...@redhat.com > wrote:
> 
> 
> Hi,
> 
> I just wanted to find out if we have some documentation for the
> developers that would give hint on different aspects of the engine
> development in terms of coding.
> 
> Like:
> - code style (we have the config in sources)
> - what libraries are (not) allowed (guava..ehm..)
> - how should CDI be used (see https://gerrit.ovirt.org/#/c/63747/ )
> - what files need to be touched to properly support translations
> - ...
> 
> 
> It would be cool to have something brief we can search easily. I know
> it can get outdated fast, but if we use it only for the important
> stuff and review it for each major release (as the major rules do not
> change much) we should be able to keep it usable.
> 
> --
> Martin Sivák
> SLA / oVirt
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Type Safety on the Frontend (refactoring of all async queries)

[Martin Betak]

Hi All,

today a *major* frontend refactoring was merged [1].
Its purpose was to remove some of the most significant artifacts of legacy C# 
code. 

Huge thanks to Vojtech and Alex for heavenly patience during code review!

For a complete description please see the commit message, but here are
some highlights for programmers that need to touch FE code:

1) AsyncQuery, AsyncCallback and Converter are now generic and typesafe 
   (formerly INewAsyncCallback and IAsyncConverter).

2) It is no longer possible (or desirable) to PASS MODEL to queries. 
   From within Model sublasses `new AsyncQuery<>(...)` will have the correct 
value set 
   (see patch for impl details) and if you want to run a query on a different 
model 
   (so the infrastructure would set the progress spinner on that model 
instead), 
   usually for the purpose of a ListModel showing a popup window - where we 
want to 
   display the spinner on the dialog window instead, you can use the 
   `myWindowModel.asyncQuery(...)` helper factory to create query with model 
set 
   to myWindowModel.

   // note: this also has the nice side-effect of eliminating the harmful 
pattern of using 
   // the model value from within callbacks by casting it to surrounding model 
type (while that 
   // value is already available by feature of java inner classes)

3) AsyncDataProvider's API methods now deal with strongly typed queries. This 
is facilitated 
   through usage of strongly typed Converters (please see some defined 
converters -> usually 
   you just need a CastingConverter<> or a ListConverter<>). 
   Please use this architectural layer of strongly typed AsyncDataProvider to 
your advantage. 
   (as plain  Fronted.runQuery does not provide any guarantees and leaves you 
at risk of 
   mis-casting the return value).

If you have any more questions please feel free to ask me or look at the patch 
for details
(warning: it's quite big!). 

This kind of major change necessarily touched all areas of the code (those 
issuing queries...)
and there is of course some probability that something may break. But I believe 
this risk is 
well outweight by the benefits. Nevertheless if you find any issue please let 
me know and 
we can fix it together.

Thanks again to anyone who contributed to this noble effort.

Best regards :-)

Martin

[1] https://gerrit.ovirt.org/#/c/60822/
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] DEV engine-setup fails with [ ERROR ] Failed to execute stage 'Misc configuration': 'OVESETUP_DWH_CORE/enable'

[Martin Betak]

Hi all,

on latest master of engine I'm encountering the following error during
engine setup:

2016-04-19 13:08:59 DEBUG otopi.context context._executeMethod:128 Stage misc 
METHOD 
otopi.plugins.ovirt_**FILTERED**_setup.ovirt_**FILTERED**.config.dwh_database.Plugin._miscDWHConfig
2016-04-19 13:08:59 DEBUG otopi.context context._executeMethod:142 method 
exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 131, in 
_executeMethod
if method['condition']():
  File 
"/home/mbetak/work/ovirt-**FILTERED**s/master/share/ovirt-**FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-setup/ovirt-**FILTERED**/config/dwh_database.py",
 line 48, in 
self.environment[o**FILTERED**cons.DWHCoreEnv.ENABLE]
KeyError: 'OVESETUP_DWH_CORE/enable'
2016-04-19 13:08:59 ERROR otopi.context context._executeMethod:151 Failed to 
execute stage 'Misc configuration': 'OVESETUP_DWH_CORE/enable'

do I need DWH installed now also as a part of DevEnv? Or is there other setup 
dependency I'm missing?

Thanks for any help.

Best regards,

Martin
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Proposing Martin Betak as oVirt UI maintainer

[Martin Betak]

Thanks everyone for their expressed support!

I hope I'll be able to continue working towards improving our existing 
frontend codebase and whatever may serve as the UI in the future :-)

Best regards,

Martin B.

- Original Message -
> From: "Tomas Jelinek" 
> To: "Martin" 
> Cc: "devel" 
> Sent: Monday, February 8, 2016 10:19:34 AM
> Subject: Re: [ovirt-devel] Proposing Martin Betak as oVirt UI maintainer
> 
> congrats! :)
> 
> - Original Message -
> > From: "David Caro" 
> > To: "Vojtech Szocs" 
> > Cc: "Michal Skrivanek" , "devel" 
> > Sent: Monday, February 8, 2016 10:11:30 AM
> > Subject: Re: [ovirt-devel] Proposing Martin Betak as oVirt UI maintainer
> > 
> > On 02/03 09:00, Vojtech Szocs wrote:
> > > Hello, UI folks!
> > > 
> > > I'd like to propose Martin Betak as oVirt UI maintainer.
> > 
> > 
> > I've added him to the ovirt-engine-webadmin gerrit group, let me know if
> > there's anything else needed
> > 
> > > 
> > > Over time, Martin made some significant UI contributions:
> > > 
> > > - improving UiCommon by utilizing Java generics [1]
> > > 
> > > - adding GinUiBinder [2] to allow @Inject'ing GIN-managed
> > >   objects into GWT widgets created in ui.xml templates
> > > 
> > > - upgrade GWT version to 2.6.1 [3] for both oVirt webapps
> > > 
> > > - providing excellent feedback and ideas on oVirtJS project
> > >   [4] while demonstrating outstanding JavaScript knowledge
> > > 
> > > [1] https://gerrit.ovirt.org/#/c/32907/
> > > [2] https://gerrit.ovirt.org/#/c/34954/
> > > [3] https://gerrit.ovirt.org/#/c/32135/
> > > [4] https://gerrit.ovirt.org/#/c/49466/
> > > 
> > > Personal note: Martin is familiar with oVirt GWT UI infra
> > > as a whole (GWT-P component model, advanced GWT features,
> > > UiCommon integration etc). His grasp on JS and its current
> > > eco-system is very solid.
> > > 
> > > Regards,
> > > Vojtech
> > > ___
> > > Devel mailing list
> > > Devel@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
> > 
> > --
> > David Caro
> > 
> > Red Hat S.L.
> > Continuous Integration Engineer - EMEA ENG Virtualization R&D
> > 
> > Tel.: +420 532 294 605
> > Email: dc...@redhat.com
> > IRC: dcaro|dcaroest@{freenode|oftc|redhat}
> > Web: www.redhat.com
> > RHT Global #: 82-62605
> > 
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Changing the name of VDSM in oVirt 4.0.

[Martin Betak]

- Original Message -
> From: "Yaniv Dary" 
> To: "Martin Perina" 
> Cc: "Martin Betak" , "devel" 
> Sent: Tuesday, January 26, 2016 4:55:10 PM
> Subject: Re: [ovirt-devel] Changing the name of VDSM in oVirt 4.0.
> 
> Yaniv Dary
> Technical Product Manager
> Red Hat Israel Ltd.
> 34 Jerusalem Road
> Building A, 4th floor
> Ra'anana, Israel 4350109
> 
> Tel : +972 (9) 7692306
> 8272306
> Email: yd...@redhat.com
> IRC : ydary
> 
> 
> On Tue, Jan 26, 2016 at 5:47 PM, Martin Perina  wrote:
> 
> >
> >
> > - Original Message -
> > > From: "Martin Betak" 
> > > To: "Yaniv Dary" 
> > > Cc: "devel" 
> > > Sent: Tuesday, January 26, 2016 4:41:29 PM
> > > Subject: Re: [ovirt-devel] Changing the name of VDSM in oVirt 4.0.
> > >
> > > +1
> > >
> > > Honestly I think that any name is better and more descriptive than
> > "VDSM" :)
> > > "host-agent" seems appropriate to me.
> >
> 
> I think the 'ovirt-' initial is needed to mark this is part of the upstream
> project.

Sure, I meant to silently imply the "ovirt-" prefix :)

> 
> 
> > +1
> >
> >
> > But more important is to align engine and VDSM version. Wwe build them
> > together for one release, so they should have the same release version,
> > for example in oVirt 4.0 we should have
> >
> > ovirt-engine 4.0.0
> > ovirt-host-agent 4.0.0
> >
> >
> > Martin Perina
> >
> > >
> > > Best regards,
> > >
> > > Martin B.
> > >
> > > - Original Message -
> > > > From: "Yaniv Dary" 
> > > > To: "devel" 
> > > > Sent: Tuesday, January 26, 2016 4:29:46 PM
> > > > Subject: [ovirt-devel] Changing the name of VDSM in oVirt 4.0.
> > > >
> > > > Hi,
> > > > I wanted to bring this up to get feedback on this proposed change. VDSM
> > > > doesn't align to other project under the oVirt umbrella like
> > ovirt-engine,
> > > > ovirt-node, ovirt-guest-agent etc..
> > > >
> > > > I suggest for ease of use and tracking we change the versioning to
> > align to
> > > > the engine (4.0.0 in oVirt 4.0 GA) to make it easy to know which
> > version
> > > > was
> > > > in which release and also change the package naming to something like
> > > > ovirt-host-manager\ovirt-host-agent.
> > > >
> > > > What do you think on this? What do you think the name should be?
> > > > Yaniv Dary
> > > > Technical Product Manager
> > > > Red Hat Israel Ltd.
> > > > 34 Jerusalem Road
> > > > Building A, 4th floor
> > > > Ra'anana, Israel 4350109
> > > >
> > > > Tel : +972 (9) 7692306
> > > > 8272306
> > > > Email: yd...@redhat.com IRC : ydary
> > > >
> > > > ___
> > > > Devel mailing list
> > > > Devel@ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > ___
> > > Devel mailing list
> > > Devel@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
> > >
> >
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Changing the name of VDSM in oVirt 4.0.

[Martin Betak]

+1 

Honestly I think that any name is better and more descriptive than "VDSM" :)
"host-agent" seems appropriate to me.

Best regards,

Martin B.

- Original Message -
> From: "Yaniv Dary" 
> To: "devel" 
> Sent: Tuesday, January 26, 2016 4:29:46 PM
> Subject: [ovirt-devel] Changing the name of VDSM in oVirt 4.0.
> 
> Hi,
> I wanted to bring this up to get feedback on this proposed change. VDSM
> doesn't align to other project under the oVirt umbrella like ovirt-engine,
> ovirt-node, ovirt-guest-agent etc..
> 
> I suggest for ease of use and tracking we change the versioning to align to
> the engine (4.0.0 in oVirt 4.0 GA) to make it easy to know which version was
> in which release and also change the package naming to something like
> ovirt-host-manager\ovirt-host-agent.
> 
> What do you think on this? What do you think the name should be?
> Yaniv Dary
> Technical Product Manager
> Red Hat Israel Ltd.
> 34 Jerusalem Road
> Building A, 4th floor
> Ra'anana, Israel 4350109
> 
> Tel : +972 (9) 7692306
> 8272306
> Email: yd...@redhat.com IRC : ydary
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Push notifications in 4.0 backend

[Martin Betak]

- Original Message -
> From: "Marek Libra" 
> To: "Martin Perina" 
> Cc: "Piotr Kliczewski" , "Michal Skrivanek" 
> , "engine-de...@ovirt.org"
> 
> Sent: Wednesday, November 25, 2015 5:19:31 PM
> Subject: Re: [ovirt-devel] Push notifications in 4.0 backend
> 
> 
> 
> - Original Message -
> > From: "Martin Perina" 
> > To: "Eli Mesika" 
> > Cc: "Piotr Kliczewski" , "engine-de...@ovirt.org"
> > , "Michal Skrivanek"
> > 
> > Sent: Wednesday, 25 November, 2015 11:20:49 AM
> > Subject: Re: [ovirt-devel] Push notifications in 4.0 backend
> > 
> > 
> > 
> > - Original Message -
> > > From: "Eli Mesika" 
> > > To: "Vojtech Szocs" 
> > > Cc: "Piotr Kliczewski" , "Michal Skrivanek"
> > > , "engine-de...@ovirt.org"
> > > 
> > > Sent: Wednesday, November 25, 2015 10:42:35 AM
> > > Subject: Re: [ovirt-devel] Push notifications in 4.0 backend
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Vojtech Szocs" 
> > > > To: "Martin Betak" 
> > > > Cc: "engine-de...@ovirt.org" , "Piotr Kliczewski"
> > > > , "Michal Skrivanek"
> > > > 
> > > > Sent: Monday, November 23, 2015 6:22:45 PM
> > > > Subject: Re: [ovirt-devel] Push notifications in 4.0 backend
> > > > 
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Martin Betak" 
> > > > > To: "Vojtech Szocs" 
> > > > > Cc: "Einav Cohen" , "engine-de...@ovirt.org"
> > > > > , "Roy Golan" ,
> > > > > "Roman Mohr" , "Michal Skrivanek"
> > > > > ,
> > > > > "Piotr Kliczewski" ,
> > > > > "Tomas Jelinek" , "Alexander Wels"
> > > > > ,
> > > > > "Greg Sheremeta" ,
> > > > > "Scott Dickerson" , "Arik Hadas"
> > > > > ,
> > > > > "Allon Mureinik" ,
> > > > > "Shmuel Melamud" , "Jakub Niedermertl"
> > > > > , "Marek Libra"
> > > > > , "Martin Perina" , "Alona
> > > > > Kaplan"
> > > > > , "Martin Mucha"
> > > > > 
> > > > > Sent: Thursday, November 19, 2015 1:53:07 PM
> > > > > Subject: Re: Push notifications in 4.0 backend
> > > > > 
> > > > > Hi All,
> > > > > 
> > > > > I have created a PoC patch [1] demonstrating the idea of annotating
> > > > > basic CRUD commands to publish CDI events. It is not meant as 100%
> > > > > solution, but as a simplification of the common use cases when
> > > > > one would inject CDI event with given qualifier and fire it after
> > > > > successful completion of transaction.
> > > > 
> > > > The patches (mentioned below) look interesting.
> > > > 
> > > > At this point, it would be great if backend core maintainers
> > > > voiced their opinions on the general idea of firing CDI events
> > > > in response to important actions happening on Engine, such as
> > > > backend commands being executed. So, what do you think guys?
> > > 
> > > +1
> > > 
> > > I am for it, I think it may reduce load from our DB
> > 
> > +1
> > 
> The load reduction can be achieved and seems like not a big deal to implement
> it.
> +1

Yes, the DB load reduction is perhaps the bigest boon of this effort :-).

My first patch makes very convenient to fire notification from basic CRUD 
management
operations that manipulate main business entities. The next (and probably more
complicated) step will be a refactoring of the monitoring code to support CDI 
(injections and events) and have it fire events of VmDynamic payload. Then on
the event we can have listening scheduler (balancing), HA, ... and other parts
that not only won't have to issue expensive DB queries (e.g. GetVmsRunningOnVds)
but also can be more decoupled from monitoring (e.g. VdsEventListener).

But this will be a little bit more involved than just annotating all CRUD 
commands
with one annotation :-)

> 
> > > 
> > > > 
> > > >

Re: [ovirt-devel] Push notifications in 4.0 backend

[Martin Betak]

Hi All,

I have created a PoC patch [1] demonstrating the idea of annotating
basic CRUD commands to publish CDI events. It is not meant as 100%
solution, but as a simplification of the common use cases when
one would inject CDI event with given qualifier and fire it after
successful completion of transaction.

The usage of this annotations is demonstrated on several basic CRUD
commands in [2] on StoragePool, VDS, VDSGroup, .etc

As always, comments and suggestions are very welcome!

Thank you and best regards,

Martin 

[1] infra: https://gerrit.ovirt.org/#/c/48696/
[2] usage: https://gerrit.ovirt.org/#/c/48697/



- Original Message -
> From: "Vojtech Szocs" 
> To: "Martin Betak" , "Einav Cohen" 
> Cc: "engine-de...@ovirt.org" , "Roy Golan" 
> , "Roman Mohr" ,
> "Michal Skrivanek" , "Piotr Kliczewski" 
> , "Tomas Jelinek"
> , "Alexander Wels" , "Greg Sheremeta" 
> , "Scott
> Dickerson" 
> Sent: Friday, November 13, 2015 9:31:45 PM
> Subject: Re: Push notifications in 4.0 backend
> 
> Hi everyone,
> 
> assuming that 4.0 UI will be based on the existing GWT technology,
> I'd like to improve two things which I believe are very important:
> 
> #1 goal: improve GWT compilation times
>- don't use standard GWT i18n mechanism which yields separate
>  permutation vector, but use our own i18n mechanism instead
>- in practice, compiling for X browsers and Y languages should
>  result in GWT compiler processing X permutations (not X * Y)
>- this will also directly impact GWT debug performance, making
>  GWT debugging experience less painful for developers
> 
> #2 goal: improve UX related to backend operations
>- replace periodic polling with push notifications that inform
>  UI of changes in oVirt "system" as they happen
>- in practice, UI becomes reactive instead of proactive, which
>  has several benefits (reduced HTTP load on Engine being the
>  most obvious one)
> 
> So what Martin wrote in email below directly relates to #2 goal.
> 
> Push notifications improve user experience regardless of specific
> UI technology, regardless of whether we improve existing REST API
> (e.g. introduce data aggregations) or not.
> 
> For me, it's a big +1.
> 
> Having BLL commands firing CDI events upon execution makes sense.
> That said, I'd suggest to start with a simple implementation and
> proceed from there.
> 
> What Martin suggested:
> 
>   void onVmChanged(@Observes @Updated VM vm)
> 
> could be even simplified into:
> 
>   void onCommandExecuted(@Observes @CommandExecuted UpdateVmCommand cmd)
> 
> and still it would bring value to the general idea, which is the
> ability to detect changes in oVirt "system" as they happen along
> with the ability to react upon such changes.
> 
> I'm interested what Engine backend maintainers' thoughts are.
> 
> Regards,
> Vojtech
> 
> 
> - Original Message -
> > From: "Martin Betak" 
> > To: "engine-de...@ovirt.org" 
> > Cc: "Roy Golan" , "Roman Mohr" ,
> > "Michal Skrivanek" ,
> > "Piotr Kliczewski" , "Vojtech Szocs"
> > , "Tomas Jelinek" 
> > Sent: Wednesday, November 11, 2015 4:34:11 PM
> > Subject: Push notifications in 4.0 backend
> > 
> > Hi All,
> > 
> > I would like to take this opportunity to start a discussion
> > about the possibility of implementing a user facing change notifications.
> > 
> > The benefit of this would be to remove the need for periodic polling
> > from frontends and other services that consume our REST API.
> > 
> > Also implmenting a common infrastructure on the backend for event
> > notifications (e.g. CDI events) would further reduce the internal
> > need for polling the DB by the backend itself, maybe even reducing
> > the need to use DB for some things and just keep them in memory and
> > updated by CDI event observers.
> > 
> > There are many solutions how to provide the user-facing part of the
> > notifications:
> > Doctor Rest, MQTT, websocket, server-sent events, ... .  Ideally these
> > notifications
> > should be consumable both by web browser (webadmin/userportal) but also by
> > other services (such as ManageIQ), or other REST clients such as moVirt
> > android client.
> > 
> > But regardless of the chosen user-facing transport, I believe a common
> > infrastructure
> > can be implemented on the BLL layer with the usage of CDI events fired from
> > 

[ovirt-devel] Push notifications in 4.0 backend

[Martin Betak]

Hi All,

I would like to take this opportunity to start a discussion
about the possibility of implementing a user facing change notifications.

The benefit of this would be to remove the need for periodic polling 
from frontends and other services that consume our REST API.

Also implmenting a common infrastructure on the backend for event
notifications (e.g. CDI events) would further reduce the internal
need for polling the DB by the backend itself, maybe even reducing 
the need to use DB for some things and just keep them in memory and
updated by CDI event observers.

There are many solutions how to provide the user-facing part of the 
notifications:
Doctor Rest, MQTT, websocket, server-sent events, ... .  Ideally these 
notifications 
should be consumable both by web browser (webadmin/userportal) but also by 
other services (such as ManageIQ), or other REST clients such as moVirt android 
client.

But regardless of the chosen user-facing transport, I believe a common 
infrastructure
can be implemented on the BLL layer with the usage of CDI events fired from 
commands.
I see 2 major sources of changes in the engine (please correct me if this is 
wrong):
1) CRUD & management commands
2) Vms/Hosts monitoring

the changes originating from 2) are AFAIK very localized and not so numerous so 
a manual
firing of appropriate events for VMs and Hosts could be added here.

The 1) case is more extensive in terms of required code changes. While a manual 
solution
would still be feasible, I believe there is place for a more 
automated/declarative way. 

One solution for 1) that comes to my mind are simple command-level annotations 
covering the
Created, Updated, Removed (C, U, and D from CRUD) cases. The goal here is to 
fire the
appropriate CDI events when an entity is created/updated/deleted. Since 
commands usually
contain getters for entities they work with (such as getVm(), getVds(), 
getStorageDomain() ...)
It should be sufficient for the most common simple cases (of course this will 
not cover 
everything) to use annotation @Creates, @Updates, @Removes on the commands 
classes, where
parameters of the annotation should specify the getter method that returns the 
affected entity
(VM/VDS/StorageDomain...). This could be specified by the entity class token or 
method name 
(depending on the level of "magic" one prefers :-) and the CommandBase 
infrastructure would
then collect those annotations and upon successful completion of the command 
fire the 
appropriate events.

Example #1:
@Updates('getVm') // or @Updates(VM.class)?
public class UpdateVmCommand<...>  extends VmManagementComandBase ...

Note that since Java 8 we have repeatable annotations so we can have more 
complex commands
that affect more entities.

Example #2:
@Updates(Vm.class)
@Updates(VmTemplate.class)
// possibly also some @Creates and @Removes annotations or their combination
public class ContrivedExampleCommand extends SomeCommandBase

the infrastructure would then look upon successful completion of the command on 
the getVm() 
and getVmTemplate() methods, invoke them, determine the resulting types of 
entities VM and VmTemplate
and since the annotations used were @Updates fire CDI event equivalent to

  @Inject
  @Updated // our custom CDI qualifier
  Event vmChangedEvent;

and anologously for VmTemplate.

But regardless of the exact implementation of the CDI event firing: whether 
manual, the above 
proposal, or some crazy usage of AspectJ - the interface for the rest of the 
code should always
be the like this:

public void onVmChanged(@Observes @Updated VM vm) {
// 
}

On top of this, I believe, we can build the user-facing part of push 
notifications and also
improve our existing backend code.

Thank you for reading this long email and I welcome any comments or 
counter-proposals you
might have on this topic :-)

Best regards,

Martin
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ATN] Introduction of RESTAPI metamodel

[Martin Betak]

- Original Message -
> From: "Juan Hernández" 
> To: "Roman Mohr" 
> Cc: devel@ovirt.org
> Sent: Tuesday, October 27, 2015 2:41:38 PM
> Subject: Re: [ovirt-devel] [ATN] Introduction of RESTAPI metamodel
> 
> On 10/27/2015 02:10 PM, Roman Mohr wrote:
> > 
> > 
> > On Tue, Oct 27, 2015 at 1:45 PM, Juan Hernández  > > wrote:
> > 
> > On 10/27/2015 12:55 PM, Roman Mohr wrote:
> > >
> > >
> > > On Tue, Oct 27, 2015 at 12:16 PM, Juan Hernández  > > 
> > > >> wrote:
> > >
> > > On 10/27/2015 11:28 AM, Roman Mohr wrote:
> > > >
> > > >
> > > > On Tue, Oct 27, 2015 at 10:47 AM, Juan Hernández
> > > > mailto:jhern...@redhat.com>
> > >
> > > > 
> >  > > >
> > > > On 10/27/2015 10:16 AM, Roman Mohr wrote:
> > > > >
> > > > >
> > > > > On Mon, Oct 26, 2015 at 5:32 PM, Juan Hernández
> > > > > mailto:jhern...@redhat.com>
> > >
> > > 
> > >>
> > > > > 
> > >
> > > 
> >  wrote:
> > > > >
> > > > > On 10/26/2015 04:56 PM, Roman Mohr wrote:
> > > > > > Hi Juan,
> > > > > >
> > > > > > The way to specify the contract look pretty
> > clean and
> > > nice.
> > > > > > I would love to read a few words about the big
> > > picture. What
> > > > is the
> > > > > > final scenario?
> > > > > >
> > > > >
> > > > > The motivation for this change is that currently we
> > > don't have
> > > > a central
> > > > > place where the RESTAPI is specified, rather we
> > have several
> > > > different
> > > > > places, using several different technologies:
> > > > >
> > > > > * XML schema for the data model.
> > > > > * JAX-RS for part of the operational model
> > (without the
> > > > parameters).
> > > > > * rsdl_metadata.yaml for the parameters of the
> > > operational model.
> > > > >
> > > > > This makes it difficult to infer information about
> > > > > the
> > > model. For
> > > > > example, the generators of the SDKs have to
> > download the XML
> > > > schema, and
> > > > > the RSDL (which is generated from the JAX-RS
> > interfaces
> > > using
> > > > reflection
> > > > > and combining it with the information from the
> > > > rsdl_metadata.yaml file)
> > > > > and then they have to do their own computations to
> > extract
> > > > what they
> > > > > need.
> > > > >
> > > > > Same happens with the CLI: it has to extract the
> > information
> > > > it needs
> > > > > from the Python code generated for the Python SDK,
> > > > > yet
> > > another
> > > > level of
> > > > > indirection.
> > > > >
> > > > >
> > > > > You are right, that definitely needs to be cleaned up. I
> > > just want to
> > > > > discuss a few points below with you.
> > > > >
> > > > >
> > > > >
> > > > > We are also lacking a comprehensive reference
> > > documentation of the
> > > > > RESTAPI. What we currently have has been written by
> > > hand, and
> > > > gets out
> > > > > of sync very quickly, and we don't even notice.
> > > > >
> > > > >
> > > > > Did you also consider swagger? It is made for exactly
> > > > > that
> > > purpose.
> > > > > I created a demo in [1] which uses resteasy, weld,
> > > hibernate-validator
> > > > > and swagger to demonstrate how to do DRY with jaxrs.
> > > > > Would be great to hear you thoughts on that.
> > > > >
> > > > > And there i

Re: [ovirt-devel] [ATN] Introduction of RESTAPI metamodel

[Martin Betak]

- Original Message -
> From: "Juan Hernández" 
> To: "Roman Mohr" 
> Cc: devel@ovirt.org
> Sent: Tuesday, October 27, 2015 1:45:58 PM
> Subject: Re: [ovirt-devel] [ATN] Introduction of RESTAPI metamodel
> 
> On 10/27/2015 12:55 PM, Roman Mohr wrote:
> > 
> > 
> > On Tue, Oct 27, 2015 at 12:16 PM, Juan Hernández  > > wrote:
> > 
> > On 10/27/2015 11:28 AM, Roman Mohr wrote:
> > >
> > >
> > > On Tue, Oct 27, 2015 at 10:47 AM, Juan Hernández  > > 
> > > >> wrote:
> > >
> > > On 10/27/2015 10:16 AM, Roman Mohr wrote:
> > > >
> > > >
> > > > On Mon, Oct 26, 2015 at 5:32 PM, Juan Hernández
> > > > mailto:jhern...@redhat.com>
> > >
> > > > 
> >  > > >
> > > > On 10/26/2015 04:56 PM, Roman Mohr wrote:
> > > > > Hi Juan,
> > > > >
> > > > > The way to specify the contract look pretty clean and
> > nice.
> > > > > I would love to read a few words about the big
> > picture. What
> > > is the
> > > > > final scenario?
> > > > >
> > > >
> > > > The motivation for this change is that currently we
> > don't have
> > > a central
> > > > place where the RESTAPI is specified, rather we have
> > > > several
> > > different
> > > > places, using several different technologies:
> > > >
> > > > * XML schema for the data model.
> > > > * JAX-RS for part of the operational model (without the
> > > parameters).
> > > > * rsdl_metadata.yaml for the parameters of the
> > operational model.
> > > >
> > > > This makes it difficult to infer information about the
> > model. For
> > > > example, the generators of the SDKs have to download the
> > > > XML
> > > schema, and
> > > > the RSDL (which is generated from the JAX-RS interfaces
> > using
> > > reflection
> > > > and combining it with the information from the
> > > rsdl_metadata.yaml file)
> > > > and then they have to do their own computations to extract
> > > what they
> > > > need.
> > > >
> > > > Same happens with the CLI: it has to extract the
> > > > information
> > > it needs
> > > > from the Python code generated for the Python SDK, yet
> > another
> > > level of
> > > > indirection.
> > > >
> > > >
> > > > You are right, that definitely needs to be cleaned up. I
> > just want to
> > > > discuss a few points below with you.
> > > >
> > > >
> > > >
> > > > We are also lacking a comprehensive reference
> > documentation of the
> > > > RESTAPI. What we currently have has been written by
> > hand, and
> > > gets out
> > > > of sync very quickly, and we don't even notice.
> > > >
> > > >
> > > > Did you also consider swagger? It is made for exactly that
> > purpose.
> > > > I created a demo in [1] which uses resteasy, weld,
> > hibernate-validator
> > > > and swagger to demonstrate how to do DRY with jaxrs.
> > > > Would be great to hear you thoughts on that.
> > > >
> > > > And there is the great swagger-ui [8] to display the
> > documentation
> > > in a
> > > > more human readable way.
> > > >
> > >
> > > Yes, I considered Swagger, and rejected it because it is JSON
> > centric,
> > > and I think JSON isn't as good as Java to represent the
> > contracts of our
> > > RESTAPI.
> > >
> > >
> > > You just write plain jax-rs, swagger just creates a description out
> > > of
> > > it. So  the source defining the contract is pure java (jax-rs with
> > some
> > > swagger annotations for description, etc.).
> > > Or am I missing the point here?
> > >
> > 
> > If I understand correctly the Swagger core is a JSON (or YAML)
> > specification of the API. From that you can generate JAX-RS annotated
> > code, not the other way around. So the specification document that you
> > write is a JSON document.
> > 
> > 
> > You are right, my terminology here was not clear. Swagger is just a
> > specification. Swagger-core and swagger-jaxrs are the ones which can
> > create the documnetation out of JAX-RS resources.
> >  
> > 
> > Alternatively, you can use the Swagger annotations to

Re: [ovirt-devel] Doctor Rest PostgreSQL report

[Martin Betak]

- Original Message -
> From: "Piotr Kliczewski" 
> To: "Martin Betak" 
> Cc: "engine-de...@ovirt.org" , "Vojtech Szocs" 
> , "Einav Cohen"
> 
> Sent: Thursday, October 1, 2015 6:34:02 PM
> Subject: Re: [ovirt-devel] Doctor Rest PostgreSQL report
> 
> On Thu, Oct 1, 2015 at 5:21 PM, Martin Betak  wrote:
> 
> >
> >
> >
> >
> > - Original Message -
> > > From: "Piotr Kliczewski" 
> > > To: "Martin Betak" 
> > > Cc: "engine-de...@ovirt.org" 
> > > Sent: Thursday, October 1, 2015 5:08:39 PM
> > > Subject: Re: [ovirt-devel] Doctor Rest PostgreSQL report
> > >
> > > Martin,
> > >
> > > Looking at the stats you generated I can see that there is almost no
> > > difference for cpu and memory. Load seems to be at the same level for
> > both.
> > > I tried to understand the differences by looking and # of calls and total
> > > time (top 10) but there was almost no difference. The only slight
> > difference
> > > I can see is in avg time. It seems that we haven't generate enough load
> > to
> > > see significant improvement. How many client have you used during
> > testing?
> >
> > This benchmark was without any clients. This was just to demonstrate the
> > load
> > Doctor generates on DB in context of existing DB load by backend (mosty
> > VM/Host
> > monitoring), which was I believe one of Barak's major concerns: how
> > expensive it
> > is when we *don't* have thousands of connected users.
> >
> >
> I am not sure what we are testing here. I can see a lot of queries which
> were generated
> by host monitoring. You said that you had around 200+ hosts and you run
> full dump for
> doctor every 5 seconds. Next you moved to 1 second intervals.
> I seems that host monitoring generated enough load to hide any impact of
> doctor rest
> queries.

Yes, this is exactly what we wanted to estimate. Whether even the most naive
implementation of Doctor connector imposes some significant impact on the 
overall system.
Essentially, how much do we have to pay to get all the benefits of Doctor Rest.

As you say, from these measurements it would seem that the price is negligible.

Martin

> 
> 
> 
> > >
> > > I am not sure how much it would take but what do you think about using
> > > doctor as caching service for the engine and run few tests.
> > > I wonder what would be the result of such PoC.
> >
> > I would like to move to more real world tests as soon as we have at least
> > some working prototype of next gen UI (most significantly the new
> > Dashboard).
> >
> > I understand that the UI discussions (adding Vojtech to CC) have progressed
> > recently so as soon as we have some actual frontend code to work with I
> > would
> > like to hook it up to Doctor and make some end-to-end measurements.
> >
> > Martin
> >
> > >
> > > Thanks,
> > > Piotr
> > >
> > >
> > > On Thu, Oct 1, 2015 at 3:25 PM, Martin Betak  wrote:
> > >
> > > > Hi All,
> > > >
> > > > so I installed a few more plugins to the pgCluu and PostgreSQL itself.
> > > > Now I have also the overall system load and total numbers for specific
> > > > queries.
> > > >
> > > > If you look at the report, database 'engine' and 'Statement statistics'
> > > > we can clearly see that the overwhelming majority of DB time is spent
> > in
> > > > GetVmsRunningOnVds() stored procedure.
> > > >
> > > > Turining on Doctor Rest with 5 second full-dump interval you can see
> > > > that the calls used by DoctorCacheManager (GetAllFromVms,
> > > > GetAllFromVds)
> > > > have hard time to add up to at least 1% of the overall load.
> > > >
> > > > Also you can see the 'System': CPU and memory statistics that those are
> > > > largely
> > > > unaffected by running Doctor service alongside engine.
> > > >
> > > > I also tried setting the full update interval to Doctor to 1 second to
> > see
> > > > how
> > > > this would go - so essentially each second do a full dump of business
> > > > entities -
> > > > and this moved the overall Doctor overhead to ~2.5% of total DB load.
> > > >
> > > > Of course for the UI purposes interval in the range of 3-5 seconds
&g

Re: [ovirt-devel] Doctor Rest PostgreSQL report

[Martin Betak]

- Original Message -
> From: "Piotr Kliczewski" 
> To: "Martin Betak" 
> Cc: "engine-de...@ovirt.org" 
> Sent: Thursday, October 1, 2015 5:08:39 PM
> Subject: Re: [ovirt-devel] Doctor Rest PostgreSQL report
> 
> Martin,
> 
> Looking at the stats you generated I can see that there is almost no
> difference for cpu and memory. Load seems to be at the same level for both.
> I tried to understand the differences by looking and # of calls and total
> time (top 10) but there was almost no difference. The only slight difference
> I can see is in avg time. It seems that we haven't generate enough load to
> see significant improvement. How many client have you used during testing?

This benchmark was without any clients. This was just to demonstrate the load
Doctor generates on DB in context of existing DB load by backend (mosty VM/Host
monitoring), which was I believe one of Barak's major concerns: how expensive it
is when we *don't* have thousands of connected users. 

> 
> I am not sure how much it would take but what do you think about using
> doctor as caching service for the engine and run few tests.
> I wonder what would be the result of such PoC.

I would like to move to more real world tests as soon as we have at least 
some working prototype of next gen UI (most significantly the new Dashboard).

I understand that the UI discussions (adding Vojtech to CC) have progressed
recently so as soon as we have some actual frontend code to work with I would
like to hook it up to Doctor and make some end-to-end measurements.

Martin

> 
> Thanks,
> Piotr
> 
> 
> On Thu, Oct 1, 2015 at 3:25 PM, Martin Betak  wrote:
> 
> > Hi All,
> >
> > so I installed a few more plugins to the pgCluu and PostgreSQL itself.
> > Now I have also the overall system load and total numbers for specific
> > queries.
> >
> > If you look at the report, database 'engine' and 'Statement statistics'
> > we can clearly see that the overwhelming majority of DB time is spent in
> > GetVmsRunningOnVds() stored procedure.
> >
> > Turining on Doctor Rest with 5 second full-dump interval you can see
> > that the calls used by DoctorCacheManager (GetAllFromVms,
> > GetAllFromVds)
> > have hard time to add up to at least 1% of the overall load.
> >
> > Also you can see the 'System': CPU and memory statistics that those are
> > largely
> > unaffected by running Doctor service alongside engine.
> >
> > I also tried setting the full update interval to Doctor to 1 second to see
> > how
> > this would go - so essentially each second do a full dump of business
> > entities -
> > and this moved the overall Doctor overhead to ~2.5% of total DB load.
> >
> > Of course for the UI purposes interval in the range of 3-5 seconds should
> > be more
> > than acceptable in my opinion.
> >
> > As always - questions and remarks are more than welcome :-)
> >
> > Best regards,
> >
> > Martin
> >
> > - Original Message -
> > > From: "Martin Betak" 
> > > To: "Piotr Kliczewski" 
> > > Cc: "engine-de...@ovirt.org" 
> > > Sent: Wednesday, September 30, 2015 4:52:12 PM
> > > Subject: Re: [ovirt-devel] Doctor Rest PostgreSQL report
> > >
> > > - Original Message -
> > > > From: "Piotr Kliczewski" 
> > > > To: "Martin Betak" 
> > > > Cc: "engine-de...@ovirt.org" , "Eli Mesika"
> > > > , "Martin Perina"
> > > > 
> > > > Sent: Wednesday, September 30, 2015 3:29:28 PM
> > > > Subject: Re: Doctor Rest PostgreSQL report
> > > >
> > > > Martin,
> > > >
> > > > For me it would be great to understand how cpu, memory changes over
> > time
> > > > for the engine. I would like to see the same for doctor service.
> > > > I was not able to find it but it would be great to understand how many
> > > > queries there were for both tests and how log it took to run them.
> > >
> > > Yes, right now I'm looking for other tools to provide me exactly with
> > that.
> > > I just wanted to share the preliminary aggregated statistics.
> > >
> > > >
> > > > It would be good to understand implications of running doctor on the
> > same
> > > > machine as engine and on other machine.
> > > >
> > >
> > > Indeed, this is precisely what I'm testing. The attached reports were
> 

Re: [ovirt-devel] Doctor Rest PostgreSQL report

[Martin Betak]

- Original Message -
> From: "Piotr Kliczewski" 
> To: "Martin Betak" 
> Cc: "engine-de...@ovirt.org" , "Eli Mesika" 
> , "Martin Perina"
> 
> Sent: Wednesday, September 30, 2015 3:29:28 PM
> Subject: Re: Doctor Rest PostgreSQL report
> 
> Martin,
> 
> For me it would be great to understand how cpu, memory changes over time
> for the engine. I would like to see the same for doctor service.
> I was not able to find it but it would be great to understand how many
> queries there were for both tests and how log it took to run them.

Yes, right now I'm looking for other tools to provide me exactly with that.
I just wanted to share the preliminary aggregated statistics.

> 
> It would be good to understand implications of running doctor on the same
> machine as engine and on other machine.
> 

Indeed, this is precisely what I'm testing. The attached reports were with
Doctor running on the same machine as the engine.

> Thanks,
> Piotr
> 
> 
> On Wed, Sep 30, 2015 at 3:13 PM, Martin Betak  wrote:
> 
> > Hi All,
> >
> > I performed a stress test using FakeVDSM environment with 200+ hosts
> > and 500+ VMs.
> >
> > Attached are generated HTML reports for this environment.
> > In both cases I tried to simulate some random load using existing
> > webadmin. In the '_doctor' case the simple connector from [1] was
> > running *in addition to* the legacy UI.
> >
> > The used pgCluu tool [2] which may be useful
> > for DB experts for some further insight.
> >
> > I wanted to send this out as soon as possible so we can better analyze
> > our current performance and the possible impact Doctor Rest
> > integration would have on the system.
> >
> > Please feel free to review the attached reports and/or suggest other
> > ways/tools how to better benchmark the DB load caused by Doctor Rest.
> >
> > Thank you very much.
> >
> > Best regards
> >
> > Martin
> >
> > [1] https://gerrit.ovirt.org/#/c/45233/
> > [2] http://pgcluu.darold.net/
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] incoming migration semaphore (and possible SLA consequences)

[Martin Betak]

+1

- Original Message -
From: "Tomas Jelinek" 
To: "Martin Sivak" 
Cc: "engine-de...@ovirt.org" , "Martin Betak" 
, "Francesco Romani" , "Martin Polednik" 
, "Roy Golan" 
Sent: Wednesday, September 30, 2015 1:15:29 PM
Subject: Re: incoming migration semaphore (and possible SLA consequences)



- Original Message -
> From: "Martin Sivak" 
> To: "Tomas Jelinek" 
> Cc: "engine-de...@ovirt.org" , "Martin Betak" 
> , "Francesco Romani"
> , "Martin Polednik" , "Roy Golan" 
> 
> Sent: Wednesday, September 30, 2015 10:41:42 AM
> Subject: Re: incoming migration semaphore (and possible SLA consequences)
> 
> > Please note that we would also like to enrich the scheduler to be aware of
> > max incoming migrations
> > limit thus preventing the storms, but it is a separate topic (no patches
> > around yet).
> 
> This is easy to do, but please make sure you distinguish migration
> from a VM start.
> 
> It might also be better to only use the number of ongoing migrations
> for penalizing the host. In that case the storm would be smaller and
> an overloaded host would still be a possible migration destination
> when there is no better host to use (because of other constraints for
> example).
> 
> There is also the consideration of what happens when a Maintenance
> mode is triggered. The user might want the "storm" to happen to be
> able to save VMs from a compromised host before it fails completely.
> This might work fine when the scoring approach is used.

The combination of re-try on VDSM and a scoring approach on engine sounds good 
to me, since the storms should not happen, but when they do,
they are intentional so VDSM should perform the migration as requested by 
engine.

> 
> Martin
> 
> 
> On Tue, Sep 29, 2015 at 3:35 PM, Tomas Jelinek  wrote:
> > Hi all,
> >
> > as part of the effort to enhance the migration convergence [1] we are
> > proposing a semaphore for incoming migrations [2] (similar to outgoing).
> > It's purpose is to protect the destination host from migration storms where
> > too many migrations are coming to it from different sources.
> >
> > There are basically 3 ways how to do it (with pros/cons):
> >
> > 1: when the destination host refuses the migration, the source host tries
> > it again later (considering no migration will take forever after some time
> > the migration will succeed to start)
> >  (+) pros:
> >(+) if the engine wants to migrate to a specific host (and only to the
> >specific host because user did pick it) than it only sends the command
> >and it will happen (now or later)
> >(+) will not interfere with engine re-runs since the migration will fail
> >only when there is a real issue
> >(+) will be consistent with the current outgoing semaphore (since the
> >outgoing semaphore also waits until has capacity and than starts the
> >migration)
> >(+) VDSM is more autonomous because after the engine sends the command,
> >VDSM will do it even if engine disappears in this moment
> >  (-) cons:
> >(-) re-try on VDSM is not common
> >(-) if the user does not pick a specific destination and he just wants
> >to migrate the machine out of the source, waiting on the destination to
> >have capacity can be wasteful since failing the migration and picking a
> >different host could lead to better results
> >
> > 2: when the destination host refuses the migration, the source host returns
> > to engine "migration failed" and the engine will have to handle it somehow
> >  (+) pros:
> >(+) simpler vdsm (try to migrate, if the destination does not have
> >capacity, fail)
> >(+) lets the engine to pick a different destination host
> >  (-) cons:
> >(-) not consistent with the outgoing migration semaphore (since if there
> >are more VMs waiting for outgoing migrations semaphore, the migration
> >does not fail but waits)
> >(-) engine would have to handle different kinds of migration failed
> >reasons
> >(-) VDSM is not autonomous - if the engine disappears the migration will
> >not be started
> >(-) Here I'm not sure about the consequences to scheduler but I think it
> >would have to be reworked to accommodate the different kinds of re-run.
> >Any ideas from someone more familiar with this? Roy, Martin?
> >
> > 3: (hybrid) - if the user picks a specific host, VDSM will use the first
> > way, if the user will 

Re: [ovirt-devel] Vdsm: extending maintainers team

[Martin Betak]

- Original Message -
> From: "Dan Kenigsberg" 
> To: devel@ovirt.org
> Cc: pklic...@redhat.com
> Sent: Tuesday, August 4, 2015 10:58:53 AM
> Subject: [ovirt-devel] Vdsm: extending maintainers team
> 
> If you follow Vdsm development, you probably have noticed that we are
> short of active maintainers.
> 
> Thankfully, we have have great developers that - in my opinion - can
> fill that gap. I am impressed by the quality of their reviews, their
> endurance, and most importantly - their ability to unbreak whatever
> code they approve.
> 
> I'd like to nominate
> - Nir Soffer - for storage
> - Francesco Romani - for virt

+1

> - Piotr Kliczewski - for infra

+1

> 
> For the mean while, I would like to keep my own single point of merger
> (unless I'm away, of course).
> 
> Active and former maintainers: please approve
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [3.6 deep dive] - VFIO - host device passthrough support

[Martin Betak]

Just to clarify the calendar confusion, this deep dive *WILL* happen
today. You can follow on https://www.youtube.com/watch?v=gUxcuPmbxuY
and ask questions on the #ovirt channel.

> 01:00 PM to 01:45 PM
> (GMT) Greenwich Mean Time - Dublin / Edinburgh / Lisbon / London
> 
> 
>   Where   https://www.youtube.com/watch?v=gUxcuPmbxuY
> 
>   Message Abstract: --- Host device passthrough 
> is a feature that
>   enables users to directly assign some of hypervisor's devices to guest 
> VMs
>   when additional performance or just direct access to hardware is 
> required
>   (i.e. GPU, NIC, ...) Feature page:
>   http://www.ovirt.org/Features/hostdev_passthrough google Hangout link:
>   https://plus.google.com/events/cfof8tbjnq9pgq2n7tr3o11eubo youtube link:
>   https://www.youtube.com/watch?v=gUxcuPmbxuY View your event at
>   
> https://www.google.com/calendar/event?action=VIEW&eid=MjNiNnF2MWIxN2hwaG12bDZyYzJwdTZqOXMgdXNlcnNAb3ZpcnQub3Jn&tok=MTgjYmF6dWxheUByZWRoYXQuY29tZmQ2ZGI1ZjhmY2U0NTVlNDRlY2UxODRlZmM3OWNiNjMwZDAwZTI0Ng&ctz=Asia/Jerusalem&hl=en.
> This event invitation was sent from Yahoo Calendar
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master broken

[Martin Betak]

Hi all,

I am encountering another error when adding host:

2015-06-16 13:57:41,058 ERROR 
[org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] 
(http--0_0_0_0_0_0_0_0-8080-3) [38e7135f] Command 
'org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand' failed: null
2015-06-16 13:57:41,059 ERROR 
[org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] 
(http--0_0_0_0_0_0_0_0-8080-3) [38e7135f] Exception: 
java.lang.NullPointerException
at 
org.ovirt.engine.core.bll.context.DefaultCompensationContext.snapshotEntityInMemory(DefaultCompensationContext.java:140)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.context.DefaultCompensationContext.snapshotNewEntity(DefaultCompensationContext.java:101)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.AddVdsStaticToDb(AddVdsCommand.java:284)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.access$000(AddVdsCommand.java:67)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand$1.runInTransaction(AddVdsCommand.java:112)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand$1.runInTransaction(AddVdsCommand.java:109)
 [bll.jar:]
at 
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210)
 [utils.jar:]
at 
org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.executeCommand(AddVdsCommand.java:109)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1198)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1342)
 [bll.jar:]
at 
org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1949) 
[bll.jar:]
at 
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174)
 [utils.jar:]
at 
org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116)
 [utils.jar:]
at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1366) 
[bll.jar:]
at 
org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:361) 
[bll.jar:]
at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:459) 
[bll.jar:]
at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:441) 
[bll.jar:]
at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:397) 
[bll.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
[rt.jar:1.7.0_75]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
[rt.jar:1.7.0_75]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 [rt.jar:1.7.0_75]
at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_75]
at 
org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72)
 [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
at 
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) 
[jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at 
org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374)
 [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at 
org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114)
 [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final]
at 
org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125)
 [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final]
at 
org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135)
 [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final]
at 
org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36)
 [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
at 
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) 
[jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at 
org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374)
 [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
at 
org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13)
 [bll.jar:]
at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) 
[:1.7.0_75]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 [rt.jar:1.7.0_75]
at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_75]
at 
org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123)
 [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
  

Re: [ovirt-devel] pki-pkcs12-extract.sh failed to execute: Error opening output file: Permission denied

[Martin Betak]

- Original Message -
> From: "Jakub Niedermertl" 
> To: devel@ovirt.org
> Sent: Monday, April 20, 2015 1:03:51 PM
> Subject: [ovirt-devel] pki-pkcs12-extract.sh failed to execute: Error opening 
> output file: Permission denied
> 
> Hi all,
> 
> I can't run engine-setup script, following error is reported:
> 
> [ INFO  ] Creating/refreshing Engine database schema
> [ INFO  ] Upgrading CA
> [ ERROR ] Failed to execute stage 'Misc configuration': Command
> '/home/jakub/target/share/ovirt-engine/bin/pki-pkcs12-extract.sh' failed
> to execute
> 
> and the log says:
> 
> 2015-04-20 12:50:25 DEBUG
> otopi.plugins.ovirt_**FILTERED**_setup.ovirt_**FILTERED**.pki.ssh
> plugin.execute:861 execute-output:
> ('/home/jakub/target/share/ovirt-**FILTERED**/bin/pki-pkcs12-extract.sh',
> '--name=**FILTERED**', '--passin=**FILTERED**', '--cert=-') stdout:
> 
> 
> 2015-04-20 12:50:25 DEBUG
> otopi.plugins.ovirt_**FILTERED**_setup.ovirt_**FILTERED**.pki.ssh
> plugin.execute:866 execute-output:
> ('/home/jakub/target/share/ovirt-**FILTERED**/bin/pki-pkcs12-extract.sh',
> '--name=**FILTERED**', '--passin=**FILTERED**', '--cert=-') stderr:
> Error opening output file -
> -: Permission denied
> Cannot create cert
> 
> 2015-04-20 12:50:25 DEBUG otopi.context context._executeMethod:152 method
> exception
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 142, in
>   _executeMethod
> method['method']()
>   File
>   
> "/home/jakub/target/share/ovirt-**FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-setup/ovirt-**FILTERED**/pki/ssh.py",
>   line 126, in _misc
> ] = self._getSSHPublicKey(self._getEnginePublicKey())
>   File
>   
> "/home/jakub/target/share/ovirt-**FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-setup/ovirt-**FILTERED**/pki/ssh.py",
>   line 73, in _getEnginePublicKey
> '--cert=-',
>   File "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 871, in
>   execute
> command=args[0],
> RuntimeError: Command
> '/home/jakub/target/share/ovirt-**FILTERED**/bin/pki-pkcs12-extract.sh'
> failed to execute
> 2015-04-20 12:50:25 ERROR otopi.context context._executeMethod:161 Failed to
> execute stage 'Misc configuration': Command
> '/home/jakub/target/share/ovirt-**FILTERED**/bin/pki-pkcs12-extract.sh'
> failed to execute
> 
> Anyone experiencing similar issue?

Yes, I have exactly the same issue.

Martin

> 
> Jakub
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Greg Sheremeta is now an ovirt-engine UI maintainer

[Martin Betak]

Congratulations, Greg. Well deserved!

Best regards,

Martin


- Original Message -
> From: "Einav Cohen" 
> To: "Lior Vernia" , "Alona Kaplan" , 
> "Daniel Erez" ,
> "Gilad Chaplik" , "Tomas Jelinek" , 
> "Vojtech Szocs" ,
> "Alexander Wels" , "Kanagaraj" , "Tal 
> Nisan" , "Greg
> Sheremeta" 
> Cc: devel@ovirt.org
> Sent: Wednesday, March 25, 2015 5:58:40 PM
> Subject: [ovirt-devel] Greg Sheremeta is now an ovirt-engine UI maintainer
> 
> Thanks, everyone, for your support.
> Greg is now an oVirt-engine UI maintainer.
> Greg - congratulations!
> 
> 
> Regards,
> Einav
> 
> - Original Message -
> > From: "Einav Cohen" 
> > To: "Lior Vernia" , "Alona Kaplan"
> > , "Daniel Erez" ,
> > "Gilad Chaplik" , "Tomas Jelinek"
> > , "Vojtech Szocs" ,
> > "Alexander Wels" , "Kanagaraj" ,
> > "Tal Nisan" 
> > Cc: in...@ovirt.org, devel@ovirt.org
> > Sent: Wednesday, March 25, 2015 8:59:17 AM
> > Subject: [ovirt-devel] proposing Greg Sheremeta as an ovirt-engine UI
> > maintainer
> > 
> > Hello UI Maintainers,
> > 
> > I would like to propose Greg Sheremeta as an ovirt-engine
> > UI maintainer.
> > 
> > Greg started being actively involved in ovirt on July 2013,
> > contributing over 150 patches (to 'master' alone), including
> > improvements to the branding mechanism (e.g. support cascading
> > resources), grid-column width persistence, phase 1 of the
> > PatternFly styling adoption, dialog-id (tag) based mapping
> > infrastructure, multiple automated-UI-tests (Selenium) related
> > enhancements and the recent overhaul of the tool-tips throughout
> > the code.
> > 
> > Your response would be highly appreciated.
> > 
> > Many thanks in advance.
> > 
> > 
> > Regards,
> > Einav
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> > 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master won't build due to vdsm-jsonrpc-java-client-1.1.0-SNAPSHOT.jar

[Martin Betak]

- Original Message -
> From: "Greg Sheremeta" 
> To: devel@ovirt.org
> Sent: Monday, 23 March, 2015 5:39:43 PM
> Subject: [ovirt-devel] master won't build due to 
> vdsm-jsonrpc-java-client-1.1.0-SNAPSHOT.jar
> 
> I can't build ovirt-engine master due to what looks like some old
> code that got into vdsm-jsonrpc-java-client-1.1.0-SNAPSHOT.jar today.
> 
> Anyone else seeing this?
Yes, since today I have the same issue.
> 
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
> (default-compile) on project vdsbroker: Compilation failure
> [ERROR]
> /home/greg/projects/ovirt-engine-build/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/jsonrpc/JsonRpcVdsServer.java:[467,14]
> error: cannot find symbol
> 
> policy.setHeartbeat(isheartbeat);   // setHeartbeat is not a method
> // only setIncomingHeartbeat or setOutgoingHeartbeat
> 
> 
> Greg
> 
> Greg Sheremeta
> Red Hat, Inc.
> Sr. Software Engineer, RHEV
> Cell: 919-807-1086
> gsher...@redhat.com
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Problems with using gwt-debug

[Martin Betak]

Hi Amit,

this seems to be only and issue of missing `make clean` or removing the install 
prefix directory. I guess debug mode is picking up old files from before when 
the generic parameter wasn't there. 

The debug mode is working for me and other frontend guys with current master.

Best regards,

Martin



- Original Message -
> From: "Einav Cohen" 
> To: "Amit Aviram" , "Martin Betak" 
> Cc: devel@ovirt.org, "Vojtech Szocs" 
> Sent: Sunday, March 8, 2015 5:26:27 PM
> Subject: Re: [ovirt-devel] Problems with using gwt-debug
> 
> adding Martin Betak (author of [1]) which did the last modification
> to SubTabVirtualMachineVirtualDiskPresenter.java, line 38.
> [CC'ing Vojtech]
> 
> @Martin/Vojtech - any idea?
> 
> [1] https://gerrit.ovirt.org/#/c/32907/ [frontend: refactoring: Generify list
> models]
> 
> - Original Message -
> > From: "Amit Aviram" 
> > To: devel@ovirt.org
> > Sent: Sunday, March 8, 2015 7:06:31 AM
> > Subject: [ovirt-devel] Problems with using gwt-debug
> > 
> > Hi all.
> > 
> > I'm having troubles debugging oVirt's GUI with gwt-debug recently- this
> > happens on master, it fails with the following error:
> > 
> > .
> > 
> > [INFO] diagnostic ...
> > engine/frontend/webadmin/modules/webadmin/src/main/java/org/ovirt/engine/ui/webadmin/section/main/presenter/tab/virtualMachine/SubTabVirtualMachineVirtualDiskPresenter.java:38:
> > error: type org.ovirt.engine.ui.uicommonweb.models.vms.VmListModel does not
> > take parameters
> > SearchableDetailModelProvider,
> > VmDiskListModel> modelProvider) {
> >^
> > .
> > 
> > 
> > After going a few patches back, prior to the one which adds this code, the
> > issue seems to be solved..
> > 
> > Did anybody else had the same problem/know how to fix it?
> > 
> > or maybe that just happens to me locally..
> > 
> > Amit.
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> > 
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Gerrit restart

[Martin Betak]

Ok, not to jinx it, but login seems to be working now :-)

- Original Message -
From: "Martin Betak" 
To: "David Caro" 
Cc: devel@ovirt.org
Sent: Monday, February 23, 2015 1:19:04 PM
Subject: Re: [ovirt-devel] Gerrit restart

I have trouble signing-in. It returns 404: "The requested URL /login/#/c/32907/ 
was not found on this server."

Martin

- Original Message -
From: "David Caro" 
To: devel@ovirt.org
Sent: Monday, February 23, 2015 11:54:26 AM
Subject: Re: [ovirt-devel] Gerrit restart


I've done a few tests now and everything seems to work as expected, can anyone
confirm that it's working for him?

If noone replies, I'll consider it solved

cheers

On 02/23, David Caro wrote:
> 
> Hi!
> 
> After this morning upgrade, gerrit http access has stopped working, I'm trying
> to solve it but in the meantime I might have to restart gerrit a couple of
> times, you can still use git or ssh to clone repos.
> 
> 
> I'll update when finished.
> 
> 
> 
> 
> -- 
> David Caro
> 
> Red Hat S.L.
> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> 
> Tel.: +420 532 294 605
> Email: dc...@redhat.com
> Web: www.redhat.com
> RHT Global #: 82-62605



-- 
David Caro

Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D

Tel.: +420 532 294 605
Email: dc...@redhat.com
Web: www.redhat.com
RHT Global #: 82-62605

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Gerrit restart

[Martin Betak]

I have trouble signing-in. It returns 404: "The requested URL /login/#/c/32907/ 
was not found on this server."

Martin

- Original Message -
From: "David Caro" 
To: devel@ovirt.org
Sent: Monday, February 23, 2015 11:54:26 AM
Subject: Re: [ovirt-devel] Gerrit restart


I've done a few tests now and everything seems to work as expected, can anyone
confirm that it's working for him?

If noone replies, I'll consider it solved

cheers

On 02/23, David Caro wrote:
> 
> Hi!
> 
> After this morning upgrade, gerrit http access has stopped working, I'm trying
> to solve it but in the meantime I might have to restart gerrit a couple of
> times, you can still use git or ssh to clone repos.
> 
> 
> I'll update when finished.
> 
> 
> 
> 
> -- 
> David Caro
> 
> Red Hat S.L.
> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> 
> Tel.: +420 532 294 605
> Email: dc...@redhat.com
> Web: www.redhat.com
> RHT Global #: 82-62605



-- 
David Caro

Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D

Tel.: +420 532 294 605
Email: dc...@redhat.com
Web: www.redhat.com
RHT Global #: 82-62605

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ovirt-users] Announcing the moVirt Incubator Project!

[Martin Betak]

This is great news! Thanks everyone for your support.

And special thanks to Tomas for leading the development effort
and OPW project that brought us this far.

Martin 



- Original Message -
> From: "Tomas Jelinek" 
> To: "Donny D" 
> Cc: "Brian Proffitt" , "board" , 
> annou...@ovirt.org, "users" ,
> devel@ovirt.org, "Sphoorti Joglekar" , "Martin 
> Betak" ,
> mov...@ovirt.org
> Sent: Wednesday, February 18, 2015 9:00:13 AM
> Subject: Re: [ovirt-devel] [ovirt-users] Announcing the moVirt Incubator  
> Project!
> 
> Well, this is quite a milestone!
> 
> Thanx everyone!
> 
> - Original Message -
> > From: "Donny D" 
> > To: "Brian Proffitt" , "board" 
> > Cc: annou...@ovirt.org, "users" , devel@ovirt.org
> > Sent: Tuesday, February 17, 2015 5:31:29 PM
> > Subject: Re: [ovirt-devel] [ovirt-users] Announcing the moVirt Incubator
> > Project!
> > 
> > That is great project. This makes me very happy.
> 
> BTW we have been and are still heavily using your cloudspin.me during moVirt
> development and testing.
> Wanted to use this opportunity to thank you for the great service!
> 
> > 
> > 
> > 
> > Happy Connecting. Sent from my Sprint Samsung Galaxy S® 5
> > 
> > 
> >  Original message 
> > From: Brian Proffitt 
> > Date: 02/17/2015 9:17 AM (GMT-07:00)
> > To: board 
> > Cc: annou...@ovirt.org, users , devel@ovirt.org
> > Subject: [ovirt-users] Announcing the moVirt Incubator Project!
> > 
> > This message is to announce moVirt as a new oVirt incubator project,
> > approved
> > by unanimous vote of the Board mailing list!
> > 
> > Details about the project can be found at
> > 
> > http://www.ovirt.org/Project_moVirt
> > 
> > More information about the incubator at oVirt can be found at
> > 
> > http://www.ovirt.org/Incubating_an_oVirt_Subproject
> > 
> > Please check out this project and see what you can do to help!
> > 
> > --
> > Brian Proffitt
> > 
> > Community Liaison
> > oVirt
> > Open Source and Standards, Red Hat - http://community.redhat.com
> > Phone: +1 574 383 9BKP
> > IRC: bkp @ OFTC
> > ___
> > Users mailing list
> > us...@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> > 
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Evaluating Errai JAX-RS for oVirt

[Martin Betak]

Hi all,

I've been exploring the possibilities of utilizing Errai JAX-RS for REST client 
in 
our current GWT-based frontend. This would be a complement to Vojtech's 
oVirt.js library
which is very native to javascript and utilizes dynamic discovery of resources 
and actions.

Errai JAX-RS would enable us to reuse our restapi definitions of resource 
interfaces and entities 
and generate proxies automatically thus eliminating the need for extensive 
manual
code generation.

[errai jax-rs] https://docs.jboss.org/author/display/ERRAI/Errai+JAX-RS

Unfortunately in my quest I stumbled upon some issues I couldn't exactly google 
or solve 
and I would like to ask Mark or Christian, whether the below is somehow 
possible with Errai:

1) Adding custom header to every request (some form of request interceptors)
You, see our api returns by default XML and we of course want to consume JSON.
Thus adding 'Content-Type: application/json' as well as other custom 
authentication headers
is necessary for our usage.

2) Using custom JSONProvider/ObjectMapper instead of the default Jackson one. 
In our JSONProvider [1] we use custom ObjectMapper [2] mainly to map JAX-B 
annotations
to json. Without this for example errai generates marshaller for the VMs 
resource

public class VMs {
@XmlElement(name = "vm")
protected List vMs;
}

that expects json field called "vMs" instead of the JAX-B specification "vm" 
which is the actual
format returned from the API. There may be other differencies such as [3] so I 
think the best solution
would be if we could tell errai to use specific JSONProvider for generating the 
marshallers.

This are the issues that so far seem to be the most blocking ones, others may 
arise along the way.

Thank you for any help provided.

Best regards,

Martin

[1] 
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/restapi/interface/definition/src/main/java/org/ovirt/engine/api/json/JSONProvider.java
[2] 
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/restapi/interface/definition/src/main/java/org/ovirt/engine/api/json/CustomObjectMapper.java
[3] 
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/restapi/interface/definition/src/main/java/org/ovirt/engine/api/json/CustomBeanFactory.java
[resource schema] 
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/restapi/interface/definition/src/main/resources/api.xsd
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [ACTION REQUIRED] engine-setup broken on master due to missing SysPrep2K3Path

[Martin Betak]

- Original Message -
> From: "Sandro Bonazzola" 
> To: mbe...@redhat.com
> Cc: devel@ovirt.org, "Yedidyah Bar David" , "Lev Veyde" 
> , "Simone Tiraboschi"
> 
> Sent: Thursday, November 6, 2014 1:48:21 PM
> Subject: [ACTION REQUIRED] engine-setup broken on master due to missing 
> SysPrep2K3Path
> 
> 2014-11-06 13:38:41 DEBUG otopi.ovirt_engine_setup.engine_common.database
> database.execute:164 Database: 'None', Statement: '
> select version, option_value
> from vdc_options
> where option_name = %(name)s
> ', args: {'name': 'SysPrep2K3Path'}
> 2014-11-06 13:38:41 DEBUG otopi.ovirt_engine_setup.engine_common.database
> database.execute:214 Result: []
> 2014-11-06 13:38:41 DEBUG otopi.context context._executeMethod:152 method
> exception
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 142, in
>   _executeMethod
> method['method']()
>   File
>   
> "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/legacy/osinfo.py",
>   line 88, in _misc
> ).getVdcOption(name=vdco)
>   File
>   "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine/vdcoption.py",
>   line 88, in getVdcOption
> ownConnection=ownConnection,
>   File
>   "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine/vdcoption.py",
>   line 63, in getVdcOptionVersions
> name=name,
> RuntimeError: Cannot locate application option SysPrep2K3Path
> 
> Looks like
> commit 4a02e12ac9a4f2e18340c177c5f656ddf753694e
> Author: Martin Betak 
> Date:   Wed Nov 5 15:00:47 2014 +0100
> 
> core: Drop legacy SysPrep paths from config
> 
> Change-Id: I675340ae90aa724b935b0cf0a25544c06b0177eb
> Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1086768
> Signed-off-by: Martin Betak 
> 
> 
> Broke the setup.
> Can you please advise about what the setup should do now?

The upgrade script _config.sql now deletes the values from DB.
Is it possible to make this python script run before the DB upgrade scripts are 
run?
We really don't want to keep this unused values in the database after the 
correct migration
to osinfo is performed.

> 
> --
> Sandro Bonazzola
> Better technology. Faster innovation. Powered by community collaboration.
> See how it works at redhat.com
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [Devel] Proposing Roy Golan as VDSM Fake maintainer

[Martin Betak]

+1



- Original Message -
> From: "Frantisek Kobzik" 
> To: "Tomas Jelinek" 
> Cc: "engine-de...@ovirt.org" 
> Sent: Thursday, November 6, 2014 11:00:10 AM
> Subject: Re: [ovirt-devel] [Devel] Proposing Roy Golan as VDSM Fake maintainer
> 
> +1
> 
> - Original Message -
> From: "Tomas Jelinek" 
> To: "infra" , "engine-de...@ovirt.org" 
> Cc: "Liran Zelkha" , "users" 
> Sent: Thursday, November 6, 2014 9:33:07 AM
> Subject: [ovirt-devel] [Devel] Proposing Roy Golan as VDSM Fake maintainer
> 
> Hi,
> 
> I would like to propose Roy Golan as VDSM Fake
> (http://www.ovirt.org/VDSM_Fake) co-maintainer.
> 
> Your response would be appreciated.
> 
> Thanks in advance.
> Tomas
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] GWT vs. Java

[Martin Betak]

- Original Message -
> From: "Vojtech Szocs" 
> To: devel@ovirt.org
> Cc: "Einav Cohen" , "Roy Golan" , 
> "Martin Betak" 
> Sent: Friday, October 3, 2014 3:29:15 PM
> Subject: GWT vs. Java
> 
> Hi guys,
> 
> I've had a discussion with Roy about GWT vs. Java plans,
> he asked me to share this information, so here we go.
> 
> GWT 2.6
> ===
> - patch pending merge: http://gerrit.ovirt.org/#/c/32135/
> - support for Java 7 language syntax like <> operator etc.
> - once patch ^^ is merged, we can consolidate Java language
>   compliance for whole Engine -> both frontend and backend
>   can use Java 7 (yey!)
> 
> GWT 2.7
> ===
> - planned support for *only* Java 8 language syntax like
>   lambdas etc. but *without* emulating new Java 8 APIs like
>   streams etc. (Java 8 API emulation should come in GWT 3.0)
> 
> Likely, GWT 2.7 will remove GWT deRPC implementation on which
> we currently rely on. (@Martin, can you please share details?)

Following the GWT source code, the support for deprecated deRPC 
was removed in commit f18e4e23898d3e090e0fae1fed2abd97e78b62e7 
on master branch which will be almost certainly included in the 2.7 release.

> 
> This means that until we move UI to REST API, we can't upgrade
> post GWT 2.6, so this will put more pressure to work on oVirtJS
> project in consequence.
> 
> GWT 3.0
> ===
> - ETA early next year?
> - planned to fully support Java 8 -> both language and APIs
> - one of key highlights is better integration with JavaScript,
>   which fits the scenario where GWT "application" is one module
>   that's part of bigger application (hybrid approach)
> 
> Regards,
> Vojtech
> 

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Moving forward our frontend stack

[Martin Betak]

- Original Message -
> From: "Vojtech Szocs" 
> To: "Martin Betak" 
> Cc: devel@ovirt.org, "Einav Cohen" , "Alexander Wels" 
> , "Greg Sheremeta"
> , "Tomas Jelinek" , "Lior Vernia" 
> , "Daniel Erez"
> 
> Sent: Thursday, August 28, 2014 5:38:04 PM
> Subject: Re: Moving forward our frontend stack
> 
> Hey Martin!
> 
> I've just reviewed your patch, looks good overall.
> 
> Here are my thoughts:
> 
> * Jenkins CI fails on patch due to missing dependencies,
>   we need to provide following new dependencies in order
>   to proceed with the upgrade:
>  
>   org.aspectj:aspectjweaver:1.8.2
>   org.aspectj:aspectjrt:1.8.2
>   com.google.gwt:gwt-user:2.6.1
>   com.google.gwt:gwt-dev:2.6.1
>   com.google.gwt:gwt-servlet:2.6.1
>   com.google.gwt:2.6.1
>   org.codehaus.mojo:gwt-maven-plugin:2.6.1
>   com.gwtplatform:gwtp-processors:1.3.1
>   com.gwtplatform:gwtp-mvp-client:1.3.1
>   com.google.gwt.inject:gin:2.1.2
> 
>   Who will take charge of that?

I already spoke to David Caro about the jenkins failure of 
patch http://gerrit.ovirt.org/#/c/32135/. It seems the files
for appropriate versions are there but the files are corrupted.

Let us hope this can be resolved easier than Node.js :-)

Best regards,

Martin

> 
> * patch itself looks quite harmless (not too risky)
> 
> * consolidating Java source & target version across
>   frontend and backend is nice!
> 
>   -> this means we could use Java 7 features
>  also on the frontend (Java/GWT) side
> 
> * TODO-GWT tags [1] proved to be helpful [2]
> 
>   -> reminder to all UI maintainers to use TODO-GWT
>  tags whenever we have some GWT(P) workaround,
>  so that the future upgrade will be safer w.r.t.
>  existing code
> 
> [1] https://www.mail-archive.com/devel@ovirt.org/msg00761.html
> [2]
> http://gerrit.ovirt.org/#/c/32135/1/frontend/webadmin/modules/webadmin/src/main/java/org/ovirt/engine/ui/webadmin/section/main/view/SearchPanelView.java
> 
> Regards,
> Vojtech
> 
> PS: I went through GWT-Platform release notes,
> found an interesting new feature in recent GWTP
> release, worth investigating:
> 
>   https://github.com/ArcBees/GWTP/wiki/Release-Notes
>   #346 : Map more than one name token to presenter
> 
> 
> - Original Message -
> > From: "Martin Betak" 
> > To: devel@ovirt.org
> > Cc: "Vojtech Szocs" , "Einav Cohen" ,
> > "Alexander Wels" ,
> > "Greg Sheremeta" , "Tomas Jelinek"
> > , "Lior Vernia" ,
> > "Daniel Erez" 
> > Sent: Thursday, August 28, 2014 4:11:10 PM
> > Subject: Moving forward our frontend stack
> > 
> > Hello oVirt developers!
> > 
> > I have prepared patch [1] that upgrades our frontend stack
> > to use GWT version 2.6.1 (from previous 2.5.1).
> > 
> > This patch also updates GIN to version 2.1.2 and GWT-P to 1.3.1
> > 
> > Since GWT 2.6 features support for Java 7 it was possible to increment
> > language levels of all projects stuck at Java 6 (common, compat,
> > searchbackend
> > and entire of frontend).
> > 
> > To facilitate emitting bytecode compatible with Java 7 also upgrade of
> > AspectJ was
> > necessary. This patch upgrades it to AspectJ 1.8 that features even support
> > for
> > Java 8 which will save effort when upgrading to GWT 2.7/3.0 in the future.
> > 
> > Most of the changes in the patch are due to upgrade of GWT-P - i.e.
> > changing
> > packages of TokenFormatter and PlaceRequest.
> > 
> > Overall this patch is *MUCH* simpler than the previous
> > http://gerrit.ovirt.org/#/c/16739/
> > which facilitated upgrade from 2.3 to 2.5.1, and hopefully much less risky.
> > 
> > I have tested draft-compile, debug-mode and also tried to use the resulting
> > application
> > manually for some time. So far everything worked (surprisingly well!) and I
> > have not
> > detected any defects. Of course I invite anyone to test this patch on his
> > own
> > since
> > it is and upgrade of our core infrastructure.
> > 
> > That having said I think is comparatively simple and the benefits outweigh
> > the risks
> > if this upgrade is done at the beginning of ovirt-3.6 development cycle.
> > 
> > Reviews, comments and testing are very welcome :-)
> > 
> > Best regards,
> > 
> > Martin
> > 
> > [1] http://gerrit.ovirt.org/#/c/32135/
> > 
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Moving forward our frontend stack

[Martin Betak]

Hello oVirt developers!

I have prepared patch [1] that upgrades our frontend stack
to use GWT version 2.6.1 (from previous 2.5.1).

This patch also updates GIN to version 2.1.2 and GWT-P to 1.3.1

Since GWT 2.6 features support for Java 7 it was possible to increment
language levels of all projects stuck at Java 6 (common, compat, searchbackend 
and entire of frontend).

To facilitate emitting bytecode compatible with Java 7 also upgrade of AspectJ 
was 
necessary. This patch upgrades it to AspectJ 1.8 that features even support for 
Java 8 which will save effort when upgrading to GWT 2.7/3.0 in the future.

Most of the changes in the patch are due to upgrade of GWT-P - i.e. changing 
packages of TokenFormatter and PlaceRequest.

Overall this patch is *MUCH* simpler than the previous 
http://gerrit.ovirt.org/#/c/16739/
which facilitated upgrade from 2.3 to 2.5.1, and hopefully much less risky.

I have tested draft-compile, debug-mode and also tried to use the resulting 
application
manually for some time. So far everything worked (surprisingly well!) and I 
have not
detected any defects. Of course I invite anyone to test this patch on his own 
since 
it is and upgrade of our core infrastructure.

That having said I think is comparatively simple and the benefits outweigh the 
risks
if this upgrade is done at the beginning of ovirt-3.6 development cycle.

Reviews, comments and testing are very welcome :-)

Best regards,

Martin

[1] http://gerrit.ovirt.org/#/c/32135/
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Test day 1 results

[Martin Betak]

Hi all,

During the first test day I tested BZ 1108602 "Implement REST API for oVirt 
scheduler".
Most basic CRUD operations via rest worked well, but there was a problem with 
accessing
the subcollections 'filters', 'weights' and 'balances' of the 
/api/schedulingpolicies 
resource by id.

For this a bug was filed under 
https://bugzilla.redhat.com/show_bug.cgi?id=1115071

Martin
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] oVirt.js GWT wrapper prototype

[Martin Betak]

Hi Vojtech,

just one small remark to the oVirt.GWT wrapper.
Do you think it would be possible to replace 
> Sdk.get().api().getDataCenters() with
> Sdk.api().getDataCenters() or even better yet with
> Sdk.getDataCenters()
In general I think that "Flat is better than nested" and I don't think 
the collection names would introduce any collisions with the .services() 
namespace so this should be a safe change.

Other than that I really like the current prototypes for both oVirt.js
and oVirt.GWT and I'm looking forward to future development.

Best regards

Martin

- Original Message -
> From: "Vojtech Szocs" 
> To: devel@ovirt.org
> Sent: Tuesday, June 24, 2014 1:29:26 PM
> Subject: [ovirt-devel] oVirt.js GWT wrapper prototype
> 
> Hello everyone,
> 
> following the announcement of oVirt.js prototype, I've developed a sample
> GWT wrapper that provides Java API to oVirt.js for all GWT applications.
> 
> Please find the GWT wrapper patch attached. It bundles oVirt.js & Lo-Dash
> libraries via GWT module (SdkGwtWrapper) providing Java-like API based on
> oVirt.js.
> 
> In order for WebAdmin to use oVirt.js GWT wrapper, all we have to do is
> add following into WebAdmin.gwt.xml (GWT module descriptor):
> 
>   
> 
> and add following into pom.xml (Maven project descriptor):
> 
>   
> ${engine.groupId}
> ovirt-js-gwt-wrapper
> ${engine.version}
> provided
>   
> 
> and that's it.
> 
> The Java API takes inspiration from oVirt.js API. For example, to add
> new DataCenter:
> 
>   // Create data object template, 'name' and 'local' are required.
>   DataCenterTemplate dcTemplate = DataCenterTemplate.create(
> "test-dc", // name
> false // local
>   );
>   // Set optional parameters such as 'description', if necessary.
>   dcTemplate.setDescription("my-desc");
> 
>   // Obtain DataCenter resource collection.
>   ResourceCollection dcColl = Sdk.get().api().getDataCenters();
> 
>   // Add new DataCenter by running 'add' operation on 'dcColl'.
>   dcColl.add(dcTemplate).success(new SuccessCallback() {
> @Override
> public void onSuccess(DataCenter dc) {
>   Window.alert("Added: " + dc.getName());
> }
>   }).run();
> 
> The concept of resource, resource collection and operation is the same
> as presented in oVirt.js.
> 
>   dc.setName("test-dc-updated");
>   dc.setDescription("test")
>   dc.update().run(); // we could register 'success' callback here
> 
> You can see the full example by looking at SdkGwtWrapperTest class,
> located in WebAdmin codebase (org.ovirt.engine.ui.webadmin.sdk_test).
> 
> Note that 'DataCenter' and 'DataCenterTemplate' will probably be
> auto-generated from oVirt Engine REST API definition (XSD/RSDL).
> 
> As mentioned in my previous email, oVirt.js GWT wrapper ("overlay")
> can be initially part of ovirt-engine repo, while oVirt.js project
> deserves (in my opinion) a separate repo on its own.
> 
> Regards,
> Vojtech
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel