Re: [Devel] gerrit is hanging, but only in chrome

[Eyal Edri]

- Original Message -
> From: "Greg Sheremeta" 
> To: "Itamar Heim" 
> Cc: devel@ovirt.org, in...@ovirt.org
> Sent: Friday, April 4, 2014 8:14:11 PM
> Subject: Re: [Devel] gerrit is hanging, but only in chrome
> 
> Awesome. Do you have a link to the job?
> 

http://jenkins.ovirt.org/view/System%20and%20Monitoring/job/restart_gerrit_service/
it is restricted for infra members and authorized users only.

Eyal.

> - Original Message -
> > From: "Itamar Heim" 
> > To: "Greg Sheremeta" , "David Caro"
> > 
> > Cc: in...@ovirt.org, devel@ovirt.org
> > Sent: Friday, April 4, 2014 1:12:39 PM
> > Subject: Re: [Devel] gerrit is hanging, but only in chrome
> > 
> > On 04/04/2014 07:27 PM, Greg Sheremeta wrote:
> > > Console has no errors, and just one info message "Calling LP_setval from
> > > A"
> > >
> > > Clearing cache has no effect.
> > >
> > > I'll start debugging. Can someone restart gerrit?
> > 
> > I just did. please note there is a jenkins job that you can run which
> > does the same.
> > 
> > >
> > > - Original Message -
> > >> From: "David Caro" 
> > >> To: "Greg Sheremeta" 
> > >> Cc: "Itamar Heim" , in...@ovirt.org, devel@ovirt.org
> > >> Sent: Friday, April 4, 2014 11:00:40 AM
> > >> Subject: Re: [Devel] gerrit is hanging, but only in chrome
> > >>
> > >> On Fri 04 Apr 2014 04:45:25 PM CEST, Greg Sheremeta wrote:
> > >>> gerrit is hanging at the 'loading gerrit code review' screen in Chrome.
> > >>>
> > >>> Works fine in FF.
> > >>>
> > >>> Anyone know what's up?
> > >>>
> > >>> Greg
> > >>>
> > >>> Greg Sheremeta
> > >>> Red Hat, Inc.
> > >>> Sr. Software Engineer, RHEV
> > >>> Cell: 919-807-1086
> > >>> gsher...@redhat.com
> > >>> ___
> > >>> Devel mailing list
> > >>> Devel@ovirt.org
> > >>> http://lists.ovirt.org/mailman/listinfo/devel
> > >>
> > >> Can you check in the js console if there are any errors? I had an issue
> > >> not so long ago that was the other way around, I was unable to see the
> > >> page on firefox, but worked on chrome, it seemed to be a problem with a
> > >> js, tried cleaning browser cache, gerrit server cache with no luck, and
> > >> only I had that issue, around one week ago, it started working again, I
> > >> don't know why :S
> > >>
> > >> If it's that issue (it was complaining about a js db class not existing
> > >> or something similar) we can gather some info try to open a bug on
> > >> gerrit.
> > >>
> > >> --
> > >> David Caro
> > >>
> > >> Red Hat S.L.
> > >> Continuous Integration Engineer - EMEA ENG Virtualization R&D
> > >>
> > >> Email: dc...@redhat.com
> > >> Web: www.redhat.com
> > >> RHT Global #: 82-62605
> > >>
> > >>
> > 
> > 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [vdsm] vdsm sync meeting minutes April 1st, 2014

[ybronhei]

Hey all,
back from France

On 04/02/2014 11:29 AM, dan...@redhat.com wrote:

Vdsm sync call April 1st 2014
=

- cpopen:
   - Toni: there's a versioning mismatch: the version in pypi is higher
 than the one on https://github.com/ficoos/cpopen
   - Saggi: there shouldn't be any code difference
   - Yaniv should sync the two.

https://github.com/ficoos/cpopen/pull/23 <- committed 1.3 tag
https://pypi.python.org/pypi?:action=display&name=cpopen&version=1.3 <- 
uploaded latest code


   - We use two options that are missing from Python3's Popen: umask and
 deathSignal.
 - Toni to re-send his Python3 port for cpopen, with tests running on
   Python3, too.
 - Saggi to accept it.
 - Infra team to suggest missing features to Python.




- fromani described his attempts of consolidating the two
   migration-monitoring threads into one. The motivation is a finer way
   of contolling the migration down time based on progress. Reducing
   thread numbers per se is not the main motivation.

- pep8 has changed. Again. Version 1.5.1 has even more draconic
   requirements. We can comply with them, or, include our version of
   pep8/pyflakes/pylint in our git repo. danken shudders at the thought
   of copying the code, but having a git sub-module is a reasonable
   compromise.
   - Infra team to take this task (unless someone else is faster)
   - Until that happens, one need to use pep8-1.4.6 or --ignore offending
 errors.

- We've been asked to kill the separate vdsm-devel mailing list, and
   join it into devel@ovirt.org. There's some resistence in the audience,
   so we'd do it softly.

   Next week, I'll have vdsm-devel members mass-subscribed to
   devel@ovirt. If you do NOT want to be subscribed, please send me a
   private request.

   Then, for several months, we'd see how it goes, and whether we drown
   in unrelated Engine chatter.

- We had a very (too) heated debate about ignoring failures of
   setDomainRegularRole() in http://gerrit.ovirt.org/24495/ and
   http://gerrit.ovirt.org/25424.

   The pain point is that relying on domain role (master/regular) is
   faulty by design. We cannot avoid the cases where a pool has more than
   one domain with a master role written in its metadata.

   One side argued that oVirt should be fixed to handle this unescapable
   truth, or at least enumerate the places where Vdsm and Engine, both
   current and old, depend on master role uniqueness.

   The other side argued that this is not a priority task, and that we
   should try to "garbage-collect" known-bad master roles as a courtesy
   to people digging into domain metadata, and as a means to lessen the
   problem until we kill the pool concept in the upcoming version.

   I hope that I present the debate fairly enough.

Dan.




--
Yaniv Bronhaim.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] Vdsm functional tests

[ybronhei]

On 04/03/2014 07:31 PM, Douglas Schilling Landgraf wrote:

On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:

Functional tests are intended to verify that a running Vdsm instance
does what it should, when treated as a black box, over its public API.

They should be comprehensive and representative of a typical field usage
of Vdsm. It is a sin to break such a test - but we must be able to know
when such a sin is committed.

We currently have the following functional tests modules:

- sosPluginTests.py
   supervdsmFuncTests.py


Sure, count with me.


any news about it ? need my help around it?
supervdsmFuncTests.py doesn't really check much. we need to add much 
more logic there if we actually want to test the communication between 
vdsm and supervdsm (not sure if its really required.. its like checking 
calls to libvirt or sanlock or testing api calls)




- storageTests.py

- momTests.py
   virtTests.py

- networkTests.py

I'd like to have a designated developer per team (infra, storage, virt
and
network), responsible to having these tests ever-running.

When could we expect to have it running per commit on a Jenkins slaves?

Volunteers, please come forward.

Dan.







--
Yaniv Bronhaim.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Itamar Heim]

On 04/03/2014 07:51 PM, Liran Zelkha wrote:

The problem is with both updates and selects.
For selects - to get all the information for the VDS we have multiple
joins. Adding another one will hurt performance even more.
For updates - we have vds_static thats hardly changed. vds_statistics
that changes all the time. vds_dynamic is not changed allot - but is
updated all the time because of the status. I think it's best to split
it to the two existing tables (BTW - relevant for VM as well)


but we don't update it unless the status has changed, which is a rare 
occurance?

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[Devel] cpopen version inconsistencies

[Saggi Mizrahi]

Yaniv synced the github version with the code
that was released.

1.3 is now tagged.
https://github.com/ficoos/cpopen/tree/1.3.0
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Liran Zelkha]

On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim  wrote:

> On 04/03/2014 07:51 PM, Liran Zelkha wrote:
>
>> The problem is with both updates and selects.
>> For selects - to get all the information for the VDS we have multiple
>> joins. Adding another one will hurt performance even more.
>> For updates - we have vds_static thats hardly changed. vds_statistics
>> that changes all the time. vds_dynamic is not changed allot - but is
>> updated all the time because of the status. I think it's best to split
>> it to the two existing tables (BTW - relevant for VM as well)
>>
>
> but we don't update it unless the status has changed, which is a rare
> occurance?
>
Actually - no. We can definitely see times we are updating vds_dynamic with
no reason at all. I tried to create patches for that - but it happens from
many different places in the code.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[Devel] Network custom properties

[Lior Vernia]

Hello all,

Introducing the oVirt 3.5 feature of network custom properties:
http://www.ovirt.org/Features/Network_Custom_Properties

Essentially, this feature aims to solve two RFEs, that request the
ability to set bridge and ethtool options on host interfaces from the
GUI/REST:
https://bugzilla.redhat.com/show_bug.cgi?id=1080984
https://bugzilla.redhat.com/show_bug.cgi?id=1080987

It will do so by adding custom properties (key:value pairs) to networks
assigned to host interfaces, which can in turn be acted upon via hooks
when e.g. a Setup Networks command is triggered.

Two predefined keys will include bridge_opts and ethtool_opts, but any
arbitrary custom property could be supplied as well.

Please take a look at the detailed feature page and let me know if you
have any comments.

Yours, Lior.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Itamar Heim]

On 04/06/2014 11:32 AM, Liran Zelkha wrote:




On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim mailto:ih...@redhat.com>> wrote:

On 04/03/2014 07:51 PM, Liran Zelkha wrote:

The problem is with both updates and selects.
For selects - to get all the information for the VDS we have
multiple
joins. Adding another one will hurt performance even more.
For updates - we have vds_static thats hardly changed.
vds_statistics
that changes all the time. vds_dynamic is not changed allot - but is
updated all the time because of the status. I think it's best to
split
it to the two existing tables (BTW - relevant for VM as well)


but we don't update it unless the status has changed, which is a
rare occurance?

Actually - no. We can definitely see times we are updating vds_dynamic
with no reason at all. I tried to create patches for that - but it
happens from many different places in the code.


what would be updated vds_dyanmic for status not originating in update 
run time info?

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] NUMA Support - GUI Technical Session

[Gilad Chaplik]

[top posting]

Thanks you Vojtech for the elaborate expatiation, I will follow on that and 
keep you all posted.

Thanks, 
Gilad.

- Original Message -
> From: "Vojtech Szocs" 
> To: "Gilad Chaplik" 
> Cc: devel@ovirt.org, "chegu vinod" , aw...@redhat.com
> Sent: Saturday, April 5, 2014 12:08:09 AM
> Subject: Re: [Devel] NUMA Support - GUI Technical Session
> 
> Forgot to send some references on the matter:
> 
>   https://groups.google.com/forum/#!topic/google-web-toolkit/q0JUtbiUR8g
>   
> http://stackoverflow.com/questions/13681541/best-way-to-implement-a-drag-and-drop-list-in-gwt
> 
> Regards,
> Vojtech
> 
> 
> - Original Message -
> > From: "Vojtech Szocs" 
> > To: aw...@redhat.com
> > Cc: devel@ovirt.org, "chegu vinod" 
> > Sent: Friday, April 4, 2014 11:06:16 PM
> > Subject: Re: [Devel] NUMA Support - GUI Technical Session
> > 
> > Hi Gilad,
> > 
> > to my understanding, we already use _HTML5_ Drag'n'Drop support
> > (exposed by GWT API since 2.4) in "Setup Host Networks" dialog.
> > 
> > To utilize HTML5 Drag'n'Drop support in GWT widget, just extend
> > FocusPanel and mark your widget's DOM element as "draggable".
> > 
> > For example, in UnassignedNetworksPanel constructor:
> > 
> >   getElement().setDraggable( ... );
> >   addBitlessDomHandler(new DragEnterHandler() { ... });
> >   addBitlessDomHandler(new DragOverHandler() { ... });
> >   addBitlessDomHandler(new DragLeaveHandler() { ... });
> >   addBitlessDomHandler(new DropHandler() { ... });
> > 
> > In other words, GWT already exposes API for working with HTML5
> > Drag'n'Drop spec, so you don't need any 3rd party libraries.
> > The downside is that HTML5 Drag'n'Drop spec is supported only
> > in recent browsers (but this isn't an issue for us, AFAIK):
> > 
> >   http://caniuse.com/#feat=dragndrop
> > 
> > So in general we have two alternatives:
> > 
> >   1, use standard HTML5 Drag'n'Drop spec
> > 
> >  pros:
> >  + no need for 3rd party library
> >  + compliant with existing code, i.e. "Setup Host Networks"
> > 
> >  cons (not too relevant IMHO):
> >  - requires browser support of HTML5 Drag'n'Drop spec
> >  - HTML5 Drag'n'Drop spec deals with dragging data, not widgets
> >themselves (no HTML DOM re-parenting after drag finish)
> > 
> >   2, use 3rd party gwt-dnd library
> > 
> >  pros (which I don't think we really need):
> >  + emulate Drag'n'Drop support in older browsers
> >  + more advanced functionality, i.e. allows dragging widgets
> >so that HTML DOM is dynamically updated
> > 
> >  cons:
> >  - dependency on 3rd party library
> >  - this would mean we need to revisit existing code
> >(we should do Drag'n'Drop in one consistent way)
> > 
> > It's possible that I might be missing something, but I'd
> > suggest to try using HTML5 Drag'n'Drop via GWT API as the
> > first approach.
> > 
> > If we find out that HTML5 Drag'n'Drop doesn't work for us
> > in given browser(s) or if we need extra functionality, we
> > can always add gwt-dnd dependency.
> > 
> > Few more comments inline, let me know what you think.
> > 
> > Regards,
> > Vojtech
> > 
> > 
> > - Original Message -
> > > From: "Alexander Wels" 
> > > To: "Gilad Chaplik" 
> > > Cc: eco...@redhat.com, "Vojtech Szocs" ,
> > > dfedi...@redhat.com, engine-de...@ovirt.org, "chegu
> > > vinod" , lver...@redhat.com
> > > Sent: Tuesday, April 1, 2014 7:24:12 PM
> > > Subject: Re: NUMA Support - GUI Technical Session
> > > 
> > > On Tuesday, April 01, 2014 11:34:43 AM Gilad Chaplik wrote:
> > > > Hi all,
> > > > 
> > > > Here are the resolutions from the meeting:
> > > > 
> > > > * option 1
> > > > 1) Use gwt-dnd lib for oVirt's dnd (drag and drop) infrastructure.
> > > > 2) Come up with a very simple POC that covers all of NUMA-support UX
> > > > requirements. 3) Either do a POC, or get UX maintainers approval, that
> > > > moving already existing dnd features to new infrastructure
> > > > (setup-networks
> > > > and scheduling policy dialogs) is feasible and possible.
> > > > 
> > > 
> > > +1 for option 1. None of the drag and drop in the application now looks
> > > terribly hard. The gwt-dnd library simply makes things easier to control
> > > and
> > > maintain.
> > 
> > Yeah, but the downside is adding 3rd party dependency which predates GWT's
> > support for (API exposure of) HTML5 Drag'n'Drop spec.
> > 
> > > 
> > > > * option 2
> > > > Extract setup network dnd capabilities to a common general
> > > > infrastructure
> > > > and use that as an infrastructure. NOTE that setup-networks will not
> > > > use
> > > > it
> > > > in oVirt-3.5.
> > 
> > GWT's HTML5 Drag'n'Drop API works directly on DOM element level, it's just
> > a thin API overlay on top of HTML5Drag'n'Drop spec.
> > 
> > Do we really need custom DnD infra on top of that? Can't we just use GWT
> > APIs like Element.setDraggable, Drag*Handler, Drop*Handler?
> > 
> > > > 
> > > > I will start with option 1, just need UX team app

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Gilad Chaplik]

- Original Message -
> From: "Itamar Heim" 
> To: "Liran Zelkha" 
> Cc: "Gilad Chaplik" , de...@linode01.ovirt.org, 
> "engine-devel" 
> Sent: Sunday, April 6, 2014 11:33:12 AM
> Subject: Re: [Engine-devel] vds_dynamic refactor
> 
> On 04/06/2014 11:32 AM, Liran Zelkha wrote:
> >
> >
> >
> > On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim  > > wrote:
> >
> > On 04/03/2014 07:51 PM, Liran Zelkha wrote:
> >
> > The problem is with both updates and selects.
> > For selects - to get all the information for the VDS we have
> > multiple
> > joins. Adding another one will hurt performance even more.
> > For updates - we have vds_static thats hardly changed.
> > vds_statistics
> > that changes all the time. vds_dynamic is not changed allot - but
> > is
> > updated all the time because of the status. I think it's best to
> > split
> > it to the two existing tables (BTW - relevant for VM as well)
> >
> >
> > but we don't update it unless the status has changed, which is a
> > rare occurance?
> >
> > Actually - no. We can definitely see times we are updating vds_dynamic
> > with no reason at all. I tried to create patches for that - but it
> > happens from many different places in the code.
> 
> what would be updated vds_dyanmic for status not originating in update
> run time info?

We have separate DB flows for that (updateStatus and 
updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
A question: do you know if we update status in updateVdsDynamic? :-) not sure 
but I found a possible race for pending resources (cpu, mem), LOL :-)

Still holds my original thought for having vds_on_boot.

> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Liran Zelkha]

On Sun, Apr 6, 2014 at 3:18 PM, Gilad Chaplik  wrote:

> - Original Message -
> > From: "Itamar Heim" 
> > To: "Liran Zelkha" 
> > Cc: "Gilad Chaplik" , de...@linode01.ovirt.org,
> "engine-devel" 
> > Sent: Sunday, April 6, 2014 11:33:12 AM
> > Subject: Re: [Engine-devel] vds_dynamic refactor
> >
> > On 04/06/2014 11:32 AM, Liran Zelkha wrote:
> > >
> > >
> > >
> > > On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim  > > > wrote:
> > >
> > > On 04/03/2014 07:51 PM, Liran Zelkha wrote:
> > >
> > > The problem is with both updates and selects.
> > > For selects - to get all the information for the VDS we have
> > > multiple
> > > joins. Adding another one will hurt performance even more.
> > > For updates - we have vds_static thats hardly changed.
> > > vds_statistics
> > > that changes all the time. vds_dynamic is not changed allot -
> but
> > > is
> > > updated all the time because of the status. I think it's best
> to
> > > split
> > > it to the two existing tables (BTW - relevant for VM as well)
> > >
> > >
> > > but we don't update it unless the status has changed, which is a
> > > rare occurance?
> > >
> > > Actually - no. We can definitely see times we are updating vds_dynamic
> > > with no reason at all. I tried to create patches for that - but it
> > > happens from many different places in the code.
> >
> > what would be updated vds_dyanmic for status not originating in update
> > run time info?
>
> We have separate DB flows for that (updateStatus and
> updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
> A question: do you know if we update status in updateVdsDynamic? :-) not
> sure but I found a possible race for pending resources (cpu, mem), LOL :-)
>
> I think we do but not sure. Will check.


> Still holds my original thought for having vds_on_boot.
>
>
>
Let's talk f2f on Tuesday?
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Gilad Chaplik]

- Original Message -
> From: "Liran Zelkha" 
> To: "Gilad Chaplik" 
> Cc: "Itamar Heim" , de...@linode01.ovirt.org, 
> "engine-devel" 
> Sent: Sunday, April 6, 2014 3:26:24 PM
> Subject: Re: [Engine-devel] vds_dynamic refactor
> 
> On Sun, Apr 6, 2014 at 3:18 PM, Gilad Chaplik  wrote:
> 
> > - Original Message -
> > > From: "Itamar Heim" 
> > > To: "Liran Zelkha" 
> > > Cc: "Gilad Chaplik" , de...@linode01.ovirt.org,
> > "engine-devel" 
> > > Sent: Sunday, April 6, 2014 11:33:12 AM
> > > Subject: Re: [Engine-devel] vds_dynamic refactor
> > >
> > > On 04/06/2014 11:32 AM, Liran Zelkha wrote:
> > > >
> > > >
> > > >
> > > > On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim  > > > > wrote:
> > > >
> > > > On 04/03/2014 07:51 PM, Liran Zelkha wrote:
> > > >
> > > > The problem is with both updates and selects.
> > > > For selects - to get all the information for the VDS we have
> > > > multiple
> > > > joins. Adding another one will hurt performance even more.
> > > > For updates - we have vds_static thats hardly changed.
> > > > vds_statistics
> > > > that changes all the time. vds_dynamic is not changed allot -
> > but
> > > > is
> > > > updated all the time because of the status. I think it's best
> > to
> > > > split
> > > > it to the two existing tables (BTW - relevant for VM as well)
> > > >
> > > >
> > > > but we don't update it unless the status has changed, which is a
> > > > rare occurance?
> > > >
> > > > Actually - no. We can definitely see times we are updating vds_dynamic
> > > > with no reason at all. I tried to create patches for that - but it
> > > > happens from many different places in the code.
> > >
> > > what would be updated vds_dyanmic for status not originating in update
> > > run time info?
> >
> > We have separate DB flows for that (updateStatus and
> > updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
> > A question: do you know if we update status in updateVdsDynamic? :-) not
> > sure but I found a possible race for pending resources (cpu, mem), LOL :-)
> >
> > I think we do but not sure. Will check.

Of course it is, that was a rhetorical question :-) (a lot of emoticons and 
LOLs ;-))

> 
> 
> > Still holds my original thought for having vds_on_boot.
> >
> >
> >
> Let's talk f2f on Tuesday?

I'd prefer to reach conclusions here, I'd like everyone to be involved in a 
root issue like this one.

> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Kobi Ianko]

Joining in...
>From my point of view, in real life a user should have that many VDSs on one 
>Engine (from a DB point of view).
Modern DB system handles tables with millions of records and many relations, Do 
we really have a performance issue here?
We could prefer a more easy to maintain implantation in this case over DB 
performance


- Original Message -
> From: "Gilad Chaplik" 
> To: "Liran Zelkha" 
> Cc: de...@linode01.ovirt.org, "engine-devel" 
> Sent: Sunday, April 6, 2014 3:32:26 PM
> Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> 
> - Original Message -
> > From: "Liran Zelkha" 
> > To: "Gilad Chaplik" 
> > Cc: "Itamar Heim" , de...@linode01.ovirt.org,
> > "engine-devel" 
> > Sent: Sunday, April 6, 2014 3:26:24 PM
> > Subject: Re: [Engine-devel] vds_dynamic refactor
> > 
> > On Sun, Apr 6, 2014 at 3:18 PM, Gilad Chaplik  wrote:
> > 
> > > - Original Message -
> > > > From: "Itamar Heim" 
> > > > To: "Liran Zelkha" 
> > > > Cc: "Gilad Chaplik" , de...@linode01.ovirt.org,
> > > "engine-devel" 
> > > > Sent: Sunday, April 6, 2014 11:33:12 AM
> > > > Subject: Re: [Engine-devel] vds_dynamic refactor
> > > >
> > > > On 04/06/2014 11:32 AM, Liran Zelkha wrote:
> > > > >
> > > > >
> > > > >
> > > > > On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim  > > > > > wrote:
> > > > >
> > > > > On 04/03/2014 07:51 PM, Liran Zelkha wrote:
> > > > >
> > > > > The problem is with both updates and selects.
> > > > > For selects - to get all the information for the VDS we have
> > > > > multiple
> > > > > joins. Adding another one will hurt performance even more.
> > > > > For updates - we have vds_static thats hardly changed.
> > > > > vds_statistics
> > > > > that changes all the time. vds_dynamic is not changed allot -
> > > but
> > > > > is
> > > > > updated all the time because of the status. I think it's best
> > > to
> > > > > split
> > > > > it to the two existing tables (BTW - relevant for VM as well)
> > > > >
> > > > >
> > > > > but we don't update it unless the status has changed, which is a
> > > > > rare occurance?
> > > > >
> > > > > Actually - no. We can definitely see times we are updating
> > > > > vds_dynamic
> > > > > with no reason at all. I tried to create patches for that - but it
> > > > > happens from many different places in the code.
> > > >
> > > > what would be updated vds_dyanmic for status not originating in update
> > > > run time info?
> > >
> > > We have separate DB flows for that (updateStatus and
> > > updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
> > > A question: do you know if we update status in updateVdsDynamic? :-) not
> > > sure but I found a possible race for pending resources (cpu, mem), LOL
> > > :-)
> > >
> > > I think we do but not sure. Will check.
> 
> Of course it is, that was a rhetorical question :-) (a lot of emoticons and
> LOLs ;-))
> 
> > 
> > 
> > > Still holds my original thought for having vds_on_boot.
> > >
> > >
> > >
> > Let's talk f2f on Tuesday?
> 
> I'd prefer to reach conclusions here, I'd like everyone to be involved in a
> root issue like this one.
> 
> > 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Liran Zelkha]

On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko  wrote:

> Joining in...
> From my point of view, in real life a user should have that many VDSs on
> one Engine (from a DB point of view).
> Modern DB system handles tables with millions of records and many
> relations, Do we really have a performance issue here?
> We could prefer a more easy to maintain implantation in this case over DB
> performance
>
> Yes we do. We make many queries on the VDS view, which is a VERY complex
view.

>
> - Original Message -
> > From: "Gilad Chaplik" 
> > To: "Liran Zelkha" 
> > Cc: de...@linode01.ovirt.org, "engine-devel" 
> > Sent: Sunday, April 6, 2014 3:32:26 PM
> > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> >
> > - Original Message -
> > > From: "Liran Zelkha" 
> > > To: "Gilad Chaplik" 
> > > Cc: "Itamar Heim" , de...@linode01.ovirt.org,
> > > "engine-devel" 
> > > Sent: Sunday, April 6, 2014 3:26:24 PM
> > > Subject: Re: [Engine-devel] vds_dynamic refactor
> > >
> > > On Sun, Apr 6, 2014 at 3:18 PM, Gilad Chaplik 
> wrote:
> > >
> > > > - Original Message -
> > > > > From: "Itamar Heim" 
> > > > > To: "Liran Zelkha" 
> > > > > Cc: "Gilad Chaplik" ,
> de...@linode01.ovirt.org,
> > > > "engine-devel" 
> > > > > Sent: Sunday, April 6, 2014 11:33:12 AM
> > > > > Subject: Re: [Engine-devel] vds_dynamic refactor
> > > > >
> > > > > On 04/06/2014 11:32 AM, Liran Zelkha wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim  > > > > > > wrote:
> > > > > >
> > > > > > On 04/03/2014 07:51 PM, Liran Zelkha wrote:
> > > > > >
> > > > > > The problem is with both updates and selects.
> > > > > > For selects - to get all the information for the VDS we
> have
> > > > > > multiple
> > > > > > joins. Adding another one will hurt performance even
> more.
> > > > > > For updates - we have vds_static thats hardly changed.
> > > > > > vds_statistics
> > > > > > that changes all the time. vds_dynamic is not changed
> allot -
> > > > but
> > > > > > is
> > > > > > updated all the time because of the status. I think it's
> best
> > > > to
> > > > > > split
> > > > > > it to the two existing tables (BTW - relevant for VM as
> well)
> > > > > >
> > > > > >
> > > > > > but we don't update it unless the status has changed, which
> is a
> > > > > > rare occurance?
> > > > > >
> > > > > > Actually - no. We can definitely see times we are updating
> > > > > > vds_dynamic
> > > > > > with no reason at all. I tried to create patches for that - but
> it
> > > > > > happens from many different places in the code.
> > > > >
> > > > > what would be updated vds_dyanmic for status not originating in
> update
> > > > > run time info?
> > > >
> > > > We have separate DB flows for that (updateStatus and
> > > > updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
> > > > A question: do you know if we update status in updateVdsDynamic? :-)
> not
> > > > sure but I found a possible race for pending resources (cpu, mem),
> LOL
> > > > :-)
> > > >
> > > > I think we do but not sure. Will check.
> >
> > Of course it is, that was a rhetorical question :-) (a lot of emoticons
> and
> > LOLs ;-))
> >
> > >
> > >
> > > > Still holds my original thought for having vds_on_boot.
> > > >
> > > >
> > > >
> > > Let's talk f2f on Tuesday?
> >
> > I'd prefer to reach conclusions here, I'd like everyone to be involved
> in a
> > root issue like this one.
>

What is the update frequency of this field?

> >
> > >
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> >
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Gilad Chaplik]

- Original Message -
> From: "Liran Zelkha" 
> To: "Kobi Ianko" 
> Cc: "Gilad Chaplik" , de...@linode01.ovirt.org, 
> "engine-devel" 
> Sent: Sunday, April 6, 2014 3:40:13 PM
> Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> 
> On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko  wrote:
> 
> > Joining in...
> > From my point of view, in real life a user should have that many VDSs on
> > one Engine (from a DB point of view).
> > Modern DB system handles tables with millions of records and many
> > relations, Do we really have a performance issue here?
> > We could prefer a more easy to maintain implantation in this case over DB
> > performance
> >
> > Yes we do. We make many queries on the VDS view, which is a VERY complex
> view.
> 

Actually I quite agree with Kobi, what is the plan for VMs? why do we start 
with VDS... 
what is the biggest deploy do you know of?

> >
> > - Original Message -
> > > From: "Gilad Chaplik" 
> > > To: "Liran Zelkha" 
> > > Cc: de...@linode01.ovirt.org, "engine-devel" 
> > > Sent: Sunday, April 6, 2014 3:32:26 PM
> > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> > >
> > > - Original Message -
> > > > From: "Liran Zelkha" 
> > > > To: "Gilad Chaplik" 
> > > > Cc: "Itamar Heim" , de...@linode01.ovirt.org,
> > > > "engine-devel" 
> > > > Sent: Sunday, April 6, 2014 3:26:24 PM
> > > > Subject: Re: [Engine-devel] vds_dynamic refactor
> > > >
> > > > On Sun, Apr 6, 2014 at 3:18 PM, Gilad Chaplik 
> > wrote:
> > > >
> > > > > - Original Message -
> > > > > > From: "Itamar Heim" 
> > > > > > To: "Liran Zelkha" 
> > > > > > Cc: "Gilad Chaplik" ,
> > de...@linode01.ovirt.org,
> > > > > "engine-devel" 
> > > > > > Sent: Sunday, April 6, 2014 11:33:12 AM
> > > > > > Subject: Re: [Engine-devel] vds_dynamic refactor
> > > > > >
> > > > > > On 04/06/2014 11:32 AM, Liran Zelkha wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim  > > > > > > > wrote:
> > > > > > >
> > > > > > > On 04/03/2014 07:51 PM, Liran Zelkha wrote:
> > > > > > >
> > > > > > > The problem is with both updates and selects.
> > > > > > > For selects - to get all the information for the VDS we
> > have
> > > > > > > multiple
> > > > > > > joins. Adding another one will hurt performance even
> > more.
> > > > > > > For updates - we have vds_static thats hardly changed.
> > > > > > > vds_statistics
> > > > > > > that changes all the time. vds_dynamic is not changed
> > allot -
> > > > > but
> > > > > > > is
> > > > > > > updated all the time because of the status. I think it's
> > best
> > > > > to
> > > > > > > split
> > > > > > > it to the two existing tables (BTW - relevant for VM as
> > well)
> > > > > > >
> > > > > > >
> > > > > > > but we don't update it unless the status has changed, which
> > is a
> > > > > > > rare occurance?
> > > > > > >
> > > > > > > Actually - no. We can definitely see times we are updating
> > > > > > > vds_dynamic
> > > > > > > with no reason at all. I tried to create patches for that - but
> > it
> > > > > > > happens from many different places in the code.
> > > > > >
> > > > > > what would be updated vds_dyanmic for status not originating in
> > update
> > > > > > run time info?
> > > > >
> > > > > We have separate DB flows for that (updateStatus and
> > > > > updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
> > > > > A question: do you know if we update status in updateVdsDynamic? :-)
> > not
> > > > > sure but I found a possible race for pending resources (cpu, mem),
> > LOL
> > > > > :-)
> > > > >
> > > > > I think we do but not sure. Will check.
> > >
> > > Of course it is, that was a rhetorical question :-) (a lot of emoticons
> > and
> > > LOLs ;-))
> > >
> > > >
> > > >
> > > > > Still holds my original thought for having vds_on_boot.
> > > > >
> > > > >
> > > > >
> > > > Let's talk f2f on Tuesday?
> > >
> > > I'd prefer to reach conclusions here, I'd like everyone to be involved
> > in a
> > > root issue like this one.
> >
> 
> What is the update frequency of this field?
> 

which field?
status? pending resources? on boot fields?
iinm, status is updated mostly by user actions, at least in positive scenarios, 
and not that often.


> > >
> > > >
> > > ___
> > > Devel mailing list
> > > Devel@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
> > >
> >
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Gilad Chaplik]

- Original Message -
> From: "Liran Zelkha" 
> To: "Gilad Chaplik" 
> Cc: "Kobi Ianko" , de...@linode01.ovirt.org, "engine-devel" 
> 
> Sent: Sunday, April 6, 2014 8:51:02 PM
> Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> 
> On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik  wrote:
> 
> > - Original Message -
> > > From: "Liran Zelkha" 
> > > To: "Kobi Ianko" 
> > > Cc: "Gilad Chaplik" , de...@linode01.ovirt.org,
> > "engine-devel" 
> > > Sent: Sunday, April 6, 2014 3:40:13 PM
> > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> > >
> > > On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko  wrote:
> > >
> > > > Joining in...
> > > > From my point of view, in real life a user should have that many VDSs
> > on
> > > > one Engine (from a DB point of view).
> > > > Modern DB system handles tables with millions of records and many
> > > > relations, Do we really have a performance issue here?
> > > > We could prefer a more easy to maintain implantation in this case over
> > DB
> > > > performance
> > > >
> > > > Yes we do. We make many queries on the VDS view, which is a VERY
> > complex
> > > view.
> > >
> >
> > Actually I quite agree with Kobi, what is the plan for VMs? why do we
> > start with VDS...
> > what is the biggest deploy do you know of?
> >
> We start with VDS because in an idle system, with 200 hosts and several
> thousands VMs, this is what you get as the top queries against the
> database. Look at how many times getvds is called.
> [image: Inline image 1]
> BTW - the second query is an example of abusing the dynamic query
> mechanism. The 4th query (an update command) is a set of useless
> update_vds_dynamic commands.
> 
> For reference, the explain plan of get VDS is something like this:
> 
> QUERY PLAN
> 
> ---
>  Nested Loop  (cost=9.30..46.75 rows=6 width=9060) (actual
> time=0.063..0.068 rows=1 loops=1)
>Join Filter: (vds_static.vds_id = vds_statistics.vds_id)
>->  Seq Scan on vds_statistics  (cost=0.00..1.01 rows=1 width=109)
> (actual time=0.008..0.008 rows=1 loops=1)
>->  Nested Loop  (cost=9.30..45.64 rows=6 width=8983) (actual
> time=0.048..0.052 rows=1 loops=1)
>  Join Filter: (vds_groups.vds_group_id = vds_static.vds_group_id)
>  ->  Nested Loop Left Join  (cost=0.00..9.29 rows=1 width=1389)
> (actual time=0.013..0.013 rows=1 loops=1)
>->  Seq Scan on vds_groups  (cost=0.00..1.01 rows=1
> width=1271) (actual time=0.003..0.003 rows=1 loops=1)
>->  Index Scan using pk_storage_pool on storage_pool
>  (cost=0.00..8.27 rows=1 width=134) (actual time=0.008..0.008 rows=1
> loops=1)
>  Index Cond: (vds_groups.storage_pool_id = id)
>  ->  Hash Right Join  (cost=9.30..36.28 rows=6 width=7610) (actual
> time=0.033..0.037 rows=1 loops=1)
>Hash Cond: (vds_spm_id_map.vds_id = vds_static.vds_id)
>->  Seq Scan on vds_spm_id_map  (cost=0.00..22.30 rows=1230
> width=20) (actual time=0.003..0.003 rows=1 loops=1)
>->  Hash  (cost=9.29..9.29 rows=1 width=7606) (actual
> time=0.019..0.019 rows=1 loops=1)
>  Buckets: 1024  Batches: 1  Memory Usage: 2kB
>  ->  Nested Loop  (cost=0.00..9.29 rows=1 width=7606)
> (actual time=0.012..0.013 rows=1 loops=1)
>->  Seq Scan on vds_dynamic  (cost=0.00..1.01
> rows=1 width=1895) (actual time=0.006..0.006 rows=1 loops=1)
>->  Index Scan using pk_vds_static on vds_static
>  (cost=0.00..8.27 rows=1 width=5711) (actual time=0.005..0.006 rows=1
> loops=1)
>  Index Cond: (vds_id = vds_dynamic.vds_id)
>  Total runtime: 0.299 ms
> (19 rows)
> 
> It's terrible. Adding any additional join will make this worse. Please
> don't add any more tables...

Thank you for the detailed explanation, my comments:

* a very long time isn't an argument for not adding another table (should be 
neglectable);
currently we have an unrelated problem, we need to solve it.

* > We start with VDS because in an idle system, with 200 hosts and several
> thousands VMs, this is what you get as the top queries against the
> database.

so, if fetching VMs takes 10 minutes? and its get called a single time?

* you didn't reply on my of my suggestion of constructing the VDS records in 
the DB without using joins.


> 
> >
> > > >
> > > > - Original Message -
> > > > > From: "Gilad Chaplik" 
> > > > > To: "Liran Zelkha" 
> > > > > Cc: de...@linode01.ovirt.org, "engine-devel"  > >
> > > > > Sent: Sunday, April 6, 2014 3:32:26 PM
> > > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> > > > >
> > > > > - Original Message -
> > > > > > From: "Liran Zelkha" 
> > > > > > To: "Gilad Chaplik" 
> > > > > > Cc: "Itamar Heim" , de...@linode01.ovirt.org,

Re: [Devel] Network custom properties

[Dan Kenigsberg]

On Sun, Apr 06, 2014 at 11:33:29AM +0300, Lior Vernia wrote:
> Hello all,
> 
> Introducing the oVirt 3.5 feature of network custom properties:
> http://www.ovirt.org/Features/Network_Custom_Properties
> 
> Essentially, this feature aims to solve two RFEs, that request the
> ability to set bridge and ethtool options on host interfaces from the
> GUI/REST:
> https://bugzilla.redhat.com/show_bug.cgi?id=1080984
> https://bugzilla.redhat.com/show_bug.cgi?id=1080987
> 
> It will do so by adding custom properties (key:value pairs) to networks
> assigned to host interfaces, which can in turn be acted upon via hooks
> when e.g. a Setup Networks command is triggered.
> 
> Two predefined keys will include bridge_opts and ethtool_opts, but any
> arbitrary custom property could be supplied as well.
> 
> Please take a look at the detailed feature page and let me know if you
> have any comments.

Thanks for taking this feature! I think that it's very important to open
up our network configuration to all sorts of things that users want and
we have not even thought about.

I did not read all the page, but I'm worried about
http://www.ovirt.org/Features/Network_Custom_Properties#Vdsm
The network-specific custom properties should be specific to a network,
not to a setupNetwork command. They cannot be part of
@SetupNetworkOptions, as two different network being set up by a single
command may have different custom properties.

The following example dictionary adds to the confusion with what seems
to be a typo (bootproto is certainly not an option of 'bonding').

Dan.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] Vdsm functional tests

[Dan Kenigsberg]

On Sun, Apr 06, 2014 at 11:18:19AM +0300, ybronhei wrote:
> On 04/03/2014 07:31 PM, Douglas Schilling Landgraf wrote:
> >On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:
> >>Functional tests are intended to verify that a running Vdsm instance
> >>does what it should, when treated as a black box, over its public API.
> >>
> >>They should be comprehensive and representative of a typical field usage
> >>of Vdsm. It is a sin to break such a test - but we must be able to know
> >>when such a sin is committed.
> >>
> >>We currently have the following functional tests modules:
> >>
> >>- sosPluginTests.py
> >>   supervdsmFuncTests.py
> >>
> >Sure, count with me.
> 
> any news about it ? need my help around it?

Douglas still owes me a time estimate on when this be done.

> supervdsmFuncTests.py doesn't really check much. we need to add much
> more logic there if we actually want to test the communication
> between vdsm and supervdsm (not sure if its really required.. its
> like checking calls to libvirt or sanlock or testing api calls)

At the moment supervdsmFuncTests do test that supervdsm is reponsive and
that supervdsm.getProxy() works. It's not like testing libvirt api calls
since supervdsm is inside our tree. So it's like testing libvirt api
calls - within the libvirt project.

I would embrace more smarter logic into the test - but I'm not sure what
you have in mind.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] Vdsm functional tests

[Douglas Schilling Landgraf]

On 04/03/2014 12:24 PM, Dan Kenigsberg wrote:

On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote:

On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:

Functional tests are intended to verify that a running Vdsm instance
does what it should, when treated as a black box, over its public API.

They should be comprehensive and representative of a typical field usage
of Vdsm. It is a sin to break such a test - but we must be able to know
when such a sin is committed.

We currently have the following functional tests modules:

- sosPluginTests.py
   supervdsmFuncTests.py


Sure, count with me.


Thanks! When do you think you could write a job similar to
http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_functional_tests/configure
running whenever there's a change in the modules relevant to
sosPluginTests and supervdsmFuncTests?



Hi,

Instead of to split by domain like (network, infra, storage), why not 
have a single functional test job? If something fail, it should trigger 
the volunteers.


For while, I started a creation of infra based on the above one.
http://jenkins.ovirt.org/job/vdsm_infra_functional_tests

--
Cheers
Douglas
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [Devel] [Engine-devel] vds_dynamic refactor

[Liran Zelkha]

On Sun, Apr 6, 2014 at 11:03 PM, Gilad Chaplik  wrote:

> - Original Message -
> > From: "Liran Zelkha" 
> > To: "Gilad Chaplik" 
> > Cc: "Kobi Ianko" , de...@linode01.ovirt.org,
> "engine-devel" 
> > Sent: Sunday, April 6, 2014 8:51:02 PM
> > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> >
> > On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik 
> wrote:
> >
> > > - Original Message -
> > > > From: "Liran Zelkha" 
> > > > To: "Kobi Ianko" 
> > > > Cc: "Gilad Chaplik" , de...@linode01.ovirt.org,
> > > "engine-devel" 
> > > > Sent: Sunday, April 6, 2014 3:40:13 PM
> > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
> > > >
> > > > On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko  wrote:
> > > >
> > > > > Joining in...
> > > > > From my point of view, in real life a user should have that many
> VDSs
> > > on
> > > > > one Engine (from a DB point of view).
> > > > > Modern DB system handles tables with millions of records and many
> > > > > relations, Do we really have a performance issue here?
> > > > > We could prefer a more easy to maintain implantation in this case
> over
> > > DB
> > > > > performance
> > > > >
> > > > > Yes we do. We make many queries on the VDS view, which is a VERY
> > > complex
> > > > view.
> > > >
> > >
> > > Actually I quite agree with Kobi, what is the plan for VMs? why do we
> > > start with VDS...
> > > what is the biggest deploy do you know of?
> > >
> > We start with VDS because in an idle system, with 200 hosts and several
> > thousands VMs, this is what you get as the top queries against the
> > database. Look at how many times getvds is called.
> > [image: Inline image 1]
> > BTW - the second query is an example of abusing the dynamic query
> > mechanism. The 4th query (an update command) is a set of useless
> > update_vds_dynamic commands.
> >
> > For reference, the explain plan of get VDS is something like this:
> >
> > QUERY PLAN
> >
> >
> ---
> >  Nested Loop  (cost=9.30..46.75 rows=6 width=9060) (actual
> > time=0.063..0.068 rows=1 loops=1)
> >Join Filter: (vds_static.vds_id = vds_statistics.vds_id)
> >->  Seq Scan on vds_statistics  (cost=0.00..1.01 rows=1 width=109)
> > (actual time=0.008..0.008 rows=1 loops=1)
> >->  Nested Loop  (cost=9.30..45.64 rows=6 width=8983) (actual
> > time=0.048..0.052 rows=1 loops=1)
> >  Join Filter: (vds_groups.vds_group_id = vds_static.vds_group_id)
> >  ->  Nested Loop Left Join  (cost=0.00..9.29 rows=1 width=1389)
> > (actual time=0.013..0.013 rows=1 loops=1)
> >->  Seq Scan on vds_groups  (cost=0.00..1.01 rows=1
> > width=1271) (actual time=0.003..0.003 rows=1 loops=1)
> >->  Index Scan using pk_storage_pool on storage_pool
> >  (cost=0.00..8.27 rows=1 width=134) (actual time=0.008..0.008 rows=1
> > loops=1)
> >  Index Cond: (vds_groups.storage_pool_id = id)
> >  ->  Hash Right Join  (cost=9.30..36.28 rows=6 width=7610)
> (actual
> > time=0.033..0.037 rows=1 loops=1)
> >Hash Cond: (vds_spm_id_map.vds_id = vds_static.vds_id)
> >->  Seq Scan on vds_spm_id_map  (cost=0.00..22.30
> rows=1230
> > width=20) (actual time=0.003..0.003 rows=1 loops=1)
> >->  Hash  (cost=9.29..9.29 rows=1 width=7606) (actual
> > time=0.019..0.019 rows=1 loops=1)
> >  Buckets: 1024  Batches: 1  Memory Usage: 2kB
> >  ->  Nested Loop  (cost=0.00..9.29 rows=1 width=7606)
> > (actual time=0.012..0.013 rows=1 loops=1)
> >->  Seq Scan on vds_dynamic  (cost=0.00..1.01
> > rows=1 width=1895) (actual time=0.006..0.006 rows=1 loops=1)
> >->  Index Scan using pk_vds_static on
> vds_static
> >  (cost=0.00..8.27 rows=1 width=5711) (actual time=0.005..0.006 rows=1
> > loops=1)
> >  Index Cond: (vds_id =
> vds_dynamic.vds_id)
> >  Total runtime: 0.299 ms
> > (19 rows)
> >
> > It's terrible. Adding any additional join will make this worse. Please
> > don't add any more tables...
>
> Thank you for the detailed explanation, my comments:
>
> * a very long time isn't an argument for not adding another table (should
> be neglectable);
> currently we have an unrelated problem, we need to solve it.
>
Of course it is. A very long time for a query that you execute many times
is THE factor. Who said the join has no performance effect? Have you tested
it? Under load?  Under many writes/updates?

>
> * > We start with VDS because in an idle system, with 200 hosts and several
> > thousands VMs, this is what you get as the top queries against the
> > database.
>
> so, if fetching VMs takes 10 minutes? and its get called a single time?
>
 Where do you see 10 minutes? If you are looking at the red bar it's the
inherent time - total query time * number 

Re: [Devel] Vdsm functional tests

[ybronhei]

On 04/07/2014 01:42 AM, Dan Kenigsberg wrote:

On Sun, Apr 06, 2014 at 11:18:19AM +0300, ybronhei wrote:

On 04/03/2014 07:31 PM, Douglas Schilling Landgraf wrote:

On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:

Functional tests are intended to verify that a running Vdsm instance
does what it should, when treated as a black box, over its public API.

They should be comprehensive and representative of a typical field usage
of Vdsm. It is a sin to break such a test - but we must be able to know
when such a sin is committed.

We currently have the following functional tests modules:

- sosPluginTests.py
   supervdsmFuncTests.py


Sure, count with me.


any news about it ? need my help around it?


Douglas still owes me a time estimate on when this be done.


supervdsmFuncTests.py doesn't really check much. we need to add much
more logic there if we actually want to test the communication
between vdsm and supervdsm (not sure if its really required.. its
like checking calls to libvirt or sanlock or testing api calls)


At the moment supervdsmFuncTests do test that supervdsm is reponsive and
that supervdsm.getProxy() works. It's not like testing libvirt api calls
since supervdsm is inside our tree. So it's like testing libvirt api
calls - within the libvirt project.

I would embrace more smarter logic into the test - but I'm not sure what
you have in mind.


don't have yet. will think about it with douglas
if you or anyone else had any thought of infra's functional test, please 
raise it up and we'll work on it


--
Yaniv Bronhaim.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel