[ovirt-devel] Clarify NUMA feature and CPU pinning
Hi Gilad & Einav Thanks for sharing your ideas about NUMA feature and CPU pinning. It makes easier for users to understand. And something I want to clarify: 1. NUMA tuning feature and CPU pinning feature are individually in libvirt ( ovirt backend ). 2. User could configure VM CPU pining ( NUMA not included ) separately. 3. When configuring NUMA feature, user should configure VM NUMA tuning ( pin to host NUMA node & tuning mode ) and VM CPU pinning ( NUMA included ), otherwise VM will have low performance. 4. Single virtual NUMA node could pin to multiply host NUMA nodes. 5. Multiply virtual NUMA nodes could pin to single host NUMA node. 6. NUMA tuning mode have default value 'strict', and could change to 'prefer', 'interleave' Our proposal: OVirt 3.5 1. Keep current UX design, add NUMA tuning mode select box in VM dialog. 2. Keep current BE model, query and action command. 3. Consider the low performance scene ( user did not configure VM CPU pinning ) When VdsBroker check the CPU pinning text is empty, then generate the right CPU pinning text from virtual NUMA nodes ( pin to host NUMA nodes ) The next version 1. Remove current CPU pinning text feature. 2. Consider the NUMA node inside CPU pinning design. ( Should consider the individually CPU pining design not includes NUMA ) 3. Add related CPU pinning BE model, query and action command. 4. Turn off VdsBroker generate function. Do you agree ? Best Regards, Jason Liao ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [Devel] [Engine-devel] vds_dynamic refactor
- Original Message - > From: "Gilad Chaplik" > To: "Yaniv Dary" > Cc: "Liran Zelkha" , "Kobi Ianko" , > de...@linode01.ovirt.org, "engine-devel" > > Sent: Thursday, April 10, 2014 12:30:52 PM > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > - Original Message - > > From: "Yaniv Dary" > > To: "Gilad Chaplik" > > Cc: "Liran Zelkha" , "Kobi Ianko" > > , de...@linode01.ovirt.org, "engine-devel" > > > > Sent: Thursday, April 10, 2014 11:44:32 AM > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > > > - Original Message - > > > From: "Gilad Chaplik" > > > To: "Liran Zelkha" > > > Cc: "Yaniv Dary" , "Kobi Ianko" , > > > de...@linode01.ovirt.org, "engine-devel" > > > > > > Sent: Thursday, April 10, 2014 11:07:12 AM > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > - Original Message - > > > > From: "Liran Zelkha" > > > > To: "Yaniv Dary" > > > > Cc: "Gilad Chaplik" , "Kobi Ianko" > > > > , > > > > de...@linode01.ovirt.org, "engine-devel" > > > > > > > > Sent: Wednesday, April 9, 2014 4:22:39 PM > > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > > > On Wed, Apr 9, 2014 at 3:34 PM, Yaniv Dary wrote: > > > > > > > > > > Why not move only status with changes a lot to statistics and leave > > > > > everything as is? > > > > > > > > > > > > > > Exactly. No need for a new table. Use the existing ones. > > > > > > Why should the dwh read entire dynamic record for each status > > > change/available resources change? (for VMs as well). > > > > The DWH doesn't work like that. We create views over dynamic\statistics and > > dynamic\static they go the the configuration and stats tables on the > > history > > db side. > > From dynamic\statistics we collect every minute no matter what and from > > dynamic\static we collect only when update_date changes in static. > > dynamic\statistics - dynamic\static > > to fully understand it, we have 2 triggers to collect from dynamic? No, the correct answer is that we don't have any triggers. dynamic\statistics - we collect every minute for stats like status and cpu usage no matter what changed. dynamic\static - we collect either when static table changes and update_date change or once an hour if dynamic changes, but we check this with join and rejects. This is because dynamic changes too much right now. > > > We can not rely on the dynamic update_date since it changes so much, so > > from > > my point of view if we can cause minimal changes in dynamic and move status > > to statistics it would the best thing. > > What we have currently is a task that run once a hour and checks if any > > change was made in the dynamic table (via very ugly joins and rejects) and > > sync is there was any change. > > It would even be better if you also move swap_size which doesn't change > > much > > to dynamic. > > > > oh... I thought that we had a trigger to dynamic (if ^ is correct we have). > > +1 (out of 1) for you comment :-) and thanks Yaniv the elaborate explanation! > > > > > > > > > > > > > > > > > > > > > > > Yaniv > > > > > > > > > > > > > > > > > > > > From: "Liran Zelkha" > > > > > To: "Gilad Chaplik" > > > > > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > > > > > "engine-devel" > > > > > Sent: Monday, April 7, 2014 8:51:00 AM > > > > > > > > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 6, 2014 at 11:03 PM, Gilad Chaplik > > > > > wrote: > > > > >> > > > > >> - Original Message - > > > > >> > From: "Liran Zelkha" > > > > >> > To: "Gilad Chaplik" > > > > >> > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > > > > >> > "engine-devel" > > > > >> > Sent: Sunday, April 6, 2014 8:51:02 PM > > > > >> > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > >> > > > > > >> > On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik > > > > >> > > > > > >> > wrote: > > > > >> > > > > > >> > > - Original Message - > > > > >> > > > From: "Liran Zelkha" > > > > >> > > > To: "Kobi Ianko" > > > > >> > > > Cc: "Gilad Chaplik" , > > > > >> > > > de...@linode01.ovirt.org, > > > > >> > > "engine-devel" > > > > >> > > > Sent: Sunday, April 6, 2014 3:40:13 PM > > > > >> > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > >> > > > > > > > >> > > > On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Joining in... > > > > >> > > > > From my point of view, in real life a user should have that > > > > >> > > > > many > > > > >> > > > > VDSs > > > > >> > > on > > > > >> > > > > one Engine (from a DB point of view). > > > > >> > > > > Modern DB system handles tables with millions of records and > > > > >> > > > > many > > > > >> > > > > relations, Do we really have a performance issue here? > > > > >> > > > > We could prefer a more easy to maintain implantation in
Re: [ovirt-devel] Planned Gerrit maintenance
On 04/10/2014 01:38 PM, Itamar Heim wrote: On 04/09/2014 11:35 AM, Itamar Heim wrote: planning to reboot gerrit tomorrow morning (GMT-ish) for some system updates restarting gerrit service (and machine) now one more thing - I'll probably upgrade to 2.8.3 on sunday: 1. this patch alone is worth the ugprade... Fix incompatibility between "Rebase if Necessary" and "copy scores". When a project was set up with "Rebase if Necessary", one of its labels had copyAllScoresOnTrivialRebase or copyMaxScore, and a change that actually needed a trivial rebase was submitted, Gerrit first rebased the change, and in the process copied the approval for the label. It then copied all the approvals, including the one already copied, which resulted in a constraint violation on the database. https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNotes-2.8.2.html 2. but i'll upgrade directly to 2.8.3: https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNotes-2.8.3.html ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Planned Gerrit maintenance
On 04/10/2014 01:38 PM, Itamar Heim wrote: On 04/09/2014 11:35 AM, Itamar Heim wrote: planning to reboot gerrit tomorrow morning (GMT-ish) for some system updates restarting gerrit service (and machine) now ___ all done now. ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Planned Gerrit maintenance
On 04/09/2014 11:35 AM, Itamar Heim wrote: planning to reboot gerrit tomorrow morning (GMT-ish) for some system updates restarting gerrit service (and machine) now ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [VDSM] cleaning statistics retrieval
On Apr 10, 2014, at 12:25 , Dan Kenigsberg wrote: > On Wed, Apr 09, 2014 at 08:40:24AM -0400, Francesco Romani wrote: >> - Original Message - >>> From: "Dan Kenigsberg" >>> To: "Francesco Romani" >>> Cc: "vdsm-devel" , devel@ovirt.org >>> Sent: Tuesday, April 8, 2014 6:57:15 PM >>> Subject: Re: cleaning statistics retrieval >>> >>> On Tue, Apr 08, 2014 at 06:52:50AM -0400, Francesco Romani wrote: Hello VDSM developers, >>> >>> Please use devel@ovirt, and mention "vdsm" at the subject. This thread >>> in particular involves Engine as well. >> >> Right, I forgot. Sorry about that. >> I'd like to discuss and explain the plans for cleaning up Vm.getStats() in vdsm/virt/vm.py, and how it affects a bug we have: https://bugzilla.redhat.com/show_bug.cgi?id=1073478 Let's start from the bug. To make a long story short, there is a (small) race in VDSM, probably introduced by commit 8fedf8bde3c28edb07add23c3e9b72681cb48e49 The race can actually be triggered only if the VM is shutting down, so it is not that bad. Fixing the reported issue in the specific case can be done with a trivial if, and that it what I did in my initial http://gerrit.ovirt.org/#/c/25803/1/vdsm/vm.py,cm >>> >>> Could you explain how an AttributeError there moved the VM to Down? >> >> This should actually be this bug of engine >> https://bugzilla.redhat.com/show_bug.cgi?id=1072282 >> if GetVmStats fails for whatever reason, engine thinks the VM is down. > > Could you rename the Vdsm bug to exprese the in-vdsm problem, and make > clear that it confuses older Engines to think the Vm is Down? > >> And this is the core of the issue. My initial idea is to either return success and a complete, well formed statistics set, or return an error. However current engine seems to not cope properly with this, and we cannot break backward compatibility. >>> Would you be more precise? If getAllVmStats returns an errCode for one >>> of the Vms, what happens? >> >> Of course. >> >> For GetAllVmStats, AFAIK, but please correct me if I am wrong, because I'm >> not really expert on engine side, >> engine simply does not expects anything different from a list >> of either a RunningVmStats or an ExitedVmStats. >> >> So not sure (will verify just after this mail) if engine can cope with mixed >> content, >> meaning [ stats, errCode, stats, stats ... ] >> >> For GetVmStats, like Michal said, the engine expects the call to succeed >> otherwise >> it puts the VM into the Unknown state. >> >>> Looks like the only way to go is to always return success and to add a field to describe the content of the statistics (partial, complete...). THis is admittedly a far cry from the ideal solution, but it is dictated by the need to preserve the compatibility with current/old engines. >>> >>> I don't think that I understand your suggestion, but it does not sound >>> right to send a partial dictionary and to "appologize" about its >>> paritiality elsewhere. The dictionary may be "partial" for engine-4.0 >>> yet "complete" for engine-3.5. It's not for Vdsm to grade its own >>> output. >> >> I see your point (that's one of the reasons I'm not happy about this >> solution). >> Please see below for the detauls. >> please note that I'm not really happy about this solution, but, given the constraint, I don't see better alternatives. >>> >>> Please explain the benefits of describing the partial content, as I do >>> not see them. >> >> The root issue here is the getStats must always succeed, because the engine >> doesn't >> expect otherwise and thus we guarantee this to cope with old engines; >> but inside VDSM, getStats can actually fail in a lot of places >> (like in this case getBalloonInfo) >> >> So, in VDSM we can end up with a partial response, and now the question >> is: what to do with this partial response? And if there are optional fields >> in the response, how can the engine distinguish between >> >> * full response, but with an optional field missing >> >> and >> >> * partial response (because of an exception inside VDSM), >> without some field, incidentally including an optional one > > >> >> ? >> >> The VDSM 'grading' was an hint from VDSM to help engine to distinguish >> between those cases. >> >> Even if we agree this grading idea is bad, the core issue remains open: >> what to do if we end up with a partial response? >> For example, let's say we handle the getBalloonInfo exception >> (http://gerrit.ovirt.org/#/c/26539/), >> the stats object to be returned will lack >> >> * the (mandatory, expected) balloon stats >> * the (optional) migration progress - ok, bad example because this makes >> sense only during migrations, >> but other optional fields may be added later and the issue remains >> >> Again, anyone feel free to correct me if I misunderstood something about >> engine >> (o
Re: [ovirt-devel] [VDSM] cleaning statistics retrieval
On Wed, Apr 09, 2014 at 08:40:24AM -0400, Francesco Romani wrote: > - Original Message - > > From: "Dan Kenigsberg" > > To: "Francesco Romani" > > Cc: "vdsm-devel" , devel@ovirt.org > > Sent: Tuesday, April 8, 2014 6:57:15 PM > > Subject: Re: cleaning statistics retrieval > > > > On Tue, Apr 08, 2014 at 06:52:50AM -0400, Francesco Romani wrote: > > > Hello VDSM developers, > > > > Please use devel@ovirt, and mention "vdsm" at the subject. This thread > > in particular involves Engine as well. > > Right, I forgot. Sorry about that. > > > > I'd like to discuss and explain the plans for cleaning up Vm.getStats() > > > in vdsm/virt/vm.py, and how it affects a bug we have: > > > https://bugzilla.redhat.com/show_bug.cgi?id=1073478 > > > > > > Let's start from the bug. > > > > > > To make a long story short, there is a (small) race in VDSM, probably > > > introduced by commit > > > 8fedf8bde3c28edb07add23c3e9b72681cb48e49 > > > The race can actually be triggered only if the VM is shutting down, so it > > > is not that bad. > > > > > > Fixing the reported issue in the specific case can be done with a trivial > > > if, and that it what I did > > > in my initial http://gerrit.ovirt.org/#/c/25803/1/vdsm/vm.py,cm > > > > Could you explain how an AttributeError there moved the VM to Down? > > This should actually be this bug of engine > https://bugzilla.redhat.com/show_bug.cgi?id=1072282 > if GetVmStats fails for whatever reason, engine thinks the VM is down. Could you rename the Vdsm bug to exprese the in-vdsm problem, and make clear that it confuses older Engines to think the Vm is Down? > > > > And this is the core of the issue. > > > My initial idea is to either return success and a complete, well formed > > > statistics set, or return an error. > > > However current engine seems to not cope properly with this, and we cannot > > > break backward compatibility. > > Would you be more precise? If getAllVmStats returns an errCode for one > > of the Vms, what happens? > > Of course. > > For GetAllVmStats, AFAIK, but please correct me if I am wrong, because I'm > not really expert on engine side, > engine simply does not expects anything different from a list > of either a RunningVmStats or an ExitedVmStats. > > So not sure (will verify just after this mail) if engine can cope with mixed > content, > meaning [ stats, errCode, stats, stats ... ] > > For GetVmStats, like Michal said, the engine expects the call to succeed > otherwise > it puts the VM into the Unknown state. > > > > > > > > > Looks like the only way to go is to always return success and to add a > > > field to describe the content of the > > > statistics (partial, complete...). THis is admittedly a far cry from the > > > ideal solution, but it is dictated > > > by the need to preserve the compatibility with current/old engines. > > > > I don't think that I understand your suggestion, but it does not sound > > right to send a partial dictionary and to "appologize" about its > > paritiality elsewhere. The dictionary may be "partial" for engine-4.0 > > yet "complete" for engine-3.5. It's not for Vdsm to grade its own > > output. > > I see your point (that's one of the reasons I'm not happy about this > solution). > Please see below for the detauls. > > > > please note that I'm not really happy about this solution, but, given the > > > constraint, I don't see better alternatives. > > > > Please explain the benefits of describing the partial content, as I do > > not see them. > > The root issue here is the getStats must always succeed, because the engine > doesn't > expect otherwise and thus we guarantee this to cope with old engines; > but inside VDSM, getStats can actually fail in a lot of places > (like in this case getBalloonInfo) > > So, in VDSM we can end up with a partial response, and now the question > is: what to do with this partial response? And if there are optional fields > in the response, how can the engine distinguish between > > * full response, but with an optional field missing > > and > > * partial response (because of an exception inside VDSM), > without some field, incidentally including an optional one > > ? > > The VDSM 'grading' was an hint from VDSM to help engine to distinguish > between those cases. > > Even if we agree this grading idea is bad, the core issue remains open: > what to do if we end up with a partial response? > For example, let's say we handle the getBalloonInfo exception > (http://gerrit.ovirt.org/#/c/26539/), > the stats object to be returned will lack > > * the (mandatory, expected) balloon stats > * the (optional) migration progress - ok, bad example because this makes > sense only during migrations, > but other optional fields may be added later and the issue remains > > Again, anyone feel free to correct me if I misunderstood something about > engine > (or VDSM <=> engine communication) and to suggest better alternatives :\ Currently, we have way too many
[ovirt-devel] oVirt 3.3 entering security fix only phase
Hi, As you should know, we released oVirt 3.3.5 yesterday. After this release, only security fixes will be allowed on 3.3 branch and only a security update will cause a new oVirt 3.3.z release. It's time to focus on new features for 3.5.0! Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [Devel] Vdsm functional tests
On Thu, Apr 10, 2014 at 02:34:12AM -0400, Francesco Romani wrote: > > > >> When could we expect to have it running per commit on a Jenkins slaves? > > > > virt tests are running > > There is a common initialization issue with VdsProxy at the moment, once > > it's > > fixed the virt tests can be un-silenced and they will hopefully come out of > > "failing-only" mode:) > > I'd say end of this week... > > Confirmed. > This http://gerrit.ovirt.org/#/c/26514/ should fix the current VdsProxy > troubles Please find the time to review it, then! ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] new feature
I got a question regarding general mac handling: can you use the same macs on different datacenters? Am 10.04.2014 08:59, schrieb Martin Mucha: > Hi, > > I'd like to notify you about new feature, which allows to specify distinct > MAC pools, currently one per data center. > http://www.ovirt.org/Scoped_MacPoolManager > > any comments/proposals for improvement are very welcomed. > Martin. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [Devel] [Engine-devel] vds_dynamic refactor
- Original Message - > From: "Yaniv Dary" > To: "Gilad Chaplik" > Cc: "Liran Zelkha" , "Kobi Ianko" , > de...@linode01.ovirt.org, "engine-devel" > > Sent: Thursday, April 10, 2014 11:44:32 AM > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > - Original Message - > > From: "Gilad Chaplik" > > To: "Liran Zelkha" > > Cc: "Yaniv Dary" , "Kobi Ianko" , > > de...@linode01.ovirt.org, "engine-devel" > > > > Sent: Thursday, April 10, 2014 11:07:12 AM > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > - Original Message - > > > From: "Liran Zelkha" > > > To: "Yaniv Dary" > > > Cc: "Gilad Chaplik" , "Kobi Ianko" > > > , > > > de...@linode01.ovirt.org, "engine-devel" > > > > > > Sent: Wednesday, April 9, 2014 4:22:39 PM > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > On Wed, Apr 9, 2014 at 3:34 PM, Yaniv Dary wrote: > > > > > > > > Why not move only status with changes a lot to statistics and leave > > > > everything as is? > > > > > > > > > > > Exactly. No need for a new table. Use the existing ones. > > > > Why should the dwh read entire dynamic record for each status > > change/available resources change? (for VMs as well). > > The DWH doesn't work like that. We create views over dynamic\statistics and > dynamic\static they go the the configuration and stats tables on the history > db side. > From dynamic\statistics we collect every minute no matter what and from > dynamic\static we collect only when update_date changes in static. dynamic\statistics - dynamic\static to fully understand it, we have 2 triggers to collect from dynamic? > We can not rely on the dynamic update_date since it changes so much, so from > my point of view if we can cause minimal changes in dynamic and move status > to statistics it would the best thing. > What we have currently is a task that run once a hour and checks if any > change was made in the dynamic table (via very ugly joins and rejects) and > sync is there was any change. > It would even be better if you also move swap_size which doesn't change much > to dynamic. > oh... I thought that we had a trigger to dynamic (if ^ is correct we have). +1 (out of 1) for you comment :-) and thanks Yaniv the elaborate explanation! > > > > > > > > > > > > > > > > > Yaniv > > > > > > > > > > > > > > > > From: "Liran Zelkha" > > > > To: "Gilad Chaplik" > > > > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > > > > "engine-devel" > > > > Sent: Monday, April 7, 2014 8:51:00 AM > > > > > > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > > > > > > > > > > > > > > > On Sun, Apr 6, 2014 at 11:03 PM, Gilad Chaplik > > > > wrote: > > > >> > > > >> - Original Message - > > > >> > From: "Liran Zelkha" > > > >> > To: "Gilad Chaplik" > > > >> > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > > > >> > "engine-devel" > > > >> > Sent: Sunday, April 6, 2014 8:51:02 PM > > > >> > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > >> > > > > >> > On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik > > > >> > wrote: > > > >> > > > > >> > > - Original Message - > > > >> > > > From: "Liran Zelkha" > > > >> > > > To: "Kobi Ianko" > > > >> > > > Cc: "Gilad Chaplik" , > > > >> > > > de...@linode01.ovirt.org, > > > >> > > "engine-devel" > > > >> > > > Sent: Sunday, April 6, 2014 3:40:13 PM > > > >> > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > >> > > > > > > >> > > > On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Joining in... > > > >> > > > > From my point of view, in real life a user should have that > > > >> > > > > many > > > >> > > > > VDSs > > > >> > > on > > > >> > > > > one Engine (from a DB point of view). > > > >> > > > > Modern DB system handles tables with millions of records and > > > >> > > > > many > > > >> > > > > relations, Do we really have a performance issue here? > > > >> > > > > We could prefer a more easy to maintain implantation in this > > > >> > > > > case > > > >> > > > > over > > > >> > > DB > > > >> > > > > performance > > > >> > > > > > > > >> > > > > Yes we do. We make many queries on the VDS view, which is a > > > >> > > > > VERY > > > >> > > complex > > > >> > > > view. > > > >> > > > > > > >> > > > > > >> > > Actually I quite agree with Kobi, what is the plan for VMs? why do > > > >> > > we > > > >> > > start with VDS... > > > >> > > what is the biggest deploy do you know of? > > > >> > > > > > >> > We start with VDS because in an idle system, with 200 hosts and > > > >> > several > > > >> > thousands VMs, this is what you get as the top queries against the > > > >> > database. Look at how many times getvds is called. > > > >> > [image: Inline image 1] > > > >> > BTW - the second query is an example of abusing the dynamic query > > > >> > mechanism. The 4th query (an update command) is a set of useless > > > >> > update
Re: [ovirt-devel] Please help us to review our restful API design with NUMA feature on ovirt
On 04/10/2014 09:47 AM, Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC)
wrote:
> Hi Juan,
>
>
>
> Please help us to review our restful API design with NUMA feature on ovirt
>
> http://www.ovirt.org/Features/Detailed_NUMA_and_Virtual_NUMA#Interface_and_data_structure_in_restful_API
>
>
>
> We are appreciate that you or some community members could give us some
> feedback and comments, and sorry for the nag.
>
>
>
> *Best Regards,
> *Jason Liao
>
>
>
At the moment the RESTAPI design there is very light, so there is very
little to comment. Only thing I can recommend at the moment is to use
longer and clearer names for URL segments:
/hosts/{host:id}/numanodes
/vms/{vm:id}/numanodes
Do you have any additional information about how you plan to implement
the RESTAPI for NUMA?
--
Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3ºD, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [vdsm] cleaning statistics retrieval
Sorry if I'm oversimplicating this, as I'm no core dev, so please ignore or correct me where I'm wrong: Can't you simply define a versioned interface between vdsm and engine ? this interface can be split up in different sections: section a) must be reported : since 3.5 item 1 : all versions item 2 : since 3.4 item 3 : since 3.5 section b) can be reported if data is available : since 3.5 item 1 item 2 section c) ??? as you can see, you can even version the sections with all data prior to the integration of this interface falling into section a) or a special section for backwards compatibility. If such a thing doesn't exist yet it's maybe hard to introduce, but worth it. HTH Am 09.04.2014 16:25, schrieb Dan Kenigsberg: > But Vdsm cannot make this decision. Soon, Vdsm is to report the host's > boot time. Now assume that Vdsm fails to do so. Is the stats "partial"? > It's partial for engine-3.5, but it's complete for engine-3.4. > > Vdsm should tell as much of the truth that it knows. > > We could extend the "alerts" mechanism to report non-lethal errors in > getVmStats (we currently have it only in for storage domain status), > where Engine is told what's missing and why. I'm not sure if this is > really needed, though. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [Devel] [Engine-devel] vds_dynamic refactor
- Original Message - > From: "Gilad Chaplik" > To: "Liran Zelkha" > Cc: "Yaniv Dary" , "Kobi Ianko" , > de...@linode01.ovirt.org, "engine-devel" > > Sent: Thursday, April 10, 2014 11:07:12 AM > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > - Original Message - > > From: "Liran Zelkha" > > To: "Yaniv Dary" > > Cc: "Gilad Chaplik" , "Kobi Ianko" , > > de...@linode01.ovirt.org, "engine-devel" > > > > Sent: Wednesday, April 9, 2014 4:22:39 PM > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > On Wed, Apr 9, 2014 at 3:34 PM, Yaniv Dary wrote: > > > > > > Why not move only status with changes a lot to statistics and leave > > > everything as is? > > > > > > > > Exactly. No need for a new table. Use the existing ones. > > Why should the dwh read entire dynamic record for each status > change/available resources change? (for VMs as well). The DWH doesn't work like that. We create views over dynamic\statistics and dynamic\static they go the the configuration and stats tables on the history db side. >From dynamic\statistics we collect every minute no matter what and from >dynamic\static we collect only when update_date changes in static. We can not rely on the dynamic update_date since it changes so much, so from my point of view if we can cause minimal changes in dynamic and move status to statistics it would the best thing. What we have currently is a task that run once a hour and checks if any change was made in the dynamic table (via very ugly joins and rejects) and sync is there was any change. It would even be better if you also move swap_size which doesn't change much to dynamic. > > > > > > > > > > > > Yaniv > > > > > > > > > > > > From: "Liran Zelkha" > > > To: "Gilad Chaplik" > > > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > > > "engine-devel" > > > Sent: Monday, April 7, 2014 8:51:00 AM > > > > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > > > > > > > > > > On Sun, Apr 6, 2014 at 11:03 PM, Gilad Chaplik > > > wrote: > > >> > > >> - Original Message - > > >> > From: "Liran Zelkha" > > >> > To: "Gilad Chaplik" > > >> > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > > >> > "engine-devel" > > >> > Sent: Sunday, April 6, 2014 8:51:02 PM > > >> > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > >> > > > >> > On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik > > >> > wrote: > > >> > > > >> > > - Original Message - > > >> > > > From: "Liran Zelkha" > > >> > > > To: "Kobi Ianko" > > >> > > > Cc: "Gilad Chaplik" , > > >> > > > de...@linode01.ovirt.org, > > >> > > "engine-devel" > > >> > > > Sent: Sunday, April 6, 2014 3:40:13 PM > > >> > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > >> > > > > > >> > > > On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko > > >> > > > wrote: > > >> > > > > > >> > > > > Joining in... > > >> > > > > From my point of view, in real life a user should have that many > > >> > > > > VDSs > > >> > > on > > >> > > > > one Engine (from a DB point of view). > > >> > > > > Modern DB system handles tables with millions of records and > > >> > > > > many > > >> > > > > relations, Do we really have a performance issue here? > > >> > > > > We could prefer a more easy to maintain implantation in this > > >> > > > > case > > >> > > > > over > > >> > > DB > > >> > > > > performance > > >> > > > > > > >> > > > > Yes we do. We make many queries on the VDS view, which is a VERY > > >> > > complex > > >> > > > view. > > >> > > > > > >> > > > > >> > > Actually I quite agree with Kobi, what is the plan for VMs? why do > > >> > > we > > >> > > start with VDS... > > >> > > what is the biggest deploy do you know of? > > >> > > > > >> > We start with VDS because in an idle system, with 200 hosts and > > >> > several > > >> > thousands VMs, this is what you get as the top queries against the > > >> > database. Look at how many times getvds is called. > > >> > [image: Inline image 1] > > >> > BTW - the second query is an example of abusing the dynamic query > > >> > mechanism. The 4th query (an update command) is a set of useless > > >> > update_vds_dynamic commands. > > >> > > > >> > For reference, the explain plan of get VDS is something like this: > > >> > > > >> > QUERY PLAN > > >> > > > >> > --- > > >> > Nested Loop (cost=9.30..46.75 rows=6 width=9060) (actual > > >> > time=0.063..0.068 rows=1 loops=1) > > >> >Join Filter: (vds_static.vds_id = vds_statistics.vds_id) > > >> >-> Seq Scan on vds_statistics (cost=0.00..1.01 rows=1 width=109) > > >> > (actual time=0.008..0.008 rows=1 loops=1) > > >> >-> Nested Loop (cost=9.30..45.64 rows=6 width=8983) (actual > > >> > time=0.048..0.052 rows=1 loops=1) > > >> > Join Filter: (vds_groups.vds_group_id = > > >
Re: [ovirt-devel] [Devel] Attention: Mailing List Change
On Wed 09 Apr 2014 10:36:29 PM CEST, Alon Bar-Lev wrote: > > > - Original Message - >> From: "Alon Bar-Lev" >> To: "Itamar Heim" >> Cc: devel@ovirt.org >> Sent: Wednesday, April 9, 2014 3:22:41 PM >> Subject: Re: [ovirt-devel] [Devel] Attention: Mailing List Change >> - Original Message - > From: "Brian Proffitt" > To: "Itamar Heim" > Cc: "David Caro" , devel@ovirt.org, "Alon Bar-Lev" > > Sent: Wednesday, April 9, 2014 3:14:46 PM > Subject: Re: [Devel] Attention: Mailing List Change > > Well, shoot, I can change that now. > > The prefix [devel] will now be [ovirt-devel]. Can you please do this for all? for example users. >>> >>> I think only devel and users are interesting. the rest are are either >>> already with engine prefix (engine-patches), or very low traffic >>> (announce, etc.). >> >> especially announce... :) >> > > As I got no response, so I will try to explain using two arguments... > > 1. People that subscribe to announce usually have no filters as it is low > traffic, so it is ovirt prefix should be added to differentiate from other > activity. > +1 > 2. Consistency... important for engineering and for professional appearance. > Exceptions should have good reason. There is no good reason why not all ovirt > mailing list share the same prefix, easier to search, comfortable as eye > catcher etc. +1 > > Not sure what your argument not to have plain consistent prefix for all. > > I would like to emphasis the obvious move that is long overdue to host all > lists at ovirt.org[1], the share of experience of people is important, I > still do not understand why we do not use our human resources and experience > to understand how to improve. This applies to mailing list, our site > organization, our bug management, our release engineering, which still do not > line up with best practises. > > Regards, > Alon > > [1] http://lists.ovirt.org/pipermail/infra/2012-December/001688.html > ___ > Devel mailing list > Devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 signature.asc Description: OpenPGP digital signature ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [Devel] Proposing Yedidyah Bar David as oVirt Hosted Engine Setup maintainer
On Wed 09 Apr 2014 11:18:53 PM CEST, Greg Padgett wrote: > On 04/08/2014 02:08 AM, Sandro Bonazzola wrote: >> Hi, >> >> I would like to propose Yedidyah Bar David as oVirt Hosted Engine Setup >> co-maintainer. >> >> Yedidyah contributed to oVirt Hosted Engine Setup since early design phase >> and contributed dozens of patches. >> >> Your response would be appreciated. >> >> Thanks in advance. >> > > +1! > > ___ > Devel mailing list > Devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel I think we have enough votes :) -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 signature.asc Description: OpenPGP digital signature ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [Devel] Vdsm functional tests
On Thu 10 Apr 2014 09:55:21 AM CEST, Francesco Romani wrote: > - Original Message - >> From: "Francesco Romani" >> To: vdsm-de...@ovirt.org, devel@ovirt.org >> Sent: Thursday, April 10, 2014 8:43:46 AM >> Subject: Re: [ovirt-devel] [Devel] Vdsm functional tests >> >>> Plus ideally a better initialization to get rid of that embarrassing "sleep >>> 10" in gerrit config. We should check and wait till vdsm responds in >>> VdsProxy init. >>> We need to make the test as fast as possible, otherwise people won't run >>> them... >> >> Agreed and added to my TODO > > http://gerrit.ovirt.org/#/c/26638/ > @Toni: I'll fix a couple things (not needed branches and stuff) with the network tests and rename it to meet the "unwritten" consensus for the job names, is that ok? -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 signature.asc Description: OpenPGP digital signature ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [Devel] [Engine-devel] vds_dynamic refactor
- Original Message - > From: "Liran Zelkha" > To: "Yaniv Dary" > Cc: "Gilad Chaplik" , "Kobi Ianko" , > de...@linode01.ovirt.org, "engine-devel" > > Sent: Wednesday, April 9, 2014 4:22:39 PM > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > On Wed, Apr 9, 2014 at 3:34 PM, Yaniv Dary wrote: > > > > Why not move only status with changes a lot to statistics and leave > > everything as is? > > > > > Exactly. No need for a new table. Use the existing ones. Why should the dwh read entire dynamic record for each status change/available resources change? (for VMs as well). > > > > > > > Yaniv > > > > > > > > From: "Liran Zelkha" > > To: "Gilad Chaplik" > > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > > "engine-devel" > > Sent: Monday, April 7, 2014 8:51:00 AM > > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > > > > > > > > > > On Sun, Apr 6, 2014 at 11:03 PM, Gilad Chaplik wrote: > >> > >> - Original Message - > >> > From: "Liran Zelkha" > >> > To: "Gilad Chaplik" > >> > Cc: "Kobi Ianko" , de...@linode01.ovirt.org, > >> > "engine-devel" > >> > Sent: Sunday, April 6, 2014 8:51:02 PM > >> > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > >> > > >> > On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik > >> > wrote: > >> > > >> > > - Original Message - > >> > > > From: "Liran Zelkha" > >> > > > To: "Kobi Ianko" > >> > > > Cc: "Gilad Chaplik" , de...@linode01.ovirt.org, > >> > > "engine-devel" > >> > > > Sent: Sunday, April 6, 2014 3:40:13 PM > >> > > > Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor > >> > > > > >> > > > On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko wrote: > >> > > > > >> > > > > Joining in... > >> > > > > From my point of view, in real life a user should have that many > >> > > > > VDSs > >> > > on > >> > > > > one Engine (from a DB point of view). > >> > > > > Modern DB system handles tables with millions of records and many > >> > > > > relations, Do we really have a performance issue here? > >> > > > > We could prefer a more easy to maintain implantation in this case > >> > > > > over > >> > > DB > >> > > > > performance > >> > > > > > >> > > > > Yes we do. We make many queries on the VDS view, which is a VERY > >> > > complex > >> > > > view. > >> > > > > >> > > > >> > > Actually I quite agree with Kobi, what is the plan for VMs? why do we > >> > > start with VDS... > >> > > what is the biggest deploy do you know of? > >> > > > >> > We start with VDS because in an idle system, with 200 hosts and several > >> > thousands VMs, this is what you get as the top queries against the > >> > database. Look at how many times getvds is called. > >> > [image: Inline image 1] > >> > BTW - the second query is an example of abusing the dynamic query > >> > mechanism. The 4th query (an update command) is a set of useless > >> > update_vds_dynamic commands. > >> > > >> > For reference, the explain plan of get VDS is something like this: > >> > > >> > QUERY PLAN > >> > > >> > --- > >> > Nested Loop (cost=9.30..46.75 rows=6 width=9060) (actual > >> > time=0.063..0.068 rows=1 loops=1) > >> >Join Filter: (vds_static.vds_id = vds_statistics.vds_id) > >> >-> Seq Scan on vds_statistics (cost=0.00..1.01 rows=1 width=109) > >> > (actual time=0.008..0.008 rows=1 loops=1) > >> >-> Nested Loop (cost=9.30..45.64 rows=6 width=8983) (actual > >> > time=0.048..0.052 rows=1 loops=1) > >> > Join Filter: (vds_groups.vds_group_id = > >> > vds_static.vds_group_id) > >> > -> Nested Loop Left Join (cost=0.00..9.29 rows=1 width=1389) > >> > (actual time=0.013..0.013 rows=1 loops=1) > >> >-> Seq Scan on vds_groups (cost=0.00..1.01 rows=1 > >> > width=1271) (actual time=0.003..0.003 rows=1 loops=1) > >> >-> Index Scan using pk_storage_pool on storage_pool > >> > (cost=0.00..8.27 rows=1 width=134) (actual time=0.008..0.008 rows=1 > >> > loops=1) > >> > Index Cond: (vds_groups.storage_pool_id = id) > >> > -> Hash Right Join (cost=9.30..36.28 rows=6 width=7610) > >> > (actual > >> > time=0.033..0.037 rows=1 loops=1) > >> >Hash Cond: (vds_spm_id_map.vds_id = vds_static.vds_id) > >> >-> Seq Scan on vds_spm_id_map (cost=0.00..22.30 > >> >rows=1230 > >> > width=20) (actual time=0.003..0.003 rows=1 loops=1) > >> >-> Hash (cost=9.29..9.29 rows=1 width=7606) (actual > >> > time=0.019..0.019 rows=1 loops=1) > >> > Buckets: 1024 Batches: 1 Memory Usage: 2kB > >> > -> Nested Loop (cost=0.00..9.29 rows=1 > >> > width=7606) > >> > (actual time=0.012..0.013 rows=1 loops=1) > >> >-> Seq Scan on vds_dynami
Re: [ovirt-devel] [Devel] Vdsm functional tests
- Original Message - > From: "Francesco Romani" > To: vdsm-de...@ovirt.org, devel@ovirt.org > Sent: Thursday, April 10, 2014 8:43:46 AM > Subject: Re: [ovirt-devel] [Devel] Vdsm functional tests > > > Plus ideally a better initialization to get rid of that embarrassing "sleep > > 10" in gerrit config. We should check and wait till vdsm responds in > > VdsProxy init. > > We need to make the test as fast as possible, otherwise people won't run > > them... > > Agreed and added to my TODO http://gerrit.ovirt.org/#/c/26638/ -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
[ovirt-devel] Please help us to review our restful API design with NUMA feature on ovirt
Hi Juan, Please help us to review our restful API design with NUMA feature on ovirt http://www.ovirt.org/Features/Detailed_NUMA_and_Virtual_NUMA#Interface_and_data_structure_in_restful_API We are appreciate that you or some community members could give us some feedback and comments, and sorry for the nag. Best Regards, Jason Liao ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [ovirt-users] Feature Page: Mac Pool per DC (was: new feature)
On 04/10/2014 09:59 AM, Martin Mucha wrote: Hi, I'd like to notify you about new feature, which allows to specify distinct MAC pools, currently one per data center. http://www.ovirt.org/Scoped_MacPoolManager any comments/proposals for improvement are very welcomed. Martin. ___ Users mailing list us...@ovirt.org http://lists.ovirt.org/mailman/listinfo/users (changed title to reflect content) When specified mac ranges for given "scope", where there wasn't any definition previously, allocated MAC from default pool will not be moved to "scoped" one until next engine restart. Other way, when removing "scoped" mac pool definition, all MACs from this pool will be moved to default one. cna you please elaborate on this one? as for potential other "scopes" - i can think of cluster, vm pool and logical network as potential ones. one more question - how do you know to "return" the mac address to the correct pool on delete? ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
[ovirt-devel] new feature
Hi, I'd like to notify you about new feature, which allows to specify distinct MAC pools, currently one per data center. http://www.ovirt.org/Scoped_MacPoolManager any comments/proposals for improvement are very welcomed. Martin. ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
