Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt
- Original Message - From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com To: Gilad Chaplik gchap...@redhat.com, Eli Mesika emes...@redhat.com Cc: Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 3, 2014 7:25:11 AM Subject: RE: Please help us to review our database schema design with NUMA feature on ovirt Hi all, Please see my comments in line. Thanks Best Regards Shi, Xiao-Lei (Bruce) Hewlett-Packard Co., Ltd. HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile +86 18696583447 Email xiao-lei@hp.com -Original Message- From: Gilad Chaplik [mailto:gchap...@redhat.com] Sent: Thursday, April 03, 2014 9:05 AM To: Eli Mesika Cc: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ); Roy Golan; Omer Frenkel; Vinod, Chegu; Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); Doron Fediuck; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Yaniv Dary; engine-devel@ovirt.org Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt Hi all, Sorry for joining-in late. My comments (according to the db diagram section in https://docs.google.com/document/d/1-wdDkm6EDbwyoCIRPPcmbGWAcyQo_ISTY8ykDr0I6VY): 1) Join vm_numa_node and vds_numa_node to a single table (almost identical), one of the FKs can be null. [Bruce] I prefer two tables. Actually host level NUMA node and vm level NUMA node are different objects. In my understanding, vm level NUMA node is just a simulation of host level NUMA node, and host level NUMA node has more features that not in vm NUMA (like several levels of host NUMA topology mentioned by Vinod). We need to consider the extensions of host NUMA in the future. I agree with Bruce, we have no problem with more tables and constrains should work as expected and remove entries when a Host or VM is removed. I do not like tables that have 2 UUIDs when one of them is null , this is against simple DB normalization 2) No templates reference in the design, need to check it out (it might be inherently designed already :-) ); vNode can be linked to a template. [Bruce] IMO, we can consider template as a special vm, our current design supports to create vNodes in a template just like what it does in a vm. We also store today templates in the VM* tables as special entities defined by the entity_type column 3) The reason I want host's NUMA data to be in static is because it updates only once (on boot). engineerically speaking, dynamic table has a lot of traffic and that's not the case for NUMA info. Its feels to me like 'a hybrid' of static and dynamic, 3 suggestions comes to mind: - leave it in dynamic (maybe in a separate process). - have a separate flow that updates static. - come up with a third 'vds_on_boot' table (my favorite ;-P ). I will get back to you on that. [Bruce] From the onTimer codes in vdsManager (IMO it's the scheduled vds refresh action), when vds is in normal running status (up status), only statistics data will be refreshed, so I think maybe dynamic table doesn't have so much traffic, and the varied data (I mean the data varied through a power reload, like cpu topology, numa topology, ...etc) can be refreshed meanwhile. 4) vm_vds_numa_node_map is connected to vds_numa_node_statistics, why to split the tables (vds_numa_node vds_numa_node_statistics), going back to comment #1, don't we want vNode stats as well, it can be nice to show it :-) (have a vNUMA overview of the VM using guest-agent or sth, in a future phase). [Bruce] Split the tables is referring to vds_interface (vds_interface_statistics) and vm_interface (vm_interface_statistics), the statistics table will be refreshed with a high frequency. I will update the design to add the vNuma statistics for the future phase, and according to my feedback in comment #1, vNuma statistics will also in an independent table since I think there will be more statistics fields for host NUMA than for vNuma. Agree with Bruce 5) IMO vds_cpu_statistics shouldn't include any reference to NUMA, I gave that comment in the BE patch as well (remove vds_numa_node_id FK in vds_cpu_statistics), for that you should extract cpu_list to a connection table (anyway I don't like lists as a text/strings/etc.) [Bruce] Yes, I agree with you to remove the reference between vds_cpu_statistics and NUMA. Actually cpu_list is enough for current requirements. I think the connection table is needed when we consider to collect more cpu related information and do more cpu related actions in the future. 6) vm_numatune_nodeset can be removed - the vNode should hold it's pinning info; I think
Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility...
On 04/02/2014 09:06 AM, Moti Asayag wrote: - Original Message - From: Eli Mesika emes...@redhat.com To: engine-devel@ovirt.org Sent: Thursday, March 27, 2014 1:43:22 PM Subject: Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility... - Original Message - From: Moti Asayag masa...@redhat.com To: Itamar Heim ih...@redhat.com Cc: Eli Mesika emes...@redhat.com, engine-devel@ovirt.org Sent: Sunday, March 23, 2014 2:47:53 PM Subject: Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility... - Original Message - From: Itamar Heim ih...@redhat.com To: Eli Mesika emes...@redhat.com, engine-devel@ovirt.org Sent: Sunday, March 23, 2014 12:14:37 PM Subject: Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility... On 03/23/2014 12:13 PM, Eli Mesika wrote: So, as far a I understand for resolving this bug the following is OK : Block downgrading if there are Hosts or Networks (other than the default management network) or SD in the DC when downgrading. yes I don't think it is enough. There is a need to verify the management network wasn't modified to use unsupported features in the new data-center. On the same matter, we can allow downgrading if all of the networks in the data center are using only features that are supported on the target DC version and not to restrict it only to the management network. Since Moti is in PTO, talked with Mike K to get advanced with that , we came to the following conclusion: when DC is downgraded : Block downgrading if there are Hosts or Networks (other than the default management network) or SD in the DC. In case that only default management network exists we should check that all network features are still supported (This can be done by adding a method to the NetworkValidator that will check for support of all relevant features) when Cluster is downgraded : Same as above , but we don't need to check for features validation since we can not downgrade below the DC version and therefor the cluster network is supposed to be valid already Mike, please let me know If I miss something from our discussion (and thanks for your kind help :-) ) +1 from me and Mike for the above. Eli - please note there is no need to block the DC downgrade if there are hosts in the DC. Thanks, Livnat - Original Message - From: Livnat Peer lp...@redhat.com To: Moti Asayag masa...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, March 21, 2014 8:33:59 AM Subject: Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility... On 03/20/2014 09:20 PM, Moti Asayag wrote: - Original Message - From: Moti Asayag masa...@redhat.com To: Livnat Peer lp...@redhat.com Cc: Itamar Heim ih...@redhat.com, engine-devel@ovirt.org Sent: Wednesday, March 12, 2014 10:44:07 PM Subject: Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility... - Original Message - From: Livnat Peer lp...@redhat.com To: Moti Asayag masa...@redhat.com Cc: Itamar Heim ih...@redhat.com, engine-devel@ovirt.org Sent: Wednesday, March 12, 2014 10:33:45 PM Subject: Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility... On 03/12/2014 10:20 PM, Moti Asayag wrote: - Original Message - From: Livnat Peer lp...@redhat.com To: Itamar Heim ih...@redhat.com, engine-devel@ovirt.org Sent: Wednesday, March 12, 2014 12:42:44 PM Subject: Re: [Engine-devel] Change in ovirt-engine[master]: core: enable to decrease DC compatibility... On 03/12/2014 11:59 AM, Itamar Heim wrote: On 03/12/2014 12:26 AM, emes...@redhat.com wrote: Eli Mesika has submitted this change and it was merged. Change subject: core: enable to decrease DC compatibility... .. core: enable to decrease DC compatibility... enable to decrease DC compatibility version if DC has no clusters This patch enables to decrease the DC compatibility version if DC has no clusters. Eli - just saw this. I'm pretty sure it would be *bad* to downgrade a DC version if it has storage domains as well. not sure if this is checked already or not. may also be an issue with some logical network features. Most of the network features are driven from cluster level, we enable using the features on all DC level (actually =3.1) but actually enable /disable the feature when attaching the network to a cluster. So from network perspective I think it should be fine to downgrade the DC level even if there are networks in the DC (at least now this could change in future versions). Actually we block adding or updating networks if the feature is not supported on the network's DC level, for example: STP, Jumbo frames and
Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt
- Original Message - From: Eli Mesika emes...@redhat.com To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com Cc: Gilad Chaplik gchap...@redhat.com, Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 3, 2014 10:54:54 AM Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt - Original Message - From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com To: Gilad Chaplik gchap...@redhat.com, Eli Mesika emes...@redhat.com Cc: Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 3, 2014 7:25:11 AM Subject: RE: Please help us to review our database schema design with NUMA feature on ovirt Hi all, Please see my comments in line. Thanks Best Regards Shi, Xiao-Lei (Bruce) Hewlett-Packard Co., Ltd. HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile +86 18696583447 Email xiao-lei@hp.com -Original Message- From: Gilad Chaplik [mailto:gchap...@redhat.com] Sent: Thursday, April 03, 2014 9:05 AM To: Eli Mesika Cc: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ); Roy Golan; Omer Frenkel; Vinod, Chegu; Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); Doron Fediuck; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Yaniv Dary; engine-devel@ovirt.org Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt Hi all, Sorry for joining-in late. My comments (according to the db diagram section in https://docs.google.com/document/d/1-wdDkm6EDbwyoCIRPPcmbGWAcyQo_ISTY8ykDr0I6VY): 1) Join vm_numa_node and vds_numa_node to a single table (almost identical), one of the FKs can be null. [Bruce] I prefer two tables. Actually host level NUMA node and vm level NUMA node are different objects. In my understanding, vm level NUMA node is just a simulation of host level NUMA node, and host level NUMA node has more features that not in vm NUMA (like several levels of host NUMA topology mentioned by Vinod). We need to consider the extensions of host NUMA in the future. Let's open the discussion and consider them right now. vNode and Node are the same. Vinod? I agree with Bruce, we have no problem with more tables and constrains should work as expected and remove entries when a Host or VM is removed. I do not like tables that have 2 UUIDs when one of them is null , this is against simple DB normalization We are going to maintain and develop this feature. there is a huge overhead in 6 table design in compare to 4. 2) No templates reference in the design, need to check it out (it might be inherently designed already :-) ); vNode can be linked to a template. [Bruce] IMO, we can consider template as a special vm, our current design supports to create vNodes in a template just like what it does in a vm. We also store today templates in the VM* tables as special entities defined by the entity_type column +1, just need to see that this is taken care of. 3) The reason I want host's NUMA data to be in static is because it updates only once (on boot). engineerically speaking, dynamic table has a lot of traffic and that's not the case for NUMA info. Its feels to me like 'a hybrid' of static and dynamic, 3 suggestions comes to mind: - leave it in dynamic (maybe in a separate process). - have a separate flow that updates static. - come up with a third 'vds_on_boot' table (my favorite ;-P ). I will get back to you on that. [Bruce] From the onTimer codes in vdsManager (IMO it's the scheduled vds refresh action), when vds is in normal running status (up status), only statistics data will be refreshed, so I think maybe dynamic table doesn't have so much traffic, and the varied data (I mean the data varied through a power reload, like cpu topology, numa topology, ...etc) can be refreshed meanwhile. I beg to differ, dynamic table contains a lot of dynamic data: host status, used memory and pending resources (maybe more). still need to think about it, and get back to you. 4) vm_vds_numa_node_map is connected to vds_numa_node_statistics, why to split the tables (vds_numa_node vds_numa_node_statistics), going back to comment #1, don't we want vNode stats as well, it can be nice to show it :-) (have a vNUMA overview of the VM using
[Engine-devel] [ANN][TechTalk] Two UI tech talks next week!
Dear oVirt community, next week is UI tech talk week! I've prepared two presentations: 1, oVirt JavaScript SDK (Design Proposal) Monday, April 7 - 3pm CET / 9am EST / 4pm TLV time 2, Writing UI plugin with AngularJS Tuesday, April 8 - 3pm CET / 9am EST / 4pm TLV time Of these two, the first one is really important, as it covers current proposal for oVirt JavaScript SDK including plans for near future. The second one is recommended for anyone who wants to learn about oVirt UI plugins, AngularJS and how they fit together. I'll send an invitation for both presentations shortly. PDF slides will be available on day of presentation. Regards, Vojtech ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] vds_dynamic refactor
Hi list, I propose to split vds_dynamic table into 2 tables: - vds_dynamic - vds_on_boot We need a place to put all non-dynamic data that gets updated once the host is booted, and I think dynamic isn't the place for it. In first phase we will put there NUMA host topoplogy, but later on migrate all non-dynamic host data (cpu, os, etc.). thoughts? Thanks, Gilad. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] vds_dynamic refactor
Why not move it to vds_static? On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik gchap...@redhat.com wrote: Hi list, I propose to split vds_dynamic table into 2 tables: - vds_dynamic - vds_on_boot We need a place to put all non-dynamic data that gets updated once the host is booted, and I think dynamic isn't the place for it. In first phase we will put there NUMA host topoplogy, but later on migrate all non-dynamic host data (cpu, os, etc.). thoughts? Thanks, Gilad. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] oVirt JavaScript SDK (Design Proposal)
BEGIN:VCALENDAR PRODID:Zimbra-Calendar-Provider VERSION:2.0 METHOD:REQUEST BEGIN:VTIMEZONE TZID:Europe/Belgrade BEGIN:STANDARD DTSTART:16010101T03 TZOFFSETTO:+0100 TZOFFSETFROM:+0200 RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=10;BYDAY=-1SU TZNAME:CET END:STANDARD BEGIN:DAYLIGHT DTSTART:16010101T02 TZOFFSETTO:+0200 TZOFFSETFROM:+0100 RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=3;BYDAY=-1SU TZNAME:CEST END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT UID:702d6cd9-6a76-453f-be35-f81545e5d30f SUMMARY:oVirt JavaScript SDK (Design Proposal) ATTENDEE;CN=users;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailt o:us...@ovirt.org ATTENDEE;CN=engine-devel;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRU E:mailto:engine-devel@ovirt.org ORGANIZER;CN=Vojtech Szocs:mailto:vsz...@redhat.com DTSTART;TZID=Europe/Belgrade:20140407T15 DTEND;TZID=Europe/Belgrade:20140407T17 STATUS:CONFIRMED CLASS:PUBLIC X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY TRANSP:OPAQUE LAST-MODIFIED:20140403T110820Z DTSTAMP:20140403T110820Z SEQUENCE:0 DESCRIPTION:The following is a new meeting request:\n\nSubject: oVirt JavaSc ript SDK (Design Proposal) \nOrganizer: Vojtech Szocs vsz...@redhat.com \n\nTime: Monday\, April 7\, 2014\, 3:00:00 PM - 5:00:00 PM GMT +01:00 Belgr ade\, Bratislava\, Budapest\, Ljubljana\, Prague\n \nInvitees: users@ovirt.o rg\; engine-devel@ovirt.org \n\n\n*~*~*~*~*~*~*~*~*~*\n\nHi guys\,\n\nwe're planning to move oVirt UI to use REST API [1] and this session is the first step in our journey.\n\nMake sure to join if you want to learn about our pla ns for oVirt JavaScript SDK and its use in web applications.\n\nTopics cover ed:\n* Java SDK overview\n* JavaScript SDK design overview\n* JavaScript Bin ding API proposal\n* Future plans\n\nEstimated duration: 2 hours\n\nTo join this meeting\, dial into the Intercall bridge and use following conference c ode: 712 886 7405 #\nhttps://www.intercallonline.com/listNumbersByCode.actio n?confCode=7128867405\n\nPDF slides will be available on day of presentation .\n\n[1] http://www.ovirt.org/Features/Design/Using_REST_API_In_Web_UI\n\nRe gards\,\nVojtech\n BEGIN:VALARM ACTION:DISPLAY TRIGGER;RELATED=START:-PT5M DESCRIPTION:Reminder END:VALARM END:VEVENT END:VCALENDAR___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Mailing List Change
All: We have started to implement the Arch/Engine-devel list changes. Arch and engine-devel list subscribers are now subscribed to devel. At 1330 UTC (a little over one hour from now), the engine-devel mailing list will be shut down and its archives moved to devel. Arch will remain in place, but all users there are recommended to start using devel for new conversations. Thanks, Brian -- Brian Proffitt - oVirt Community Manager Open Source and Standards, Red Hat - http://community.redhat.com Phone: +1 574 383 9BKP IRC: bkp @ OFTC ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] vds_dynamic refactor
- Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Liran Zelkha liran.zel...@gmail.com Cc: Gilad Chaplik gchap...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 2:12:59 PM Subject: Re: [Engine-devel] vds_dynamic refactor - Original Message - From: Liran Zelkha liran.zel...@gmail.com To: Gilad Chaplik gchap...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 2:04:29 PM Subject: Re: [Engine-devel] vds_dynamic refactor Why not move it to vds_static? +1 on Liran's comment. I would prefer not to add more complexity to the vds tables family. Such complexity may effect performs of queries/views. If you wish, you can create a view on top of vds_static named vds_on_boot for querying of vds on boot info. Yair That means moving almost all of vds_dynamic into vds_static except of memory, pending resources and status (maybe more but not much); and there will not be any db separation between user input and on_boot data. Thanks, Gilad. On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik gchap...@redhat.com wrote: Hi list, I propose to split vds_dynamic table into 2 tables: - vds_dynamic - vds_on_boot We need a place to put all non-dynamic data that gets updated once the host is booted, and I think dynamic isn't the place for it. In first phase we will put there NUMA host topoplogy, but later on migrate all non-dynamic host data (cpu, os, etc.). thoughts? Thanks, Gilad. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt
- Original Message - From: Gilad Chaplik gchap...@redhat.com To: Eli Mesika emes...@redhat.com Cc: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 3, 2014 1:46:32 PM Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt - Original Message - From: Eli Mesika emes...@redhat.com To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com Cc: Gilad Chaplik gchap...@redhat.com, Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 3, 2014 10:54:54 AM Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt - Original Message - From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com To: Gilad Chaplik gchap...@redhat.com, Eli Mesika emes...@redhat.com Cc: Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 3, 2014 7:25:11 AM Subject: RE: Please help us to review our database schema design with NUMA feature on ovirt Hi all, Please see my comments in line. Thanks Best Regards Shi, Xiao-Lei (Bruce) Hewlett-Packard Co., Ltd. HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile +86 18696583447 Email xiao-lei@hp.com -Original Message- From: Gilad Chaplik [mailto:gchap...@redhat.com] Sent: Thursday, April 03, 2014 9:05 AM To: Eli Mesika Cc: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ); Roy Golan; Omer Frenkel; Vinod, Chegu; Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); Doron Fediuck; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Yaniv Dary; engine-devel@ovirt.org Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt Hi all, Sorry for joining-in late. My comments (according to the db diagram section in https://docs.google.com/document/d/1-wdDkm6EDbwyoCIRPPcmbGWAcyQo_ISTY8ykDr0I6VY): 1) Join vm_numa_node and vds_numa_node to a single table (almost identical), one of the FKs can be null. [Bruce] I prefer two tables. Actually host level NUMA node and vm level NUMA node are different objects. In my understanding, vm level NUMA node is just a simulation of host level NUMA node, and host level NUMA node has more features that not in vm NUMA (like several levels of host NUMA topology mentioned by Vinod). We need to consider the extensions of host NUMA in the future. Let's open the discussion and consider them right now. vNode and Node are the same. Vinod? I agree with Bruce, we have no problem with more tables and constrains should work as expected and remove entries when a Host or VM is removed. I do not like tables that have 2 UUIDs when one of them is null , this is against simple DB normalization We are going to maintain and develop this feature. there is a huge overhead in 6 table design in compare to 4. Counting how many tables you should manage is out of context. The only thing I can care of is database design From DB design perspective if you have a main table A with PK PK-A if you have any other table X that includes PK-A as a column Cx , then there should be a foreign key constraint that handles removals from A and clean corresponding X table records accordingly. This you can not do when you have 2 UUIDs as you suggested (not to mention that this is considered as bad DB design with no doubt) Add to that the fact that when managing both UUIDs in one table, you are supposed to get deadlocks from scenarios that wouldn't cause that if each entity is managed in its own table. I don't see any point to open this to discussion, we had worked hard to separate tables that used this bad method in the past and caused us a bunch of unexpected problems. Please develop your DB changes according to normal well accepted guidelines I will not ack a change that violates simple DB design rules. Thanks 2) No templates reference in
Re: [Engine-devel] [Users] Writing UI plugin with AngularJS
(cc'ing devel list) Einav is right, there's 3.5 SLA features overview meeting starting at 4pm CET. I'll move the session one hour earlier, from 3-5pm to 2-4pm CET (to 8-10am EST). I'll also add devel list on CC list of both sessions. Vojtech - Original Message - From: Einav Cohen eco...@redhat.com To: Vojtech Szocs vsz...@redhat.com Cc: users us...@ovirt.org, engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 2:17:39 PM Subject: Re: [Users] Writing UI plugin with AngularJS Vojtech, this is conflicting with the '3.5 SLA features overview' meeting. maybe move this meeting to start one hour earlier (2:00 PM Brno time)? - Original Message - From: Vojtech Szocs vsz...@redhat.com To: users us...@ovirt.org, engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 7:15:43 AM Subject: [Users] Writing UI plugin with AngularJS The following is a new meeting request: Subject: Writing UI plugin with AngularJS Organizer: Vojtech Szocs vsz...@redhat.com Time: Tuesday, April 8, 2014, 3:00:00 PM - 5:00:00 PM GMT +01:00 Belgrade, Bratislava, Budapest, Ljubljana, Prague Invitees: us...@ovirt.org; engine-devel@ovirt.org *~*~*~*~*~*~*~*~*~* Hi guys, this session is for those of you interested in learning about oVirt UI plugins, AngularJS and how they fit together. Includes UI plugin AngularJS overview, writing UI plugin step-by-step and live demo. Topics covered: * AngularJS fly-through * oVirt UI plugins fly-through * UI plugin with AngularJS walk-through * Live demo Estimated duration: 1.5 hours To join this meeting, dial into the Intercall bridge and use following conference code: 712 886 7405 # https://www.intercallonline.com/listNumbersByCode.action?confCode=7128867405 PDF slides + source code will be available on day of presentation. Regards, Vojtech ___ Users mailing list us...@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Writing UI plugin with AngularJS
BEGIN:VCALENDAR PRODID:Zimbra-Calendar-Provider VERSION:2.0 METHOD:REQUEST BEGIN:VTIMEZONE TZID:Europe/Belgrade BEGIN:STANDARD DTSTART:16010101T03 TZOFFSETTO:+0100 TZOFFSETFROM:+0200 RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=10;BYDAY=-1SU TZNAME:CET END:STANDARD BEGIN:DAYLIGHT DTSTART:16010101T02 TZOFFSETTO:+0200 TZOFFSETFROM:+0100 RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=3;BYDAY=-1SU TZNAME:CEST END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT UID:feb29ae9-8446-4acc-94fe-f7f6e05417ab SUMMARY:Writing UI plugin with AngularJS ATTENDEE;CN=users;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailt o:us...@ovirt.org ATTENDEE;CN=engine-devel;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRU E:mailto:engine-devel@ovirt.org ATTENDEE;CN=Thomas Jelinek;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=T RUE:mailto:tjeli...@redhat.com ATTENDEE;CN=Lior Vernia;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE :mailto:lver...@redhat.com ATTENDEE;CN=Steve Gordon;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRU E:mailto:sgor...@redhat.com ATTENDEE;CN=Mooli Tayer;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE :mailto:mta...@redhat.com ATTENDEE;CN=Greg Sheremeta;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=T RUE:mailto:gsher...@redhat.com ATTENDEE;CN=Alexander Wels;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=T RUE:mailto:aw...@redhat.com ATTENDEE;CN=Ramesh Nachimuthu;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSV P=TRUE:mailto:rnach...@redhat.com ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:devel@o virt.org ORGANIZER;CN=Vojtech Szocs:mailto:vsz...@redhat.com DTSTART;TZID=Europe/Belgrade:20140408T14 DTEND;TZID=Europe/Belgrade:20140408T16 STATUS:CONFIRMED CLASS:PUBLIC X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY TRANSP:OPAQUE LAST-MODIFIED:20140403T131557Z DTSTAMP:20140403T131557Z SEQUENCE:1 DESCRIPTION:The following meeting has been modified:\n\nSubject: Writing UI plugin with AngularJS \nOrganizer: Vojtech Szocs vsz...@redhat.com \n\nT ime: Tuesday\, April 8\, 2014\, 2:00:00 PM - 4:00:00 PM GMT +01:00 Belgrade\ , Bratislava\, Budapest\, Ljubljana\, Prague [MODIFIED]\n \nInvitees: users@ ovirt.org\; engine-devel@ovirt.org\; tjeli...@redhat.com\; lver...@redhat.co m\; sgor...@redhat.com\; mta...@redhat.com\; gsher...@redhat.com\; awels@red hat.com\; rnach...@redhat.com\; de...@ovirt.org \n\n\n*~*~*~*~*~*~*~*~*~*\n\ nUPDATE: we will start one hour earlier\, at 2pm CET / 8am EST / 3pm TLV tim e\n\nHi guys\, \n\nthis session is for those of you interested in learning a bout oVirt UI plugins\, AngularJS and how they fit together. \n\nIncludes UI plugin AngularJS overview\, writing UI plugin step-by-step and live demo. \n\nTopics covered: \n* AngularJS fly-through \n* oVirt UI plugins fly-thro ugh \n* UI plugin with AngularJS walk-through \n* Live demo \n\nEstimated du ration: 1.5 hours \n\nTo join this meeting\, dial into the Intercall bridge and use following conference code: 712 886 7405 # \nhttps://www.intercallonl ine.com/listNumbersByCode.action?confCode=7128867405 \n\nPDF slides + source code will be available on day of presentation. \n\nRegards\, \nVojtech \n BEGIN:VALARM ACTION:DISPLAY TRIGGER;RELATED=START:-PT5M DESCRIPTION:Reminder END:VALARM END:VEVENT END:VCALENDAR___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] vds_dynamic refactor
- Original Message - From: Gilad Chaplik gchap...@redhat.com To: Yair Zaslavsky yzasl...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 4:00:25 PM Subject: Re: [Engine-devel] vds_dynamic refactor - Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Liran Zelkha liran.zel...@gmail.com Cc: Gilad Chaplik gchap...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 2:12:59 PM Subject: Re: [Engine-devel] vds_dynamic refactor - Original Message - From: Liran Zelkha liran.zel...@gmail.com To: Gilad Chaplik gchap...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 2:04:29 PM Subject: Re: [Engine-devel] vds_dynamic refactor Why not move it to vds_static? +1 on Liran's comment. +1 as well , vds_static is the place for that I would prefer not to add more complexity to the vds tables family. Such complexity may effect performs of queries/views. If you wish, you can create a view on top of vds_static named vds_on_boot for querying of vds on boot info. Yair That means moving almost all of vds_dynamic into vds_static except of memory, pending resources and status (maybe more but not much); and there will not be any db separation between user input and on_boot data. Why we should have such separation ? Thanks, Gilad. On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik gchap...@redhat.com wrote: Hi list, I propose to split vds_dynamic table into 2 tables: - vds_dynamic - vds_on_boot We need a place to put all non-dynamic data that gets updated once the host is booted, and I think dynamic isn't the place for it. In first phase we will put there NUMA host topoplogy, but later on migrate all non-dynamic host data (cpu, os, etc.). thoughts? Thanks, Gilad. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] vds_dynamic refactor
I would be happy if we can lose vds_dynamic all together, and just keep vds_static and vds_statistics. Our performance will be happy too ;-) On Thu, Apr 3, 2014 at 4:33 PM, Eli Mesika emes...@redhat.com wrote: - Original Message - From: Gilad Chaplik gchap...@redhat.com To: Yair Zaslavsky yzasl...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 4:00:25 PM Subject: Re: [Engine-devel] vds_dynamic refactor - Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Liran Zelkha liran.zel...@gmail.com Cc: Gilad Chaplik gchap...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 2:12:59 PM Subject: Re: [Engine-devel] vds_dynamic refactor - Original Message - From: Liran Zelkha liran.zel...@gmail.com To: Gilad Chaplik gchap...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Thursday, April 3, 2014 2:04:29 PM Subject: Re: [Engine-devel] vds_dynamic refactor Why not move it to vds_static? +1 on Liran's comment. +1 as well , vds_static is the place for that I would prefer not to add more complexity to the vds tables family. Such complexity may effect performs of queries/views. If you wish, you can create a view on top of vds_static named vds_on_boot for querying of vds on boot info. Yair That means moving almost all of vds_dynamic into vds_static except of memory, pending resources and status (maybe more but not much); and there will not be any db separation between user input and on_boot data. Why we should have such separation ? Thanks, Gilad. On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik gchap...@redhat.com wrote: Hi list, I propose to split vds_dynamic table into 2 tables: - vds_dynamic - vds_on_boot We need a place to put all non-dynamic data that gets updated once the host is booted, and I think dynamic isn't the place for it. In first phase we will put there NUMA host topoplogy, but later on migrate all non-dynamic host data (cpu, os, etc.). thoughts? Thanks, Gilad. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt
- Original Message - From: Gilad Chaplik gchap...@redhat.com To: Eli Mesika emes...@redhat.com Cc: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 3, 2014 4:04:55 AM Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt Hi all, Sorry for joining-in late. My comments (according to the db diagram section in https://docs.google.com/document/d/1-wdDkm6EDbwyoCIRPPcmbGWAcyQo_ISTY8ykDr0I6VY): 1) Join vm_numa_node and vds_numa_node to a single table (almost identical), one of the FKs can be null. 2) No templates reference in the design, need to check it out (it might be inherently designed already :-) ); vNode can be linked to a template. 3) The reason I want host's NUMA data to be in static is because it updates only once (on boot). engineerically speaking, dynamic table has a lot of traffic and that's not the case for NUMA info. Its feels to me like 'a hybrid' of static and dynamic, 3 suggestions comes to mind: - leave it in dynamic (maybe in a separate process). - have a separate flow that updates static. - come up with a third 'vds_on_boot' table (my favorite ;-P ). I will get back to you on that. i dont agree here, static is user options, there is no flow to automatically update it, and shouldn't be. this is the exact definition of dynamic for vds. there is not much updates to this table, getCaps when host moves to up and there are few fields that are updated from the statistics cycle, which is basically bad. 4) vm_vds_numa_node_map is connected to vds_numa_node_statistics, why to split the tables (vds_numa_node vds_numa_node_statistics), going back to comment #1, don't we want vNode stats as well, it can be nice to show it :-) (have a vNUMA overview of the VM using guest-agent or sth, in a future phase). 5) IMO vds_cpu_statistics shouldn't include any reference to NUMA, I gave that comment in the BE patch as well (remove vds_numa_node_id FK in vds_cpu_statistics), for that you should extract cpu_list to a connection table (anyway I don't like lists as a text/strings/etc.) 6) vm_numatune_nodeset can be removed - the vNode should hold it's pinning info; I think that vNode (node according to comment #1) should be connected to a connection table that points to vdsNode (also Node table) itself (kinda complicated but does the trick - nested link). 7) Please add delete-cascade info all over, i.e. what happens when we remove a host/vm/node. 8) Is it possible to put a db constraint that mapping between vNode and Node will be deleted once the VM isn't UP? Thanks, Gilad. - Original Message - From: Eli Mesika emes...@redhat.com To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com Cc: Gilad Chaplik gchap...@redhat.com, Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, April 1, 2014 10:10:37 AM Subject: Re: Please help us to review our database schema design with NUMA feature on ovirt - Original Message - From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com To: Gilad Chaplik gchap...@redhat.com, Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com, Eli Mesika emes...@redhat.com, Chegu Vinod chegu_vi...@hp.com Cc: Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, April 1, 2014 5:13:34 AM Subject: RE: Please help us to review our database schema design with NUMA feature on ovirt Assemble the related discussions in this mail session. Hi Vinod, On 3/31/2014 2:38 AM, Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ) wrote: We put host level NUMA fields in vds_dynamic because these information are from host itself, and NUMA topology may be changed if the host's hardware make a change. Can you please elaborate ? Are you thinking about resource (cpu and/or memory) hot plug on the host ? [Bruce] It's not about resource hot plug. In ovirt engine, there is a scheduled task which will refresh hosts' and vms' information periodically. Only the dynamic and statistics data will be updated during the refresh. So I
