[Engine-devel] [ANN] oVirt 3.4.0 GA postponed to Monday, March 24
Hi, due to 2 blocker bugs we found, the GA is pushed to Monday, March 24. This should provide sufficient time for us to properly fix and test oVirt 3.4, in order to make sure we have no critical issues in the 3.4.0 version we're all waiting for. Thanks for understanding, Doron ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine
- Original Message - From: Sandro Bonazzola sbona...@redhat.com To: Yedidyah Bar David d...@redhat.com, Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, March 17, 2014 9:11:20 AM Subject: Re: [Engine-devel] Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine Il 16/03/2014 11:59, Yedidyah Bar David ha scritto: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Yedidyah Bar David d...@redhat.com Cc: Sandro Bonazzola sbona...@redhat.com, Jiri Moskovcak jmosk...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Sunday, March 16, 2014 12:47:43 PM Subject: Re: Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine - Original Message - From: Yedidyah Bar David d...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Sandro Bonazzola sbona...@redhat.com, Jiri Moskovcak jmosk...@redhat.com Sent: Sunday, March 16, 2014 12:28:27 PM Subject: Re: Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine Might be better to discuss this on bugzilla. Bugzilla is not a mailing list. Moving to engine-devel. - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Sandro Bonazzola sbona...@redhat.com Cc: Yedidyah Bar David d...@redhat.com, Jiri Moskovcak jmosk...@redhat.com Sent: Sunday, March 16, 2014 12:01:51 PM Subject: Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine https://bugzilla.redhat.com/show_bug.cgi?id=1076530 Sandro, I think this would be solved by a better validation during setup / deployment. This can't be done during Validation in the otopi sense of the word. At that point the engine does not exist yet and so we can't know what versions it supports etc. Why not? You have the vdsm supported versions in a file (dsaversion IIRC) and you should be able to get the relevant engine info before or after deploying the DB. The VM does not exist yet at that point. How can you know what the user will install on it? You can tell them what they *should* install - e.g. The highest compatibility version supported by this host is 3.4, you should install a 3.4 engine inside the engine VM. But we can't know what the user actually did until after we connect to the installed and working engine. It might be possible (didn't check) to check the versions right before trying to add the host to the cluster. This means we do not want to abort (as we can do during Validation if something does not pass it). What can we do? Perhaps offer a few options: 1. Do abort (will do mostly what happens today) 2. Let the user try to manually fix, probably by trying to change the compatibility version of the cluster, and then try adding the host again 3. Try to fix ourselves (same) and try adding again 4. Best would be to someone upgrade libvirt and reconfigure vdsm. Not sure that's easy or even possible at this stage, where VM is running and we do not want to loose it. We can check VDSM caps in late setup / customization and abort if cluster compatibility is not 3.4. I'm not sure that VDSM 3.3 is enough for running hosted engine. We can warn the user about the minimum version of oVirt engine that must be installed inside the VM and after that we can check oVirt engine cluster compatibility and refuse to continue until the cluster have a correct support level. This will require manual changes like upgrading the engine in the VM or fix cluster compatibility level if we find an invalid value. If we can assist the user with fixing cluster compatibility level to avoid a malformed end result this is the solution I'd go with. In this way the user will always get a working setup, with the relevant information. ie- something like: Current engine settings does not support your vdsm version please select how you wish to proceed: (1) Manually upgrade vdsm (2) Lower cluster compatibility to support your installed vdsm (3) Abort Option (2) should do a simple db update to make sure the user has a running setup. Thoughts? Thinking about this again, I am not sure the current behavior is that bad. Fixing by re-installing with the correct versions is probably way simpler than fixing after installation is (mostly) complete. I'm not keen on adding hosted-engine logic into the engine code. Not sure about that. Not that it would help much, because the root problem will still have to be solved, but in principle it might be a good thing if the engine knows that killing some host will kill itself, and so try harder to not do that and just leave it in some zombie, requires-manual-action state. This is obviously more important during normal operation than during installation. -- Didi -- Sandro Bonazzola Better technology. Faster innovation
Re: [Engine-devel] Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine
- Original Message - From: Yedidyah Bar David d...@redhat.com To: Sandro Bonazzola sbona...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, March 17, 2014 10:21:39 AM Subject: Re: [Engine-devel] Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine I'm not keen on adding hosted-engine logic into the engine code. Not sure about that. Not that it would help much, because the root problem will still have to be solved, but in principle it might be a good thing if the engine knows that killing some host will kill itself, and so try harder to not do that and just leave it in some zombie, requires-manual-action state. This is obviously more important during normal operation than during installation. BTW, no-one addressed the original point in the bug. We might solve the specific compatibility level issue, but the general question still remains: Should the engine treat differently the VM and host that are running itself? Knowing that killing them means committing suicide (or, in other words, probably require manual cli action from the user)? This is what I'm trying to avoid. It is possible but will add special logic we should try to avoid. If the setup makes sure you have a running deployment you can leave it as is. the user will always be able to do bad things in the system and we should not nanny them, as long as they are aware of what they do. -- Didi ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine
- Original Message - From: Yedidyah Bar David d...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Sandro Bonazzola sbona...@redhat.com, Jiri Moskovcak jmosk...@redhat.com Sent: Sunday, March 16, 2014 12:28:27 PM Subject: Re: Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine Might be better to discuss this on bugzilla. Bugzilla is not a mailing list. Moving to engine-devel. - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Sandro Bonazzola sbona...@redhat.com Cc: Yedidyah Bar David d...@redhat.com, Jiri Moskovcak jmosk...@redhat.com Sent: Sunday, March 16, 2014 12:01:51 PM Subject: Bug 1076530 – engine shouldn't kill the vds running the VM with the hosted engine https://bugzilla.redhat.com/show_bug.cgi?id=1076530 Sandro, I think this would be solved by a better validation during setup / deployment. This can't be done during Validation in the otopi sense of the word. At that point the engine does not exist yet and so we can't know what versions it supports etc. Why not? You have the vdsm supported versions in a file (dsaversion IIRC) and you should be able to get the relevant engine info before or after deploying the DB. It might be possible (didn't check) to check the versions right before trying to add the host to the cluster. This means we do not want to abort (as we can do during Validation if something does not pass it). What can we do? Perhaps offer a few options: 1. Do abort (will do mostly what happens today) 2. Let the user try to manually fix, probably by trying to change the compatibility version of the cluster, and then try adding the host again 3. Try to fix ourselves (same) and try adding again 4. Best would be to someone upgrade libvirt and reconfigure vdsm. Not sure that's easy or even possible at this stage, where VM is running and we do not want to loose it. Thinking about this again, I am not sure the current behavior is that bad. Fixing by re-installing with the correct versions is probably way simpler than fixing after installation is (mostly) complete. I'm not keen on adding hosted-engine logic into the engine code. Not sure about that. Not that it would help much, because the root problem will still have to be solved, but in principle it might be a good thing if the engine knows that killing some host will kill itself, and so try harder to not do that and just leave it in some zombie, requires-manual-action state. This is obviously more important during normal operation than during installation. -- Didi ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Consider iowait add into usage CPU percentage
- Original Message -
From: Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com
To: vdsm-devel vdsm-de...@lists.fedorahosted.org, engine-devel@ovirt.org
Cc: Chegu Vinod chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
HPservers-Core-OE-PSC)
shangchun.li...@hp.com
Sent: Wednesday, March 12, 2014 4:09:59 AM
Subject: Re: [Engine-devel] Consider iowait add into usage CPU percentage
Hi All,
Any feedback for this topic?
Add Doron and Martin in the mail list.
Best Regards,
Jason Liao
From: vdsm-devel-boun...@lists.fedorahosted.org
[mailto:vdsm-devel-boun...@lists.fedorahosted.org] On Behalf Of Liao, Chuan
(Jason Liao, HPservers-Core-OE-PSC)
Sent: 2014 年 3 月 11 日 14:02
To: vdsm-devel; engine-devel@ovirt.org
Cc: Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Vinod, Chegu;
Gilad Chaplik
Subject: [vdsm] Consider iowait add into usage CPU percentage
Hi All,
On engine core, The usage CPU percentage is calculated by %sys + %usr
class VdsBrokerObjectsBuilder
function updateVDSStatisticsData
vds.setCpuSys(AssignDoubleValue(xmlRpcStruct, VdsProperties.cpu_sys));
vds.setCpuUser(AssignDoubleValue(xmlRpcStruct, VdsProperties.cpu_user));
if (vds.getCpuSys() != null vds.getCpuUser() != null) {
vds.setUsageCpuPercent((int) (vds.getCpuSys() + vds.getCpuUser()));
}
On vdsm, The %sys, %usr and %idle is calculated like below workflow,
class API
function getStats
decStats = self._cif._hostStats.get()
class clientIF
function __init__
self._hostStats = sampling.HostStatsThread(log=log)
self._hostStats.start()
class HostStatsThread
function get
hs0, hs1 = self._samples[0], self._samples[-1]
…
jiffies = (hs1.totcpu.user - hs0.totcpu.user) % (2 ** 32)
stats['cpuUser'] = jiffies / interval / self._ncpus
jiffies = (hs1.totcpu.sys - hs0.totcpu.sys) % (2 ** 32)
stats['cpuSys'] = jiffies / interval / self._ncpus
stats['cpuIdle'] = max(0.0, 100.0 - stats['cpuUser'] - stats['cpuSys'])
class HostSample
function __init__
self.totcpu = TotalCpuSample()
class TotalCpuSample
function __init__
self.user, userNice, self.sys, self.idle = \
map(int, file('/proc/stat').readline().split()[1:5])
self.user += userNice
Question1: Why stats[‘cpuIdle’] do not use the sampling data totcpu.idle for
calculating?
I have a vague memory of this calculation which tries to level the numbers.
ie- the assumption is that user and sys adds up to something which is less than
100.
If this is not the case (multiple core / SMT) idle will be set to 0.
But possibly Dan remembers something else?
Question2: There is another data named iowait in /proc/stat, do we need to
consider add this into usage CPU percentage for calculating?
Since the above excludes iowait, I wouldn't add it here. Otherwise
we may have inconsistency with the numbers.
Best Regards,
Jason Liao
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] which data structure is better
- Original Message - From: Eli Mesika emes...@redhat.com To: Liran Zelkha liran.zel...@gmail.com Cc: engine-devel@ovirt.org, Chegu Vinod chegu_vi...@hp.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com Sent: Thursday, February 20, 2014 2:42:36 PM Subject: Re: [Engine-devel] which data structure is better - Original Message - From: Liran Zelkha liran.zel...@gmail.com To: Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com Cc: engine-devel@ovirt.org, Chegu Vinod chegu_vi...@hp.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com Sent: Thursday, February 20, 2014 11:40:57 AM Subject: Re: [Engine-devel] which data structure is better Hi Please don't add rapidly changing data to VDSDynamic - it has major performance implications. So, choose option B. Actually, try to expose relevant data in VDSDynamic and VDSStatistics, and VDS should call VDSDynamic and VDSStatistics and merge the data from both entities. Agree.We had lately several bottle-necks around the VDS/VM updates and Liran had improved the relevant queries and added batch-update stuff, so we want to keep the performance gain Jason, in order for us to know which node can accommodate which VM we need to get the memory statistics of each NUMA node as well (usage and free). Also, if possible CPU usage in every node. On Thu, Feb 20, 2014 at 11:31 AM, Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com wrote: Hi All, I am Jason Liao from HP who are in charge of NUMA and Virtual NUMA feature. Now I have some concept about the host NUMA topology data structure on engine core We have VDS, VDSDynamic, VDSStatic, VdsStatistics object on engine core. And we have NUMA topology information: ListNumaNode numaNodeList NumaNode String ID # update from GetCapabilitiesVDSCommand ListString cpuList # update from GetCapabilitiesVDSCommand Int totalMem # update from GetCapabilitiesVDSCommand Int freeMem # update from GetStatsVDSCommand A. Add this data structure into VDSDynamic We should change the GetStatsVDSCommand update the VDSDynamic data. B. Add this data structure into VDS, and build the data structure from VDSDynamic, VdsStatistics I prefer B. does anybody have some comments? Best Regards, Jason Liao ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Spice proxy test day support
Hi all, I was testing this feature today and would like to summarize my findings; 1. Feature page http://www.ovirt.org/Features/Spice_Proxy Quite informative. Nevertheless things we should improve: - Please add a contact person in case of question / issues (feature owner?). - Detail design? Nothing mentioned with regards to the implementation. This could be in a separate page, but in such a case I'd add a link to that page. - Missing specific examples. ie- supported protocols: engine-config -s SpiceProxyDefault=http://my-ip:8080 * Did you know that http is a default if no scheme stated? All the above should make this cool feature more easy to use by non-developers. 2. Test cases - System level (using engine-config) - Cluster level - VM-Pool level For each of the above I configured a proxy, and on the proxy machine used: nc -l 8080 to capture the traffic. In all cases things worked quite nicely. I saw incoming connection request which is what I expected for. 3. All-in-all: We have the functionality working which is the important part. Going forward we should try to improve some of the things I mentioned above. Nice work, Doron P.S. During my tests I managed to open 2 non-related bugs on engine-config and network QoS mapper (Class cast issue). ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] oVirt test day tomorrow
Just a reminder that tomorrow we plan to have oVirt 3.4 2nd test day. More details coming soon. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] FOSDEM slides + recordings available
http://www.ovirt.org/FOSDEM_2014#Sessions I will continue to update with recordings (and last slides) as it becomes available. This is also a good chance to thank everyone involved. People who practiced, spoke, volunteered for stand shifts, Video handling, helping with the oVirt-live, carrying maps, laptops, handout-sheets, poster-sign to/from UBL and other logistics. Special thanks to Joop, René, Jorick, Robert and Alexander who joined us in ULB . We got a big and very positive exposure for oVirt, which as you'll be able to hear is getting a lot of community attention now. See you in FOSDEM 2015 ;) Doron ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [Users] oVirt 3.4.0 release schedule updated
- Original Message - From: Eli Mesika emes...@redhat.com To: Sandro Bonazzola sbona...@redhat.com Cc: arch a...@ovirt.org, engine-devel engine-devel@ovirt.org, vdsm-devel vdsm-de...@fedorahosted.org, VDSM Project Development vdsm-de...@lists.fedorahosted.org, us...@ovirt.org Sent: Thursday, January 9, 2014 10:40:28 AM Subject: Re: [Users] oVirt 3.4.0 release schedule updated - Original Message - From: Sandro Bonazzola sbona...@redhat.com To: engine-devel engine-devel@ovirt.org, VDSM Project Development vdsm-de...@lists.fedorahosted.org, vdsm-devel vdsm-de...@fedorahosted.org, arch a...@ovirt.org, us...@ovirt.org Sent: Wednesday, January 8, 2014 6:47:58 PM Subject: [Users] oVirt 3.4.0 release schedule updated oVirt team has updated the release schedule for 3.4.0 [1] These are tentative planning dates and may change General availability: 2014-02-24 oVirt 3.4 Second Test Day: 2014-02-19 RC Build: 2014-02-17 oVirt 3.4 Test Day: 2014-01-27 Please note that some guys from the TLV office could not attend to that due to an Advanced Python Course taking place (27-30 JAN) Hi Sandro, Since there's some activity also around FOSDEM, I suggest we will do the first test day on January 23. Please let me know if there are any objections. Beta release: 2014-01-20 Branching / Feature freeze: 2014-01-15 Alpha release: 2014-01-09 more details on test days, etc to come in the next few weeks [1] http://www.ovirt.org/OVirt_3.4_release-management#Timeline -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Users mailing list us...@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list us...@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Broken devel setup on Gentoo?
On 12/31/2013 08:10 PM, Alon Bar-Lev wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, December 31, 2013 7:17:40 PM Subject: Re: [Engine-devel] Broken devel setup on Gentoo? An unrelated issue I noticed when moving from 3.3 branch to master I re-built everything, yet I was unable to login. I had to update the DB with a new password to login. I suspect theres another issue here. I am unsure I follow, you cannot use the same prefix for both branches. What the exact sequence? 1. Built everything on ovirt-3.3 branch. = works well. 2. git checkout origin/master git reset --hard origin/master rm -rf share bin etc usr var 3. Cherry pick patch from gerrit 4. make clean install-dev 5. run setup = I'd expect db to be recreated. If not then we cannot login to the system. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Broken devel setup on Gentoo?
On 01/01/2014 10:57 AM, Alon Bar-Lev wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, January 1, 2014 10:31:07 AM Subject: Re: [Engine-devel] Broken devel setup on Gentoo? On 12/31/2013 08:10 PM, Alon Bar-Lev wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, December 31, 2013 7:17:40 PM Subject: Re: [Engine-devel] Broken devel setup on Gentoo? An unrelated issue I noticed when moving from 3.3 branch to master I re-built everything, yet I was unable to login. I had to update the DB with a new password to login. I suspect theres another issue here. I am unsure I follow, you cannot use the same prefix for both branches. What the exact sequence? 1. Built everything on ovirt-3.3 branch. = works well. 2. git checkout origin/master git reset --hard origin/master rm -rf share bin etc usr var 3. Cherry pick patch from gerrit 4. make clean install-dev 5. run setup = I'd expect db to be recreated. If not then we cannot login to the system. Are you removing the pki keys? the entire configuration? why not remove the entire PREFIX? Database will not re-created if already exist, there will be an attempt to upgrade it, and as you removed the pki keys the password is cannot be decrypted. If you remove PREFIX you should also drop/create database to start scratch, or just use another empty database when installing. Regards, Alon I'm not using PREFIX. What are the alternatives? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Broken devel setup on Gentoo?
On 01/01/2014 11:23 AM, Alon Bar-Lev wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, January 1, 2014 11:18:45 AM Subject: Re: [Engine-devel] Broken devel setup on Gentoo? On 01/01/2014 10:57 AM, Alon Bar-Lev wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, January 1, 2014 10:31:07 AM Subject: Re: [Engine-devel] Broken devel setup on Gentoo? On 12/31/2013 08:10 PM, Alon Bar-Lev wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, December 31, 2013 7:17:40 PM Subject: Re: [Engine-devel] Broken devel setup on Gentoo? An unrelated issue I noticed when moving from 3.3 branch to master I re-built everything, yet I was unable to login. I had to update the DB with a new password to login. I suspect theres another issue here. I am unsure I follow, you cannot use the same prefix for both branches. What the exact sequence? 1. Built everything on ovirt-3.3 branch. = works well. 2. git checkout origin/master git reset --hard origin/master rm -rf share bin etc usr var 3. Cherry pick patch from gerrit 4. make clean install-dev 5. run setup = I'd expect db to be recreated. If not then we cannot login to the system. Are you removing the pki keys? the entire configuration? why not remove the entire PREFIX? Database will not re-created if already exist, there will be an attempt to upgrade it, and as you removed the pki keys the password is cannot be decrypted. If you remove PREFIX you should also drop/create database to start scratch, or just use another empty database when installing. Regards, Alon I'm not using PREFIX. What are the alternatives? how come[1]? $ make clean install-dev PREFIX=$HOME/ovirt-engine so you are running it as root and install into /usr/local? the entire concept of devenv is to use it as non root and install to user accessible directory (aka PREFIX). if you want to start fresh (by removing the PREFIX) then you should also have fresh (empty) database. Regards, Alon Bar-Lev [1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;hb=HEAD#l70 I'm not using root. There's a special user account which has ~/ovirt-engine including the git repo inside. So the prefix is the same as the git repo and removing it removes git... ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Broken devel setup on Gentoo?
While refreshing my devel environment, setup fails.
Log file shows this:
2013-12-31 16:26:05 DEBUG otopi.plugins.otopi.services.openrc
plugin.executeRaw:366 execute: ('/etc/init.d/ovirt-websocket-proxy',
'-q', 'status'), executable='None', cwd=
'None', env=None
2013-12-31 16:26:05 DEBUG otopi.plugins.otopi.services.openrc
plugin.executeRaw:389 execute-result:
('/etc/init.d/ovirt-websocket-proxy', '-q', 'status'), exception
Traceback (most recent call last):
File /usr/lib64/python2.7/site-packages/otopi/plugin.py, line 376,
in executeRaw
env=env,
File /usr/lib64/python2.7/subprocess.py, line 711, in __init__
errread, errwrite)
File /usr/lib64/python2.7/subprocess.py, line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Should there be a redirection to use ovirt-websocket-proxy in a local
script rather than daemon?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Broken devel setup on Gentoo?
On 12/31/2013 04:54 PM, Alon Bar-Lev wrote:
- Original Message -
From: Doron Fediuck dfedi...@redhat.com
To: engine-devel@ovirt.org
Cc: Alon Bar-Lev abar...@redhat.com
Sent: Tuesday, December 31, 2013 4:50:48 PM
Subject: [Engine-devel] Broken devel setup on Gentoo?
While refreshing my devel environment, setup fails.
Log file shows this:
2013-12-31 16:26:05 DEBUG otopi.plugins.otopi.services.openrc
plugin.executeRaw:366 execute: ('/etc/init.d/ovirt-websocket-proxy',
'-q', 'status'), executable='None', cwd=
'None', env=None
2013-12-31 16:26:05 DEBUG otopi.plugins.otopi.services.openrc
plugin.executeRaw:389 execute-result:
('/etc/init.d/ovirt-websocket-proxy', '-q', 'status'), exception
Traceback (most recent call last):
File /usr/lib64/python2.7/site-packages/otopi/plugin.py, line 376,
in executeRaw
env=env,
File /usr/lib64/python2.7/subprocess.py, line 711, in __init__
errread, errwrite)
File /usr/lib64/python2.7/subprocess.py, line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Should there be a redirection to use ovirt-websocket-proxy in a local
script rather than daemon?
http://gerrit.ovirt.org/#/q/I40f061ec6cf658e1d8ec5c3663698ceca7569b38,n,z
Thanks.
I'm running on this branch (ovirt-3.3) and still hitting this issue.
I checked
packaging/setup/plugins/ovirt-engine-setup/config/websocket_proxy.py
and I can see the fix (now in line 258), but the issue is still there.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Broken devel setup on Gentoo?
On 12/31/2013 06:45 PM, Alon Bar-Lev wrote:
- Original Message -
From: Alon Bar-Lev alo...@redhat.com
To: Doron Fediuck dfedi...@redhat.com
Cc: engine-devel@ovirt.org
Sent: Tuesday, December 31, 2013 5:28:24 PM
Subject: Re: [Engine-devel] Broken devel setup on Gentoo?
- Original Message -
From: Doron Fediuck dfedi...@redhat.com
To: Alon Bar-Lev alo...@redhat.com
Cc: engine-devel@ovirt.org
Sent: Tuesday, December 31, 2013 5:20:24 PM
Subject: Re: [Engine-devel] Broken devel setup on Gentoo?
On 12/31/2013 04:54 PM, Alon Bar-Lev wrote:
- Original Message -
From: Doron Fediuck dfedi...@redhat.com
To: engine-devel@ovirt.org
Cc: Alon Bar-Lev abar...@redhat.com
Sent: Tuesday, December 31, 2013 4:50:48 PM
Subject: [Engine-devel] Broken devel setup on Gentoo?
While refreshing my devel environment, setup fails.
Log file shows this:
2013-12-31 16:26:05 DEBUG otopi.plugins.otopi.services.openrc
plugin.executeRaw:366 execute: ('/etc/init.d/ovirt-websocket-proxy',
'-q', 'status'), executable='None', cwd=
'None', env=None
2013-12-31 16:26:05 DEBUG otopi.plugins.otopi.services.openrc
plugin.executeRaw:389 execute-result:
('/etc/init.d/ovirt-websocket-proxy', '-q', 'status'), exception
Traceback (most recent call last):
File /usr/lib64/python2.7/site-packages/otopi/plugin.py, line 376,
in executeRaw
env=env,
File /usr/lib64/python2.7/subprocess.py, line 711, in __init__
errread, errwrite)
File /usr/lib64/python2.7/subprocess.py, line 1308, in
_execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Should there be a redirection to use ovirt-websocket-proxy in a local
script rather than daemon?
http://gerrit.ovirt.org/#/q/I40f061ec6cf658e1d8ec5c3663698ceca7569b38,n,z
Thanks.
I'm running on this branch (ovirt-3.3) and still hitting this issue.
I checked
packaging/setup/plugins/ovirt-engine-setup/config/websocket_proxy.py
and I can see the fix (now in line 258), but the issue is still there.
Please send setup log file.
Correct, the issue is that setup is trying to acquire service status before
stopping.
Fix is available[1]
[1] http://gerrit.ovirt.org/#/q/I0454e3f136ad1181bc01c55411b33b1e2fc20e4c,n,z
Thanks for the quick fix, verified both for 3.3 and in master.
An unrelated issue I noticed when moving from 3.3 branch to master I
re-built everything,
yet I was unable to login. I had to update the DB with a new password to
login.
I suspect theres another issue here.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Help to review the design of Feature/NUMA and Virtual NUMA
Adding arch list as this may effect other sub-projects as well. - Original Message - From: Chuan Liao (Jason, MCXS-CQ) chuan.l...@hp.com To: engine-devel@ovirt.org Sent: Friday, December 13, 2013 9:37:59 AM Subject: [Engine-devel] Help to review the design of Feature/NUMA and Virtual NUMA Hi Everyone, I am Jason Liao from HP, now focus on the NUMA feature integration into oVirt. Now we finish the first step of High-level design document. Please help to review the design of Features/NUMA_and_Virtual_NUMA on oVirt community wiki page. If anyone have some question and suggestion, please let me know. Thanks all. Best Regards, Jason Liao ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] oVirt 3.3 feature freeze and delivery to testing repo
- Original Message - | From: Alon Bar-Lev alo...@redhat.com | To: Itamar Heim ih...@redhat.com | Cc: Moran Goldboim mgold...@redhat.com, Doron Fediuck dfedi...@redhat.com, Greg Padgett | gpadg...@redhat.com, Omer Frenkel ofren...@redhat.com, Michal Skrivanek mskri...@redhat.com, Mike | Kolesnik mkole...@redhat.com, Livnat Peer lp...@redhat.com, Ayal Baron aba...@redhat.com, Federico | Simoncelli fsimo...@redhat.com, engine-devel engine-devel@ovirt.org | Sent: Tuesday, July 16, 2013 4:04:46 PM | Subject: Re: [Engine-devel] oVirt 3.3 feature freeze and delivery to testing repo | | | | - Original Message - | From: Itamar Heim ih...@redhat.com | To: Moran Goldboim mgold...@redhat.com, Doron Fediuck | dfedi...@redhat.com, Greg Padgett | gpadg...@redhat.com, Omer Frenkel ofren...@redhat.com, Michal | Skrivanek mskri...@redhat.com, Mike | Kolesnik mkole...@redhat.com, Livnat Peer lp...@redhat.com, Ayal | Baron aba...@redhat.com, Federico | Simoncelli fsimo...@redhat.com | Cc: engine-devel engine-devel@ovirt.org | Sent: Tuesday, July 16, 2013 4:03:46 PM | Subject: Re: [Engine-devel] oVirt 3.3 feature freeze and delivery to | testing repo | | On 07/15/2013 05:45 PM, Moran Goldboim wrote: | Maintainers and all, | | due to some misunderstanding regarding feature freeze for ovirt 3.3 | release (suppose to be today, but marked as Jul 17th on release page), | we are extending feature freeze till wendesday. | | feature owners: | -please update 3.3 release page [1] with the feature status | -please update testing page for your feature | | package owners- once your component is ready (should be done till | wendesday) | -tag your repo for 3.3 beta release | -provide srpm to mike | -provide fc19/el6 rpms builds using mock to mike | | mike if you would like to set a place people can upload it too please | let us know. | | | [1]http://www.ovirt.org/OVirt_3.3_release-management#Features_Status_Table | | | what's the status of these which seem a shame to release without (and | all are supposed to be on their final step)? | - glance integration (federico) | - Neutron (quantum) integration (livnat/mike)? | | I am working on releasing beta of the prerequisites. | | - cloud-init (greg/omer)? | - plugable scheduling (doron)? Working now on merging the new internal scheduler. The proxy (pluggable part) will take a few more days. So if there's a re-spin the proxy will be in. | | Thanks, | Itamar | ___ | Engine-devel mailing list | Engine-devel@ovirt.org | http://lists.ovirt.org/mailman/listinfo/engine-devel | | ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Add traffic shaping parameters for a network interface.
- Original Message -
From: Giuseppe Vallarelli gvall...@redhat.com
To: Livnat Peer lp...@redhat.com
Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
Kenigsberg dan...@redhat.com
Sent: Tuesday, June 11, 2013 5:34:16 PM
Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
interface.
- Original Message -
| From: Giuseppe Vallarelli gvall...@redhat.com
| To: Livnat Peer lp...@redhat.com
| Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
| Kenigsberg dan...@redhat.com
| Sent: Tuesday, June 11, 2013 3:07:32 PM
| Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
| interface.
|
| Related to QoS parameters reporting to the engine. Looks like they're
| already
| available, I tried to use vdsClient with list verb and I've got the devices
| list where a 'specParams' is defined (it's empty because I didn't try it
| with
| my last patch).
|
| devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
| 'network':
| 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
| 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address': {'slot':
| '0x03', 'bus': '0x00', 'domain': '0x',
| 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}
| ...]
|
| Perhaps I'm missing something, any ideas/hints ?
| Thanks Giuseppe
A few pastes:
creating the vm: http://paste.fedoraproject.org/17953/37096041/
dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
I've tried this out thanks to Toni, losing sanity with vdsClient..
Giuseppe
Thanks Giuseppe.
Is this also reported by vdsm in getVmStats?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Add traffic shaping parameters for a network interface.
- Original Message -
From: Giuseppe Vallarelli gvall...@redhat.com
To: Doron Fediuck dfedi...@redhat.com
Cc: engine-devel@ovirt.org
Sent: Wednesday, June 12, 2013 1:47:03 PM
Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
interface.
- Original Message -
| From: Giuseppe Vallarelli gvall...@redhat.com
| To: Doron Fediuck dfedi...@redhat.com
| Cc: engine-devel@ovirt.org
| Sent: Wednesday, June 12, 2013 9:28:19 AM
| Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
| interface.
|
|
|
| - Original Message -
| | From: Doron Fediuck dfedi...@redhat.com
| | To: Giuseppe Vallarelli gvall...@redhat.com
| | Cc: Livnat Peer lp...@redhat.com, engine-devel@ovirt.org, Dan
| | Kenigsberg dan...@redhat.com
| | Sent: Wednesday, June 12, 2013 9:05:35 AM
| | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
| | interface.
| |
| |
| |
| | - Original Message -
| | From: Giuseppe Vallarelli gvall...@redhat.com
| | To: Livnat Peer lp...@redhat.com
| | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
| | Kenigsberg dan...@redhat.com
| | Sent: Tuesday, June 11, 2013 5:34:16 PM
| | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
| | network
| | interface.
| |
| | - Original Message -
| | | From: Giuseppe Vallarelli gvall...@redhat.com
| | | To: Livnat Peer lp...@redhat.com
| | | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org,
| | | Dan
| | | Kenigsberg dan...@redhat.com
| | | Sent: Tuesday, June 11, 2013 3:07:32 PM
| | | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
| | | network
| | | interface.
| | |
| | | Related to QoS parameters reporting to the engine. Looks like they're
| | | already
| | | available, I tried to use vdsClient with list verb and I've got the
| | | devices
| | | list where a 'specParams' is defined (it's empty because I didn't try
| | | it
| | | with
| | | my last patch).
| | |
| | | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
| | | 'network':
| | | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
| | | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address':
| | | {'slot':
| | | '0x03', 'bus': '0x00', 'domain': '0x',
| | | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
| | | 'interface'}
| | | ...]
| | |
| | | Perhaps I'm missing something, any ideas/hints ?
| | | Thanks Giuseppe
| |
| | A few pastes:
| | creating the vm: http://paste.fedoraproject.org/17953/37096041/
| | dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
| |
| | I've tried this out thanks to Toni, losing sanity with vdsClient..
| |
| | Giuseppe
| |
| |
| | Thanks Giuseppe.
| | Is this also reported by vdsm in getVmStats?
| |
|
| Unfortunately is not reported using getVmStats, I'm looking into it.
| Giuseppe
New paste: http://paste.fedoraproject.org/18161/71033933/
Giuseppe
Brilliant, thanks!
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Add traffic shaping parameters for a network interface.
- Original Message -
From: Dan Kenigsberg dan...@redhat.com
To: Doron Fediuck dfedi...@redhat.com
Cc: Giuseppe Vallarelli gvall...@redhat.com, engine-devel@ovirt.org
Sent: Wednesday, June 12, 2013 3:59:43 PM
Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
interface.
On Wed, Jun 12, 2013 at 06:54:52AM -0400, Doron Fediuck wrote:
- Original Message -
From: Giuseppe Vallarelli gvall...@redhat.com
To: Doron Fediuck dfedi...@redhat.com
Cc: engine-devel@ovirt.org
Sent: Wednesday, June 12, 2013 1:47:03 PM
Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
interface.
- Original Message -
| From: Giuseppe Vallarelli gvall...@redhat.com
| To: Doron Fediuck dfedi...@redhat.com
| Cc: engine-devel@ovirt.org
| Sent: Wednesday, June 12, 2013 9:28:19 AM
| Subject: Re: [Engine-devel] Add traffic shaping parameters for a
| network
| interface.
|
|
|
| - Original Message -
| | From: Doron Fediuck dfedi...@redhat.com
| | To: Giuseppe Vallarelli gvall...@redhat.com
| | Cc: Livnat Peer lp...@redhat.com, engine-devel@ovirt.org, Dan
| | Kenigsberg dan...@redhat.com
| | Sent: Wednesday, June 12, 2013 9:05:35 AM
| | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
| | network
| | interface.
| |
| |
| |
| | - Original Message -
| | From: Giuseppe Vallarelli gvall...@redhat.com
| | To: Livnat Peer lp...@redhat.com
| | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org,
| | Dan
| | Kenigsberg dan...@redhat.com
| | Sent: Tuesday, June 11, 2013 5:34:16 PM
| | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
| | network
| | interface.
| |
| | - Original Message -
| | | From: Giuseppe Vallarelli gvall...@redhat.com
| | | To: Livnat Peer lp...@redhat.com
| | | Cc: Doron Fediuck dfedi...@redhat.com,
| | | engine-devel@ovirt.org,
| | | Dan
| | | Kenigsberg dan...@redhat.com
| | | Sent: Tuesday, June 11, 2013 3:07:32 PM
| | | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
| | | network
| | | interface.
| | |
| | | Related to QoS parameters reporting to the engine. Looks like
| | | they're
| | | already
| | | available, I tried to use vdsClient with list verb and I've got
| | | the
| | | devices
| | | list where a 'specParams' is defined (it's empty because I didn't
| | | try
| | | it
| | | with
| | | my last patch).
| | |
| | | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
| | | 'network':
| | | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
| | | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address':
| | | {'slot':
| | | '0x03', 'bus': '0x00', 'domain': '0x',
| | | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
| | | 'interface'}
| | | ...]
| | |
| | | Perhaps I'm missing something, any ideas/hints ?
| | | Thanks Giuseppe
| |
| | A few pastes:
| | creating the vm: http://paste.fedoraproject.org/17953/37096041/
| | dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
| |
| | I've tried this out thanks to Toni, losing sanity with vdsClient..
| |
| | Giuseppe
| |
| |
| | Thanks Giuseppe.
| | Is this also reported by vdsm in getVmStats?
| |
|
| Unfortunately is not reported using getVmStats, I'm looking into it.
| Giuseppe
New paste: http://paste.fedoraproject.org/18161/71033933/
(using hyperlinks instead of inlining the suggested API is unfriendly to
reviewers and to list archives)
network = {'vnet0': {'macAddr': 'D0:67:E5:F0:75:B4', 'outbound': {'average':
'1024'}, 'rxDropped': '0', 'txDropped': '0', 'inbound': {'average': '1024'},
'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state':
'unknown', 'speed': '1000', 'name': 'vnet0'}}
Giuseppe
Brilliant, thanks!
Why does it fit into getVmStats? getVmStats should report
dynamically-changing properties. Bandwidth limitations are more static
control entities, why should we report them every 10 seconds or so?
Dan
I have no problems with having it in something which runs less frequently.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Guest Reboot
- Original Message - From: Martin Betak mbe...@redhat.com To: engine-devel@ovirt.org Sent: Friday, June 7, 2013 7:32:31 PM Subject: [Engine-devel] Guest Reboot Hi, engine-devel Customers request the ability to reboot a VM with a single click so I started designing the overall architecture and planing out the required changes to the respective components. You can find the wiki page for the initial draft at [1] and I would like to ask you for your input on my general design and potential issues that could arise in some corner cases. Please feel free to respond to this thread or add to the Issues section of [1]. Thanks in advance for your opinions :-) Martin [1] http://www.ovirt.org/Features/Guest_Reboot Hi Martin, Starting version 3.3 we'll enable a fully operational watchdog device. This device is capable of auto-restarting a guest in case something goes wrong in the guest (BSOD / kernel panic). This may handle some of your use cases but not all of them. My only request from you is to verify the reboot expected behavior in VMs who are actively using a watchdog device set to restart the guest. Otherwise users will see conflicts and/or double reboots. The VM configuration may or may not include a guest agent and a watchdog device. A watchdog device may reset (reboot) the guest, but may also pause it for debugging purposes or simply do nothing. So a table that lists VM configuration and expected behavior should probably clarify everything and prevent implementation issues. Doron ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Add traffic shaping parameters for a network interface.
- Original Message -
From: Giuseppe Vallarelli gvall...@redhat.com
To: Dan Kenigsberg dan...@redhat.com
Cc: engine-devel@ovirt.org
Sent: Monday, June 10, 2013 6:04:27 PM
Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
interface.
- Original Message -
| From: Dan Kenigsberg dan...@redhat.com
| To: Giuseppe Vallarelli gvall...@redhat.com
| Cc: engine-devel@ovirt.org
| Sent: Monday, June 10, 2013 4:22:54 PM
| Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
| interface.
|
| On Mon, Jun 10, 2013 at 08:56:21AM -0400, Giuseppe Vallarelli wrote:
| Hi Guys, I've recently submitted a patch to support traffic shaping for a
| network interface (http://gerrit.ovirt.org/#/c/15445/).
| This work is needed in order to support
| http://www.ovirt.org/Features/Network_QoS .
| Given:
|
| 'specParams': {'inbound': {'average': '1000', 'peak': '5000', 'burst':
| '1024'},
| 'outbound': {'average': '128', 'burst': '256'}}}
|
| Generated xml is the following one:
|
|
|
|
|
|
| As you can see I tried to keep the data structure as flat as possible
| having the bandwidth element not carrying any useful information.
| Feedback is highly appreciated.
|
|
| The issue has not been mentioned on the wiki page, but may need a means
| to report the currently-configured QoS of each vNIC from Vdsm to Engine.
| For example, when a VM is de-hibernated, we may want to tell whether its
| QoS needs to be set according to a recently-tweaked policy.
|
| I suggest that we use the getVmList verb of Vdsm, which is intended to
| report static properties of one Vm (or all of them).
|
| On the other hand, Engine would want to blindly set new values whenever
| in doubt. In such a case, I think that reporting of QoS can be avoided.
|
| Dan.
|
I'm not sure I've understood completely the issue in discussion, doesn't the
engine knows already
which are the QoS profile applied to each vNIC ? The last 'tweaked' profile
is the one that should
be applied after de-hibernation. This means that on the engine side we should
keep track of profile
change, if a change happens de-hibernating a vm triggers a QoS profile update
on the host of the
latest profile. I'm not aware of the implementation details so I might be
wrong.
Giuseppe.
The idea is to handle scenarios where something went wrong;
For example, VDSM crash while starting a new VM, engine crash, etc.
So the engine should be able to ask for current QoS for reporting,
and (re-)apply it if out of sync.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Network Quality of Service 3.3 - Feature design
- Original Message - From: Livnat Peer lp...@redhat.com To: Ofri Masad oma...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, June 4, 2013 12:02:35 PM Subject: Re: [Engine-devel] Network Quality of Service 3.3 - Feature design On 06/02/2013 09:58 AM, Ofri Masad wrote: Hi all, A new Feature page for Network Quality of Service feature was published. http://www.ovirt.org/Features/Design/Network_QoS You are more than welcome to share you thoughts and insights. Hi Ofri, Here is another suggestion for you to consider, this suggestion is realted only to QoS on the VNIC level - Introducing a new entity - VNIC Profile. The VNIC profile would include all the properties of a VNIC: - network, - Qos, - Port mirroring, - custom properties From now on a user would choose a VNIC profile when he defines a VNIC (instead of choosing a network and defining properties directly on the VNIC) A network could have multiple profiles defined on it. A User would need permissions to use a profile instead of the current state that we require permission to use a network. The benefits of this approach : 1. Limiting the user to a specific QoS on a network is easy you give the user permission to use a specific profile. 2. A user can add a new VNIC but he would be limited to QoS defined on the profile he is able to use (which eliminates the problem that a user can add a VNIC to it's VM but won't get any bandwidth limitations). 3. An administrator does not add VNIC QoS properties on the network entity (to serve as defaults) which are not relevant for non-VM network. 4. The network admin who creates the VNIC profile is also the one who can configure the QoS to that Network. 5. The separation between user portal and admin portal is very clear, in user portal we expose only the profile name and in the admin portal a user can view the profile details. 6. We would leave custom properties also on the VNIC level not only the profile level so a user can send VM specific data. 7.I can also describe upgrade path but maybe we should discuss the general concept before diving into the details. Hi Livnat, This design creates a new feature of network profile, which has QoS included in it, so it's bigger the the original intention. Having that said, I agree with the concept as we need to take a more holistic approach which considers other areas of the system, such as SLA policies and instance types. So in this view I'll just add that going forward the QoS element of the profile will be a reference to a policy entity which will include network QoS as well as other QoS elements. At this point we're going back to the drawing board to update the design and will publish an update asap. Doron Livnat Thanks, Ofri. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] oVirt Scheduler API Design
- Original Message - From: Gilad Chaplik gchap...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, May 28, 2013 5:34:11 PM Subject: [Engine-devel] oVirt Scheduler API Design Hi all, Attached links for oVirt Scheduler API design wiki page. High level overview: http://www.ovirt.org/Features/oVirtScheduler Detailed design: http://www.ovirt.org/Features/oVirtSchedulerAPI You are more than welcome to share you thoughts. Thanks, Gilad. Adding to users' list for all those asking about the new oVirt scheduling features. Feel free to review and feedback. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Hi Dave, Just to make sure I fully understand, I'll repeat your basic arguments; 1. It takes time to query a big number of hosts (hundreds). 2. When backend is booting, a user may start a VM on a host which was hacked during the downtime of the engine. If the above is your concern, it shouldn't be so. The reason is, that no host will become operational before you get a response from the attestation server and allow it to become operational. So a user cannot start a new VM on a non-operational host. What this means is that your thread may need to update the user by sending a periodic event that a large scale attestation operation is in progress. Other than that, maybe your thread can work in smaller groups if it gets better results? ie- instead of one query for 300 hosts, maybe you can run 3 serialized queries for 100 hosts each? If this does not help, maybe you can run a short query for something like 10 hosts, which should get an answer relatively fast. The you can issue a query for the other 290 hosts which will take longer. In this way the system may get 10 hosts to work with quite fast, and later on the other 290 hosts will join... So this can actually be configurable to a 2-phase process; a short query and a longer one. The admin can choose the short query size based on his setup, and the longer query can pick up all the other hosts. What do you think? Doron - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Doron Fediuck dfedi...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Saturday, April 27, 2013 9:36:44 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated Hi, Our current consideration is add a new thread in engine's side to attest all of hosts (aggregated query from attestation sever) one time in case of engine's rebooting. There is still one potential issue under extreme condition, saying, hundreds of nodes in a datacenter, attest all of hosts still may take couple of mins, let's say, one hacked untrusted node before receiving the latest status may considered as a trusted host, so, the worst case in a datacenter with hundreds of nodes is, 1. engine is down for some reasons and boot up again, some trusted nodes may be hacked and rebooted during this period. 2. our thread is running to get all of node's status (trust /untrusted), may take couple of mins in large datacenter. 2. user create VMs on these hacked nodes and believe these VMs are trusted VMs launched on trusted nodes. 3. our thread get the correct status of these untrusted nodes, set these nodes as non-operational. 4. all of these trusted VMs running on these untrusted nodes are expected to migrate to other trusted node. So, the question is in a trusted cluster with hundreds of nodes some VMs expected to create on trusted nodes may actually create on untrusted nodes instead, and this time may last for couple of mins. (worst case in my view is 10 mins with 1000 nodes). Does this acceptable from your point of view? Or any other suggestion? Best Regards, Dave Chen -Original Message- From: Doron Fediuck [mailto:dfedi...@redhat.com] Sent: Sunday, April 21, 2013 11:58 PM To: Chen, Wei D Cc: Ofri Masad; Oved Ourfalli; engine-devel@ovirt.org Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Ofri Masad oma...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated Ofri, Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of quartz job? Is there any class similar with InitVdsOnUpCommand for engine's initialization? Best Regards, Dave Chen org.ovirt.engine.core.bll.Backend.Initialize() Note you cannot block this method while waiting for results. Instead I suggest you fire a one-time background request from this method. -Original Message- From: Ofri Masad [mailto:oma...@redhat.com] Sent: Sunday, April 21, 2013 3:29 PM To: Chen, Wei D Cc: Oved Ourfalli; engine-devel@ovirt.org; Itamar Heim Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated Dave, If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true? Thanks, Ofri - Original Message - From: Itamar Heim ih...@redhat.com To: Ofri Masad oma
Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Hi Jimmy, When the engine starts, it will start 'learning' the current state of known hosts. So if we want to secure all hosts in 'up' state, you will need to change it to 'unassigned' only if cluster.trusted == true in- org.ovirt.engine.core.vdsbroker.ResourceManager.AddVds(VDS, boolean) This means the host will be picked up by: org.ovirt.engine.core.vdsbroker.VirtMonitoringStrategy.isMonitoringNeeded(VDS) and the attestation query should get the results and update the trust level and accordingly the vds status. Note that we assume that trust level takes precedence on other functionalities, as this flow will cause storage connections to reinitialize as well. Basically it means that booting will take longer, but this is the price of security. - Original Message - From: Gang Wei gang@intel.com To: Itamar Heim ih...@redhat.com, Doron Fediuck dfedi...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Sunday, April 28, 2013 2:06:24 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated I like the ideas of 2-phase aggregated attestation cluster-by-cluster order. But I want to understand the process more clearly. Without TCP, how does engine handle the states of existing hosts during engine booting? Will engine put all existing hosts in non-operational state and then perform some check via VDSM then turn it into operational state? Put host in non-operational state will cause VM migration, right? Or there is a global state in engine to indicate whether user is allowed to create VM? Thanks Jimmy Itamar Heim wrote on 2013-04-28: On 04/28/2013 11:34 AM, Doron Fediuck wrote: Hi Dave, Just to make sure I fully understand, I'll repeat your basic arguments; 1. It takes time to query a big number of hosts (hundreds). 2. When backend is booting, a user may start a VM on a host which was hacked during the downtime of the engine. If the above is your concern, it shouldn't be so. The reason is, that no host will become operational before you get a response from the attestation server and allow it to become operational. So a user cannot start a new VM on a non-operational host. i'd do the queries in groups of cluster, so cluste-by-cluster they get unblocked. cluster without attestation service shouldn't block on this of course. What this means is that your thread may need to update the user by sending a periodic event that a large scale attestation operation is in progress. Other than that, maybe your thread can work in smaller groups if it gets better results? ie- instead of one query for 300 hosts, maybe you can run 3 serialized queries for 100 hosts each? If this does not help, maybe you can run a short query for something like 10 hosts, which should get an answer relatively fast. The you can issue a query for the other 290 hosts which will take longer. In this way the system may get 10 hosts to work with quite fast, and later on the other 290 hosts will join... So this can actually be configurable to a 2-phase process; a short query and a longer one. The admin can choose the short query size based on his setup, and the longer query can pick up all the other hosts. What do you think? Doron - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Doron Fediuck dfedi...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Saturday, April 27, 2013 9:36:44 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated Hi, Our current consideration is add a new thread in engine's side to attest all of hosts (aggregated query from attestation sever) one time in case of engine's rebooting. There is still one potential issue under extreme condition, saying, hundreds of nodes in a datacenter, attest all of hosts still may take couple of mins, let's say, one hacked untrusted node before receiving the latest status may considered as a trusted host, so, the worst case in a datacenter with hundreds of nodes is, 1. engine is down for some reasons and boot up again, some trusted nodes may be hacked and rebooted during this period. 2. our thread is running to get all of node's status (trust /untrusted), may take couple of mins in large datacenter. 2. user create VMs on these hacked nodes and believe these VMs are trusted VMs launched on trusted nodes. 3. our thread get the correct status of these untrusted nodes, set these nodes as non-operational. 4. all of these trusted VMs running on these untrusted nodes are expected to migrate to other trusted node. So, the question is in a trusted cluster with hundreds of nodes some VMs expected to create on trusted nodes may actually create on untrusted nodes instead, and this time may last for couple of mins. (worst case in my view
Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
- Original Message - From: Oved Ourfalli ov...@redhat.com To: Itamar Heim ih...@redhat.com, Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Sunday, April 21, 2013 8:41:50 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated - Original Message - From: Itamar Heim ih...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org engine-devel@ovirt.org Sent: Saturday, April 20, 2013 5:49:47 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated On 04/19/2013 12:21 PM, Chen, Wei D wrote: Hi All, Our second approach for trusted compute pools integration with oVirt seems more simple and sensible than previous VM level approach. Welcome any comments on our latest design. Thanks in advance. Link is here, http://www.ovirt.org/Trusted_compute_pools a few nits: 1. last updated date isn't updated... 2. from reading it top to bottom, hard to understand the 2nd approach is the one to be used and not the first (vm level). 3. cluster dialog - the 'trusted' should be a checkbox, not radio button, and should only be enabled if virt service was chosen. I'd also consider putting this property in a different side tab. Perhaps Cluster policy side tab would fit? (dividing it into two sections scheduling policy and additional properties or something similar. What do you think about that? I prefer such optimizations to wait for our scheduling work to be more advanced. thanks, Itamar ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
- Original Message - From: Wei D Chen wei.d.c...@intel.com To: Ofri Masad oma...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated Ofri, Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of quartz job? Is there any class similar with InitVdsOnUpCommand for engine's initialization? Best Regards, Dave Chen org.ovirt.engine.core.bll.Backend.Initialize() Note you cannot block this method while waiting for results. Instead I suggest you fire a one-time background request from this method. -Original Message- From: Ofri Masad [mailto:oma...@redhat.com] Sent: Sunday, April 21, 2013 3:29 PM To: Chen, Wei D Cc: Oved Ourfalli; engine-devel@ovirt.org; Itamar Heim Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated Dave, If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true? Thanks, Ofri - Original Message - From: Itamar Heim ih...@redhat.com To: Ofri Masad oma...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, Wei D Chen wei.d.c...@intel.com, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated On 04/21/2013 10:13 AM, Ofri Masad wrote: Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query. are we sure this optimization is needed? how heavy/latent is the call to the attestation service? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9
Generally speaking I agree, we can drop the periodic check is this is the way we expect it to work (ie- change trust level only during reboot). The only thing I'd like to verify is what happens is we miss something. ie- let's assume the engine crashed. During the engine down time a host reboots and becomes untrusted. Now what? The same goes for network disconnect. I'd expect to run a query in such cases to make sure we do not miss a state change. - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Omer Frenkel ofren...@redhat.com, Ofri Masad oma...@redhat.com, Doron Fediuck dfedi...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 10:42:10 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 Yes, the host must be rebooted to take effect. Doron, what do you think? Best Regards, Dave Chen -Original Message- From: Omer Frenkel [mailto:ofren...@redhat.com] Sent: Thursday, April 18, 2013 3:20 PM To: Ofri Masad Cc: Chen, Wei D; Oved Ourfalli; engine-devel@ovirt.org Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 - Original Message - From: Ofri Masad oma...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 9:38:26 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 Hi Dave, Can't a host become untrusted without being rebooted? If that is really the case, there is no need for a periodic check - the trigger for the check would be the host rebooting (which is visible to the engine). +1 Thanks, Ofri - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Itamar Heim ih...@redhat.com, Ofri Masad oma...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 9:19:03 AM Subject: RE: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 I think it's more sensible, the initial status should be the real status for this host (trusted / untrusted) only the untrusted host will be set to non-operational. we just need poll this host instead of all of the hosts in the cluster if this can be done in InitVdsOnUpCommand, and we suppose this is the first time when this host add into the cluster. Follow these steps (conclusion based on your suggestion). - The first time when one host is added into a trust cluster then poll this host, the host will be in up status if get trusted result from attestation server, or else, set this host as non-operational status. - The periodic check will poll all of the hosts only once as this will cut down a lot of time needed instead of poll each host one by one, this conclusion is based on our experiments because most of time consumed is in parallel. - When host is down for a different reason and up again, we suppose InitVdsOnUpCommand will be triggered again (right?), so we will poll this host again to get the real status(trusted / untrusted). Then mark the host as up or non-operational. right As a trusted host flip to untrusted and take effective only under the condition of this host has been hacked and rebooted, so we proposal no need to add periodic check if any host's reboot will invoke InitVdsOnUpCommand and we add attestation logic in InitVdsOnUpCommand is enough. My proposal would be like this (no periodic check is needed, will simply our integration) - The first time when one host is added into a trust cluster then poll this host, the host will be in up status if get trusted result from attestation server, or else, set this host as non-operational status. - The periodic check will poll all of the hosts only once as this will cut down a lot of time needed instead of poll each host one by one, this conclusion is based on our experiments because most of time consumed is in parallel. - (up - nonoperationl / trusted - untrusted)When host is down for a different reason and up again, we suppose InitVdsOnUpCommand will be triggered again (right?) and we also suppose the host will be non-operational if host is down for some reason (right), so we will poll this host again to get the real status(trusted / untrusted) after host's rebooting. - (nonoperationl - up / untrusted - trusted) Admin will activate this host manually after admin fix the issue of this host. Attestation logic will be added into VdsManager:activate () as the clue you give me before. Any suggestion? Best Regards, Dave Chen -Original Message- From: Itamar Heim [mailto:ih...@redhat.com] Sent: Wednesday, April 17, 2013 8:31
Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9
True, I just want folks to be aware of it. So this should resolve it all. Now just make sure to optimize the attestation call. - Original Message - From: Ofri Masad oma...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 11:31:28 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 We run the query each time the host is moving to UP state. Which means, we query all the hosts on engine restart. if the host was unreachable or down for any reason - we will query it again before moving to UP state. Ofri - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: Omer Frenkel ofren...@redhat.com, Ofri Masad oma...@redhat.com, Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 11:23:47 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 Generally speaking I agree, we can drop the periodic check is this is the way we expect it to work (ie- change trust level only during reboot). The only thing I'd like to verify is what happens is we miss something. ie- let's assume the engine crashed. During the engine down time a host reboots and becomes untrusted. Now what? The same goes for network disconnect. I'd expect to run a query in such cases to make sure we do not miss a state change. - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Omer Frenkel ofren...@redhat.com, Ofri Masad oma...@redhat.com, Doron Fediuck dfedi...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 10:42:10 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 Yes, the host must be rebooted to take effect. Doron, what do you think? Best Regards, Dave Chen -Original Message- From: Omer Frenkel [mailto:ofren...@redhat.com] Sent: Thursday, April 18, 2013 3:20 PM To: Ofri Masad Cc: Chen, Wei D; Oved Ourfalli; engine-devel@ovirt.org Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 - Original Message - From: Ofri Masad oma...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 9:38:26 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 Hi Dave, Can't a host become untrusted without being rebooted? If that is really the case, there is no need for a periodic check - the trigger for the check would be the host rebooting (which is visible to the engine). +1 Thanks, Ofri - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Itamar Heim ih...@redhat.com, Ofri Masad oma...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org Sent: Thursday, April 18, 2013 9:19:03 AM Subject: RE: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 I think it's more sensible, the initial status should be the real status for this host (trusted / untrusted) only the untrusted host will be set to non-operational. we just need poll this host instead of all of the hosts in the cluster if this can be done in InitVdsOnUpCommand, and we suppose this is the first time when this host add into the cluster. Follow these steps (conclusion based on your suggestion). - The first time when one host is added into a trust cluster then poll this host, the host will be in up status if get trusted result from attestation server, or else, set this host as non-operational status. - The periodic check will poll all of the hosts only once as this will cut down a lot of time needed instead of poll each host one by one, this conclusion is based on our experiments because most of time consumed is in parallel. - When host is down for a different reason and up again, we suppose InitVdsOnUpCommand will be triggered again (right?), so we will poll this host again to get the real status(trusted / untrusted). Then mark the host as up or non-operational. right As a trusted host flip to untrusted and take effective only under the condition of this host has been hacked and rebooted, so we proposal no need to add periodic check if any host's reboot will invoke InitVdsOnUpCommand and we add attestation logic in InitVdsOnUpCommand is enough. My proposal would be like this (no periodic check is needed
Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9
- Original Message - From: Itamar Heim ih...@redhat.com To: Oved Ourfalli ov...@redhat.com Cc: engine-devel@ovirt.org Sent: Monday, April 15, 2013 9:49:12 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 On 04/15/2013 09:20 AM, Oved Ourfalli wrote: - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Doron Fediuck dfedi...@redhat.com, Ofri Masad oma...@redhat.com Cc: engine-devel@ovirt.org Sent: Monday, April 15, 2013 8:54:18 AM Subject: Re: [Engine-devel] minutes for sync up on Open Attestation integration with oVirt in 4/9 Hi Doron and Ofri, Thanks for your reply, here is some other question. ii. When adding a host into the trusted cluster, the host will be attested via OAT service, only trusted hosted can be added. Would you also kindly tell me if there is any mandatory logic check when adding a host into a cluster, so we can also put attestation logic with these mandatory check together? Some example or code location is better. Thanks a lot. I think there are two approaches, depending on the use case. #1: If the host trusted property is static, then you can narrow the tests to two locations: * AddVdsCommand - (Vds = Host) - This command is triggered when a new host is added to the system. You can use the canDoAction method (which we have on all commands, and it determines whether an action can be run or not), and there, if cluster requires trusted hosts only, you can test whether this host is trusted or not. * ChangeVdsClusterCommand - this command is used when you change the cluster of a host. So, if the target cluster requires tursted hosts, you can do a similar test in its canDoAction method. #2: If the host trusted property can change, then I'd suggest using the NonOperational property of a host. We have a class called VdsUpdateRunTimeInfo that does periodic tests of hosts, deciding what's their status, according to specific flags. The code there is quite complex, and would require refactoring at some point, but in the meantime you can use the MonitoringStrategy interface that is being used there, and implement a new monitoring strategy for Open Attestation. There, in the processHardwareCapabilities, you can test that the host is trusted. When creating the strategy, using the MonitoringStrategyFactory, you'll need to create the appropriate strategy. Currently we support either virt strategy or gluster strategy or both. In your case you would need virt+attestation / gluster+attestation / virt+gluster+attestation, according to the services deployed on the cluster. I'd go with the 2nd approach. i.e., not block the host from being added, only from being used, based on monitoring / non-op status +1. It means that the admin can add the host, but the host will not be operational until we get a positive indication from the attestation service. Does that make sense? Doron and Ofri, what did you have in mind for this feature? Regards, Oved Best Regards, Dave Chen -Original Message- From: Doron Fediuck [mailto:dfedi...@redhat.com] Sent: Wednesday, April 10, 2013 8:03 PM To: Ofri Masad Cc: Wei, Gang; Chen, Wei D; Cao, Buddy; Zhang, Lijuan Subject: Re: minutes for sync up on Open Attestation integration with oVirt in 4/9 - Original Message - From: Ofri Masad oma...@redhat.com To: Gang Wei gang@intel.com Cc: Wei D Chen wei.d.c...@intel.com, Buddy Cao buddy@intel.com, Lijuan Zhang lijuan.zh...@intel.com, Doron Fediuck dfedi...@redhat.com Sent: Wednesday, April 10, 2013 2:23:57 PM Subject: Re: minutes for sync up on Open Attestation integration with oVirt in 4/9 Hi, answers inside the mail body. - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: Gang Wei gang@intel.com, Buddy Cao buddy@intel.com, Lijuan Zhang lijuan.zh...@intel.com, Ofri Masad oma...@redhat.com Sent: Wednesday, April 10, 2013 12:12:26 PM Subject: Re: minutes for sync up on Open Attestation integration with oVirt in 4/9 Hi Dave, Adding Ofri to answer your questions. - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Gang Wei gang@intel.com, Doron Fediuck dfedi...@redhat.com, Buddy Cao buddy@intel.com, Lijuan Zhang lijuan.zh...@intel.com Sent: Wednesday, April 10, 2013 6:31:05 AM Subject: RE: minutes for sync up on Open Attestation integration with oVirt in 4/9 Hi Doron, iii.Then periodic trust check will be added into background process for all existing hosts, once becoming non-trusted, mark it as non-operational. - [Dave] Would you give me some details about the “background process” mentioned in our sync up meeting? What’s the process and where is related code
[Engine-devel] oVirt history; now in starwars!
Here's a better way to see the project history: Engine http://starlogs.net/#oVirt/ovirt-engine VDSM http://starlogs.net/#oVirt/vdsm ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] watchdog
- Original Message - From: Itamar Heim ih...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel ovirt.org Sent: Sunday, April 14, 2013 5:04:55 PM Subject: Re: [Engine-devel] watchdog On 04/11/2013 03:54 PM, Laszlo Hornyak wrote: Hi, It is not really an optimization based only on console connection. In my opinion this should be a decision factor when determining which VM to migrate from an overloaded host. If it is a desktop and no console connection, then it is better candidate than a desktop with console connection or a server with or without console connection. Basically this logic would just want to minimize the pain caused by the short lag when a VM is live migrated. Can I assume that a Desktop is only used when a console is connected? connected by spice? by vnc to host or to guest? by rdp? by remote debugging to it from a remote machine, by someone developing on it with ssh, etc... i agree the general use case you are describing make sense, but i rather we model scheduling decisions directly if/where possible Some parts of this thread are looking for perfection and on the way creating additional tasks and reviewing cycles that are only remotely related to the watchdog device; Yes, we should try and reduce the existing differentiation between desktop and a server. No, this feature is not about closing that gap. Watchdog device is about handling guest crash. I strongly suggest that merging desktop and server dialogues will be done as a different feature. The watchdog feature will take one step in the 'right' direction- adding the (currently missing) HA tab into the desktop dialog based on current design. Doron ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review
- Original Message - From: Wei D Chen wei.d.c...@intel.com To: Doron Fediuck dfedi...@redhat.com, Ofri Masad oma...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, March 29, 2013 5:00:55 AM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Thanks Doron Ofri, As to the question of cache flash, we already have our consideration and wrote them on our design page. I have no doubt that your suggestion is more reasonable, we just keep in mind that expiration is much longer that the time needed to poll all of hosts, so this is really a potential issue we ignored. Let's make estimation at first, we will have a try if our schedule is okay. Doron, we have reserved some effort to research about cluster-level policy. As ovirt is complete new to our engineers, would we finished our current features (such as ovf and rest api.) in pipeline at first? After these basic features are ready and we still have some buffer, we will make some improvement. Is this acceptable? Thanks again to Doron and Ofri. Best Regards, Dave Chen Hi Dave, Thanks for your response. Looking for additional reference I also read [1], which gave me some insights on your design. There is a difference between both architectures which has a big impact on the implementation, so I think it would be wise to explain it; OpenStack currently has no clear definition of cluster as a migration-domain the same way oVirt has. So this means you cannot use this benefit of oVirt which provides you a domain where all VMs should be able to freely migrate from any host to any other host. In OpenStack you may have more than one scheduling service, which means you may scale very large numbers of scheduling requests. In oVirt's current implementation, it will be handled by the same process, so every scheduling delay should be carefully considered to avoid a performance hit. In my suggestion of a cluster level policy, you will get the trust level you need without changing the oVirt scheduler at all(!!!), so you are using the current concepts while avoiding any performance issue you may introduce by adding trust to the scheduling logic. As you can see this should give you a much cleaner, simpler and safer implementation. Working in cluster level will require a different implementation in the engine side to make sure the cluster complies with the trust level you want. The Attestation broker and caching are still needed, but the whole scheduling part should be removed. Obviously this will give you a completely different API as well as UI and potentially OVF, which will make current implementation redundant. This is not an improvement, rather than a different design. I strongly suggest you re-think the process to better evaluate both options. Doron [1] https://wiki.openstack.org/wiki/TrustedComputingPools -Original Message- From: Doron Fediuck [mailto:dfedi...@redhat.com] Sent: Thursday, March 28, 2013 10:43 PM To: Ofri Masad Cc: engine-devel@ovirt.org; Chen, Wei D Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review - Original Message - From: Ofri Masad oma...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Thursday, March 28, 2013 12:05:02 PM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Hi Dave, I would like to raise again the question of the full cache flash for each stale cache entry found. This method can cause two unwanted situations: 1. Choosing untrusted host: lets say, for example that you have 1000 host in your pool. you look at the first host in the cache and find that its attestation hat expired. you refresh the entire pool (there are 1000 host, that must take some time). by the the time the last host was refreshed in the pool, the first host may already be expired again. but since you already checked it - you keep on with your flow and select that host, even so it has expired and may as well be untrusted. 2. infinite loop: lets say we'll try to fix what I've described in 1. then, we need to check again if the host has expired before we select it. if it is, the entire refresh process starts again. this could potentially go on forever (unless I'm missing something, and the expiration is much longer then the full re-cache process). Instead of re-caching the full cache we can do as follows: - hold the cache entries sorted by expiration (if the expiration time is the same for all hosts, so a queue is enough). - each time we need a new trusted host - select from the unexpired hosts, refresh all expired hosts (in one query). - if all hosts are expired - we can wait for the first host to be defined trusted by the attestation server and select that host
Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review
- Original Message - From: Michael Kublin mkub...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Tuesday, April 2, 2013 2:49:02 PM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review - Original Message - From: Wei D Chen wei.d.c...@intel.com To: Doron Fediuck dfedi...@redhat.com, Ofri Masad oma...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, March 29, 2013 5:00:55 AM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Thanks Doron Ofri, As to the question of cache flash, we already have our consideration and wrote them on our design page. I have no doubt that your suggestion is more reasonable, we just keep in mind that expiration is much longer that the time needed to poll all of hosts, so this is really a potential issue we ignored. Let's make estimation at first, we will have a try if our schedule is okay. Doron, we have reserved some effort to research about cluster-level policy. As ovirt is complete new to our engineers, would we finished our current features (such as ovf and rest api.) in pipeline at first? After these basic features are ready and we still have some buffer, we will make some improvement. Is this acceptable? Thanks again to Doron and Ofri. Best Regards, Dave Chen -Original Message- From: Doron Fediuck [mailto:dfedi...@redhat.com] Sent: Thursday, March 28, 2013 10:43 PM To: Ofri Masad Cc: engine-devel@ovirt.org; Chen, Wei D Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review - Original Message - From: Ofri Masad oma...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Thursday, March 28, 2013 12:05:02 PM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Hi Dave, I would like to raise again the question of the full cache flash for each stale cache entry found. This method can cause two unwanted situations: 1. Choosing untrusted host: lets say, for example that you have 1000 host in your pool. you look at the first host in the cache and find that its attestation hat expired. you refresh the entire pool (there are 1000 host, that must take some time). by the the time the last host was refreshed in the pool, the first host may already be expired again. but since you already checked it - you keep on with your flow and select that host, even so it has expired and may as well be untrusted. 2. infinite loop: lets say we'll try to fix what I've described in 1. then, we need to check again if the host has expired before we select it. if it is, the entire refresh process starts again. this could potentially go on forever (unless I'm missing something, and the expiration is much longer then the full re-cache process). Instead of re-caching the full cache we can do as follows: - hold the cache entries sorted by expiration (if the expiration time is the same for all hosts, so a queue is enough). - each time we need a new trusted host - select from the unexpired hosts, refresh all expired hosts (in one query). - if all hosts are expired - we can wait for the first host to be defined trusted by the attestation server and select that host. Ofri Dave, adding another suggestion on top of Ofri's; Generally speaking, a cluster of hosts defines many joint features (such as CPU level), which means that in the same cluster we would expect to be able to freely migrate a VM from one host to another. Current trust-pools design is breaking this concept, as you introduce a state where a VM cannot migrate from a 'safe' host into an 'unsafe' host. This leads me to the suggestion of having attestation as a cluster policy rather than a VM-level property. It means that all hosts in this cluster are constantly being monitored to be safe. If a host is declared as unsafe in the Attestation server, it will become non-operational in the engine. This will simplify the implementation since you have everything ready for you once you have a 'safe' cluster and no need to do any VM-level changes. So in this way you keep current concepts while simplifying the implementation with very little worries of performance issues. Can you please share your thoughts on this suggestion? - Original Message - From: Wei D Chen wei.d.c...@intel.com To: engine-devel@ovirt.org Sent: Friday, March 22, 2013 11:34:55 AM Subject: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your
Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review
- Original Message - From: Ofri Masad oma...@redhat.com To: Wei D Chen wei.d.c...@intel.com Cc: engine-devel@ovirt.org Sent: Thursday, March 28, 2013 12:05:02 PM Subject: Re: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Hi Dave, I would like to raise again the question of the full cache flash for each stale cache entry found. This method can cause two unwanted situations: 1. Choosing untrusted host: lets say, for example that you have 1000 host in your pool. you look at the first host in the cache and find that its attestation hat expired. you refresh the entire pool (there are 1000 host, that must take some time). by the the time the last host was refreshed in the pool, the first host may already be expired again. but since you already checked it - you keep on with your flow and select that host, even so it has expired and may as well be untrusted. 2. infinite loop: lets say we'll try to fix what I've described in 1. then, we need to check again if the host has expired before we select it. if it is, the entire refresh process starts again. this could potentially go on forever (unless I'm missing something, and the expiration is much longer then the full re-cache process). Instead of re-caching the full cache we can do as follows: - hold the cache entries sorted by expiration (if the expiration time is the same for all hosts, so a queue is enough). - each time we need a new trusted host - select from the unexpired hosts, refresh all expired hosts (in one query). - if all hosts are expired - we can wait for the first host to be defined trusted by the attestation server and select that host. Ofri Dave, adding another suggestion on top of Ofri's; Generally speaking, a cluster of hosts defines many joint features (such as CPU level), which means that in the same cluster we would expect to be able to freely migrate a VM from one host to another. Current trust-pools design is breaking this concept, as you introduce a state where a VM cannot migrate from a 'safe' host into an 'unsafe' host. This leads me to the suggestion of having attestation as a cluster policy rather than a VM-level property. It means that all hosts in this cluster are constantly being monitored to be safe. If a host is declared as unsafe in the Attestation server, it will become non-operational in the engine. This will simplify the implementation since you have everything ready for you once you have a 'safe' cluster and no need to do any VM-level changes. So in this way you keep current concepts while simplifying the implementation with very little worries of performance issues. Can you please share your thoughts on this suggestion? - Original Message - From: Wei D Chen wei.d.c...@intel.com To: engine-devel@ovirt.org Sent: Friday, March 22, 2013 11:34:55 AM Subject: [Engine-devel] Open Attestation integration with oVirt engine proposal has submitted patchset5 for your review Hi all, Before submitting this patch set, we has updated our design page, and new feature about VM template has added to this patchset. In patchset a lot of frontend changes has been imported. Welcome to review our patchset and thanks advance for your suggestion. Detailed description: http://wiki.ovirt.org/Trusted_compute_pools In this patch set, follow changes has been introduced: 1. GUI changes to support for creating a trusted VM on a trusted physical host. 2. View/Edit VM changes to enable end user switch between three run on options. 3. Template relevant changes to support end user create a trusted VM template and create trusted VM based on this template afterwards. 4. Bug fixing and code cleanup. 5. wiki design page update. Best Regards, Dave Chen ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Kiril Nesenko as a tools maintainer
+1 and wishing Kiril all the best as a maintainer; Your success it the project's success. - Original Message - From: Ofer Schreiber oschr...@redhat.com To: Keith Robertson krobe...@redhat.com Cc: Kiril Nesenko knese...@redhat.com, engine-devel@ovirt.org Sent: Wednesday, March 20, 2013 4:24:04 PM Subject: Re: [Engine-devel] Kiril Nesenko as a tools maintainer +1 - Original Message - I would like to propose that we add Kiril Nesenko as a maintainer for the Ovirt ISO Uploader, Image Uploader and Log Collector. He has been enormously helpful with the maintenance of the tools. Thanks, Keith Robertson ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Got some troubles when I want to modify oVirt GUI
- Original Message -
From: Itamar Heim ih...@redhat.com
To: Vojtech Szocs vsz...@redhat.com
Cc: engine-devel@ovirt.org
Sent: Tuesday, March 12, 2013 3:30:57 PM
Subject: Re: [Engine-devel] Got some troubles when I want to modify oVirt GUI
On 03/12/2013 12:57 PM, Vojtech Szocs wrote:
Hi,
first of all, did you consider submitting this as RFE in oVirt
bugzilla? Maybe it could be useful to have it in oVirt.
(Implementing this via UI plugin would be far too complicated, as
UI code is tightly coupled with UiCommon code in case of VM
dialog.)
my understanding this is a full blown feature, not a plugin.
Wei D - please note UI is to help the user to not make mistakes, but
validations must also happen at engine side to cover rest api, etc.
+1.
Also, a detailed wiki page may help both developer and readers
to get a better understanding of how it should look and work.
For example, how do you plan to implement the REST API changes?
Wei D, I know that Gang Wei started:
http://www.ovirt.org/Trusted_compute_pools But as you can see,
it does not mention your original intention to implement the UI
as a plugin, while as you can see it would be better to have it
as an internal UI addition. Also, as mentioned REST parts are
missing and a more detailed design for the engine is missing here.
Wei D, is there a reason why not top have a detailed page similar
to: http://www.ovirt.org/Features/Watchdog_engine_support ?
Regarding UI code changes, the general idea is to implement
business logic in UiCommon models (VmListModel, UnitVmModel, etc.)
and have UI code bind to these models. It would be best if you
just send a patch (diff) instead of specific files, it's really
hard to see what changes you made, but based on the files you
sent, here are my comments:
* Changes in UnitVmModel look good, you basically added two new
fields [privateRunVMOnSpecificHost, privateRunVMOnTrustedHost],
hooked up their *_EntityChanged methods, and implemented logic for
handling field value changes in these methods
* Changes in VmListModel look good, you used newly added
UnitVmModel fields in onSave [I assume
setTrustedHostFlag/setDedicatedVmForVds are new fields for VM
entity?], note - you might also want to update
UpdateActionAvailability disable migrating VM when
RunVMOnTrustedHost=true, etc.
* AbstractVmPopupWidget already has specificHost radio button
drop-down on Host dialog tab, and I assume you want to reuse the
drop-down (host list) for trustedHost, so just add new radio
button there:
AbstractVmPopupWidget.ui.xml line 331
g:HorizontalPanel verticalAlignment='ALIGN_MIDDLE'
g:RadioButton name=specificOrTrustedHostGroup
ui:field=specificHost
addStyleNames={style.radioButtonSpecificHost} /
g:RadioButton name=specificOrTrustedHostGroup
ui:field=trustedHost
addStyleNames={style.radioButtonSpecificHost} /
g:Label ui:field=specificHostLabel
text={constants.specificVmPopup} /
e:ListModelListBoxEditor ui:field=defaultHostEditor /
/g:HorizontalPanel
* In AbstractVmPopupWidget you to bind newly added RadioButton:
@UiField(provided = true)
@Ignore
@WithElementId(trustedHost)
public RadioButton trustedHost;
You create trustedHost widget in constructor, and in
initTabAvailabilityListeners you just add
trustedHost.addValueChangeHandler(...) to have logic when
trustedHost gets selected.
Regards,
Vojtech
- Original Message -
From: Wei D Chen wei.d.c...@intel.com
To: engine-devel@ovirt.org
Sent: Tuesday, March 12, 2013 9:48:34 AM
Subject: [Engine-devel] Got some troubles when I want to modify
oVirt GUI
Hi,
In order to add new feature to Ovirt, that is user can choose
virtual machine whether on trusted machine or not when it runs
up, we modified the relative files.
Our goal is when the user click the trusted button, Run/Migration
options are disabled. But unfortunately, we haven’t succeeded in
graphic interface.
I modified these files, I can’t see Host Tab, can you give me some
help? Maybe we need modify more files. We did the following
efforts:
(1) add a trusted radio button.
(2) Modify AbstractVmPopupWidget.ui.xml
g:HorizontalPanel verticalAlignment='ALIGN_MIDDLE'
g:RadioButton ui:field=runVMOnTrustedHost/
e:EntityModelRadioButtonEditor width=150px
ui:field=runVMOnTrustedHostEditor
addStyleNames={style.radioButton} /
/g:HorizontalPanel
(3) Modify AbstractVmPopupWidget.java
@UiField(provided = true)
@Path(value = runVMOnTrustedHost.entity)
@WithElementId(runVMOnTrustedHost)
public EntityModelRadioButtonEditor runVMOnTrustedHostEditor;
initListeners method:
object.getIsAutoAssign().getPropertyChangedEvent().addListener(new
IEventListener() {
@Override
public void
Re: [Engine-devel] new engine watchdog version
- Original Message - From: Omer Frenkel ofren...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, March 11, 2013 2:25:39 PM Subject: Re: [Engine-devel] new engine watchdog version - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Omer Frenkel ofren...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, March 11, 2013 12:15:39 PM Subject: Re: [Engine-devel] new engine watchdog version - Original Message - From: Omer Frenkel ofren...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, March 11, 2013 11:12:48 AM Subject: Re: [Engine-devel] new engine watchdog version - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Omer Frenkel ofren...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, March 11, 2013 9:59:53 AM Subject: Re: [Engine-devel] new engine watchdog version - Original Message - From: Omer Frenkel ofren...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Sunday, March 10, 2013 8:36:46 AM Subject: Re: [Engine-devel] new engine watchdog version - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Friday, March 8, 2013 7:18:59 PM Subject: [Engine-devel] new engine watchdog version Hi, I uploaded a new version of the watchdog patch. This patch is still a work in progress, it adds audit log alerts to the functionality. http://gerrit.ovirt.org/12419/ Feature page: http://www.ovirt.org/Features/Watchdog_engine_support Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel Hi, i looked at the patch and there is something i don't understand, i see you are treating the watchdog as a vm device, which is great, so why do we need to save the device details in vm_static table in addition to the vm_devices? i think its even not used at all (only setting the device in command which could be parameters, no need to persist) Hi Omer, Thanks, I hoped someone will come up with that question :) The answer is that I followed the established design patterns in the backend. See smartcard and memory balloon, probably others. The motivation for this pattern could be that in case of these devices, you must have the settings in the VM data, not separately in the devices. Also when vdsbroker builds the devices list, it just asks the device list. The redundancy is already there, we can make it differently in this case but that will present the readers with a puzzle: why this pattern in feature X, why that pattern in feature Y... So I would recommend to leave it like this for now and schedule a cleanup on device handling. Devices deserve a cleanup. Thx, Laszlo i agree there is a mess that requires clean-up, but i don't think its a good thing to keep piling up the mess, i don't like it that smartcard is there, but some other devices are ok (balloon and payload) so we already have 2 'patterns', lets go with the right one.. and answering also @Doron's question - yes the device data should be kept with the device Ok, I may have missed the other pattern, could you explain which one do you mean? Balloon does not very different from smartcard, it is there in VM. the difference is that balloon is not in vm_static table at all (the only place in the db for it is in vm_devices) and smartcard has 'is_smartcard_enabled' field in vm_static in addition to vm_devices (which is not needed..) the way i think we (currently) need to work with devices is: add a parameter for it in the parameters, and use it in add/update (/run-once?) (as done for balloon) i don't know what is the use of the field balloonEnabled in VM, i don't see any use of it.. going forward we need to think if we want to expose devices to frontend, so then we can drop the encapsulation and just use list of devices in VmBase or something like that Or VMDeviceBase... In the meantime, it seems that watchdog needs to use spec params as vga card does. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] new engine watchdog version
- Original Message - From: Omer Frenkel ofren...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Sunday, March 10, 2013 9:36:46 AM Subject: Re: [Engine-devel] new engine watchdog version - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Friday, March 8, 2013 7:18:59 PM Subject: [Engine-devel] new engine watchdog version Hi, I uploaded a new version of the watchdog patch. This patch is still a work in progress, it adds audit log alerts to the functionality. http://gerrit.ovirt.org/12419/ Feature page: http://www.ovirt.org/Features/Watchdog_engine_support Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel Hi, i looked at the patch and there is something i don't understand, i see you are treating the watchdog as a vm device, which is great, so why do we need to save the device details in vm_static table in addition to the vm_devices? i think its even not used at all (only setting the device in command which could be parameters, no need to persist) Omer, This is WIP, so a few bit are still in the works. Generally speaking model and action are needed since we should be able to use it for template creation, OVF*, and user interaction (user may choose a different model or watchdog for specific VM). Do you recommend on keeping this as spec-param? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] REST API calls from
- Original Message - From: Michael Pasternak mpast...@redhat.com To: Oved Ourfalli ov...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Monday, February 25, 2013 12:33:01 PM Subject: Re: [Engine-devel] REST API calls from On 02/24/2013 03:01 PM, Oved Ourfalli wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Michael Pasternak mpast...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 1:20:12 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Michael Pasternak mpast...@redhat.com To: Oved Ourfalli ov...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 9:47:28 AM Subject: Re: [Engine-devel] REST API calls from On 02/24/2013 09:05 AM, Oved Ourfalli wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Michael Pasternak mpast...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Thursday, February 21, 2013 6:54:59 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Michael Pasternak mpast...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 20, 2013 2:56:59 PM Subject: Re: [Engine-devel] REST API calls from On 02/14/2013 11:20 AM, Doron Fediuck wrote: - Original Message - From: Michael Pasternak mpast...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 13, 2013 12:55:36 PM Subject: Re: [Engine-devel] REST API calls from the GUI Hi Libor, This issue came across in one of the conversations i had with UX folks, but since we didn't end up with any conclusion/road map (nor discussed it properly to hear other thoughts), this is a perfect place to start this discussion, Intuitively REST is a way to go with GWT AJAX calls --- pros - api data objects can be reused by generating java classes (using jaxb) from the rest schema [1] - no backend logic will be duplicated as api abstracts the backend exposing RESTful collection/resources to operate on - development against api is easy as api describes itself in RSDL [2] cons - implementing transport layer (HTTP) under GWT - implementing own j2xml/json/yaml/... marshalling layer - implementing own error handling mechanism - implementing REST callback mechanism (in GWT) - constant maintenance of the data objects generated from the api - painful for Java developers Java-SDK pros - abstracts transport layer (leaving developer in standard Java api) - typesafe code (no need to mess with XML bulks) - has own data objects to work with - abstracts authentication/authorization (kerberos/cookie/session/etc.) - since SDK is auto-generated, it can be easily extended with required features to support UI (such as callback infrastructure for instance) cons - has to be converted in to Javascript (not sure what the impacts are in terms of AJAX calls/etc.) - probably much more cons that we're not aware of and will have to figure out with POC thoughts? [1] http[s]://server[:port]/api?schema [2] http[s]://server[:port]/api?rsdl Although started as a UI request, there are other needs who wish to use API calls with a different transport. For example a backend hook which gets a REST entry point it can use to fetch for additional data, or perform actions. In this case I'd expect an internal connection rather than creating additional connections. How would you resolve it generically enough in this context? Doron, I believe your approach a bit different, UX folks seeking for a convenient way of communicating with ovirt public api, e.g closing api-GUI gap, and theirs alternatives where native HTTP layer or Java-SDK based framework, while what you need is in-process channel to communicate with the engine itself, i understanding your will of using stable api for this (RESTapi), but not sure that doing this via JavaSDK is a good way to go simply because SDK
Re: [Engine-devel] REST API calls from
- Original Message - From: Oved Ourfalli ov...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 11:56:49 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Oved Ourfalli ov...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 3:55:28 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Oved Ourfalli ov...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 3:01:19 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Michael Pasternak mpast...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 1:20:12 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Michael Pasternak mpast...@redhat.com To: Oved Ourfalli ov...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 9:47:28 AM Subject: Re: [Engine-devel] REST API calls from On 02/24/2013 09:05 AM, Oved Ourfalli wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Michael Pasternak mpast...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Thursday, February 21, 2013 6:54:59 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Michael Pasternak mpast...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 20, 2013 2:56:59 PM Subject: Re: [Engine-devel] REST API calls from On 02/14/2013 11:20 AM, Doron Fediuck wrote: - Original Message - From: Michael Pasternak mpast...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 13, 2013 12:55:36 PM Subject: Re: [Engine-devel] REST API calls from the GUI Hi Libor, This issue came across in one of the conversations i had with UX folks, but since we didn't end up with any conclusion/road map (nor discussed it properly to hear other thoughts), this is a perfect place to start this discussion, Intuitively REST is a way to go with GWT AJAX calls --- pros - api data objects can be reused by generating java classes (using jaxb) from the rest schema [1] - no backend logic will be duplicated as api abstracts the backend exposing RESTful collection/resources to operate on - development against api is easy as api describes itself in RSDL [2] cons - implementing transport layer (HTTP) under GWT - implementing own j2xml/json/yaml/... marshalling layer - implementing own error handling mechanism - implementing REST callback mechanism (in GWT) - constant maintenance of the data objects generated from the api - painful for Java developers Java-SDK pros - abstracts transport layer (leaving developer in standard Java api) - typesafe code (no need to mess with XML bulks) - has own data objects to work with - abstracts authentication/authorization (kerberos/cookie/session/etc.) - since SDK is auto-generated, it can be easily extended with required features to support UI (such as callback infrastructure for instance) cons - has to be converted in to Javascript (not sure what the impacts are in terms of AJAX calls/etc.) - probably much more cons that we're not aware of and will have to figure out with POC thoughts? [1] http[s]://server[:port]/api?schema [2] http[s]://server[:port]/api?rsdl Although started as a UI request, there are other needs who wish to use API calls with a different transport. For example a backend hook which gets a REST entry point it can use to fetch for additional data, or perform actions. In this case I'd expect an internal connection rather than creating additional connections. How would you
Re: [Engine-devel] REST API calls from
- Original Message - From: Michael Pasternak mpast...@redhat.com To: Oved Ourfalli ov...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 9:47:28 AM Subject: Re: [Engine-devel] REST API calls from On 02/24/2013 09:05 AM, Oved Ourfalli wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Michael Pasternak mpast...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Thursday, February 21, 2013 6:54:59 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Michael Pasternak mpast...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 20, 2013 2:56:59 PM Subject: Re: [Engine-devel] REST API calls from On 02/14/2013 11:20 AM, Doron Fediuck wrote: - Original Message - From: Michael Pasternak mpast...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 13, 2013 12:55:36 PM Subject: Re: [Engine-devel] REST API calls from the GUI Hi Libor, This issue came across in one of the conversations i had with UX folks, but since we didn't end up with any conclusion/road map (nor discussed it properly to hear other thoughts), this is a perfect place to start this discussion, Intuitively REST is a way to go with GWT AJAX calls --- pros - api data objects can be reused by generating java classes (using jaxb) from the rest schema [1] - no backend logic will be duplicated as api abstracts the backend exposing RESTful collection/resources to operate on - development against api is easy as api describes itself in RSDL [2] cons - implementing transport layer (HTTP) under GWT - implementing own j2xml/json/yaml/... marshalling layer - implementing own error handling mechanism - implementing REST callback mechanism (in GWT) - constant maintenance of the data objects generated from the api - painful for Java developers Java-SDK pros - abstracts transport layer (leaving developer in standard Java api) - typesafe code (no need to mess with XML bulks) - has own data objects to work with - abstracts authentication/authorization (kerberos/cookie/session/etc.) - since SDK is auto-generated, it can be easily extended with required features to support UI (such as callback infrastructure for instance) cons - has to be converted in to Javascript (not sure what the impacts are in terms of AJAX calls/etc.) - probably much more cons that we're not aware of and will have to figure out with POC thoughts? [1] http[s]://server[:port]/api?schema [2] http[s]://server[:port]/api?rsdl Although started as a UI request, there are other needs who wish to use API calls with a different transport. For example a backend hook which gets a REST entry point it can use to fetch for additional data, or perform actions. In this case I'd expect an internal connection rather than creating additional connections. How would you resolve it generically enough in this context? Doron, I believe your approach a bit different, UX folks seeking for a convenient way of communicating with ovirt public api, e.g closing api-GUI gap, and theirs alternatives where native HTTP layer or Java-SDK based framework, while what you need is in-process channel to communicate with the engine itself, i understanding your will of using stable api for this (RESTapi), but not sure that doing this via JavaSDK is a good way to go simply because SDK is designed to operate in a client-space, while what you need is a server-space bridge for that. Michael, true but... Thinking about it differently both UI and hooks needs a client. The underlying protocols should be abstracted. This is something which will serve other functions as well. I'm not sure we would need a new abstraction here. Both UI plugins and engine plugins need some API to do basic operations, and have access to different properties in the engine. +1, that's exactly what i've suggested to begin with. The only issue is that UI plugins and engine plugins shave different expectations from performance point of view. If UI is willing to wait for a refresh that may take too long for engine plugins, which would prefer to get the information as soon as possible without going into various communication layers which are not always needed. So again- a simple solution which enables transports layers to be replaced may serve more than one functionality in a better way. In the UI plguins implementation, we gave this API, and in addition created a REST session to be used in order to do more sophisticated operations. I think
Re: [Engine-devel] REST API calls from
- Original Message - From: Oved Ourfalli ov...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 3:01:19 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Michael Pasternak mpast...@redhat.com Cc: Oved Ourfalli ov...@redhat.com, engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 1:20:12 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Michael Pasternak mpast...@redhat.com To: Oved Ourfalli ov...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Sunday, February 24, 2013 9:47:28 AM Subject: Re: [Engine-devel] REST API calls from On 02/24/2013 09:05 AM, Oved Ourfalli wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Michael Pasternak mpast...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Thursday, February 21, 2013 6:54:59 PM Subject: Re: [Engine-devel] REST API calls from - Original Message - From: Michael Pasternak mpast...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 20, 2013 2:56:59 PM Subject: Re: [Engine-devel] REST API calls from On 02/14/2013 11:20 AM, Doron Fediuck wrote: - Original Message - From: Michael Pasternak mpast...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 13, 2013 12:55:36 PM Subject: Re: [Engine-devel] REST API calls from the GUI Hi Libor, This issue came across in one of the conversations i had with UX folks, but since we didn't end up with any conclusion/road map (nor discussed it properly to hear other thoughts), this is a perfect place to start this discussion, Intuitively REST is a way to go with GWT AJAX calls --- pros - api data objects can be reused by generating java classes (using jaxb) from the rest schema [1] - no backend logic will be duplicated as api abstracts the backend exposing RESTful collection/resources to operate on - development against api is easy as api describes itself in RSDL [2] cons - implementing transport layer (HTTP) under GWT - implementing own j2xml/json/yaml/... marshalling layer - implementing own error handling mechanism - implementing REST callback mechanism (in GWT) - constant maintenance of the data objects generated from the api - painful for Java developers Java-SDK pros - abstracts transport layer (leaving developer in standard Java api) - typesafe code (no need to mess with XML bulks) - has own data objects to work with - abstracts authentication/authorization (kerberos/cookie/session/etc.) - since SDK is auto-generated, it can be easily extended with required features to support UI (such as callback infrastructure for instance) cons - has to be converted in to Javascript (not sure what the impacts are in terms of AJAX calls/etc.) - probably much more cons that we're not aware of and will have to figure out with POC thoughts? [1] http[s]://server[:port]/api?schema [2] http[s]://server[:port]/api?rsdl Although started as a UI request, there are other needs who wish to use API calls with a different transport. For example a backend hook which gets a REST entry point it can use to fetch for additional data, or perform actions. In this case I'd expect an internal connection rather than creating additional connections. How would you resolve it generically enough in this context? Doron, I believe your approach a bit different, UX folks seeking for a convenient way of communicating with ovirt public api, e.g closing api-GUI gap, and theirs alternatives where native HTTP layer or Java-SDK based framework, while what you need is in-process channel to communicate with the engine itself, i understanding your will of using stable api for this (RESTapi), but not sure that doing this via JavaSDK is a good way to go simply because SDK is designed to operate in a client-space, while what you need is a server-space bridge for that. Michael, true but... Thinking about it differently both UI and hooks needs a client. The underlying protocols should be abstracted. This is something which
Re: [Engine-devel] REST API calls from
- Original Message - From: Michael Pasternak mpast...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 20, 2013 2:56:59 PM Subject: Re: [Engine-devel] REST API calls from On 02/14/2013 11:20 AM, Doron Fediuck wrote: - Original Message - From: Michael Pasternak mpast...@redhat.com To: Libor Spevak lspe...@redhat.com Cc: engine-devel@ovirt.org, a...@ovirt.org Sent: Wednesday, February 13, 2013 12:55:36 PM Subject: Re: [Engine-devel] REST API calls from the GUI Hi Libor, This issue came across in one of the conversations i had with UX folks, but since we didn't end up with any conclusion/road map (nor discussed it properly to hear other thoughts), this is a perfect place to start this discussion, Intuitively REST is a way to go with GWT AJAX calls --- pros - api data objects can be reused by generating java classes (using jaxb) from the rest schema [1] - no backend logic will be duplicated as api abstracts the backend exposing RESTful collection/resources to operate on - development against api is easy as api describes itself in RSDL [2] cons - implementing transport layer (HTTP) under GWT - implementing own j2xml/json/yaml/... marshalling layer - implementing own error handling mechanism - implementing REST callback mechanism (in GWT) - constant maintenance of the data objects generated from the api - painful for Java developers Java-SDK pros - abstracts transport layer (leaving developer in standard Java api) - typesafe code (no need to mess with XML bulks) - has own data objects to work with - abstracts authentication/authorization (kerberos/cookie/session/etc.) - since SDK is auto-generated, it can be easily extended with required features to support UI (such as callback infrastructure for instance) cons - has to be converted in to Javascript (not sure what the impacts are in terms of AJAX calls/etc.) - probably much more cons that we're not aware of and will have to figure out with POC thoughts? [1] http[s]://server[:port]/api?schema [2] http[s]://server[:port]/api?rsdl Although started as a UI request, there are other needs who wish to use API calls with a different transport. For example a backend hook which gets a REST entry point it can use to fetch for additional data, or perform actions. In this case I'd expect an internal connection rather than creating additional connections. How would you resolve it generically enough in this context? Doron, I believe your approach a bit different, UX folks seeking for a convenient way of communicating with ovirt public api, e.g closing api-GUI gap, and theirs alternatives where native HTTP layer or Java-SDK based framework, while what you need is in-process channel to communicate with the engine itself, i understanding your will of using stable api for this (RESTapi), but not sure that doing this via JavaSDK is a good way to go simply because SDK is designed to operate in a client-space, while what you need is a server-space bridge for that. Michael, true but... Thinking about it differently both UI and hooks needs a client. The underlying protocols should be abstracted. This is something which will serve other functions as well. On 02/12/2013 06:13 PM, Libor Spevak wrote: Hi, I would like to ask, if there have been discussions about an option to call REST API services directly from the Frontend (GWT layer)? GWT compiles Java frontend-side to Javascript, calls to backend services are performed transparently by the framework using AJAX support. But, there is still a need to have a special set of data objects and the server-side logic can duplicate. Java REST API SDK enables to build thick client. The calls are realized using e.g. Apache HttClient and supported libraries. I think the requirements of GWT can be a little bit different, but something overlaps. I found several links about REST API support from GWT, so there is something for inspiration... - http://www.spiffyui.org/ - http://www.zackgrossbart.com/hackito/gwt-rest/ - http://code.google.com/p/gwt-rest/ - http://restygwt.fusesource.org/ But, do you think it would be useful and what drawbacks can occur (authentication, authorization, response times, need to support larger set of services, painful refactoring, ...)? Regards, Libor ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Watchdog support Feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, February 11, 2013 9:56:42 AM Subject: Re: [Engine-devel] Watchdog support Feature Hi Doron, If the watchdog is set to action=reset and you break the kernel, the VM will stay in UP state while waiting for the watchdog to reset the VM, and will still be in UP state while rebooting. So in theory engine will not be notified about a crash. This behavior is different when the action=poweroff because the watchdog will really put the VM in a Down state and it is engine's job to restart it if HA as soon as it receives the notification from VDSM. Thanks. Just note that poweroff requires guest co-operation which we loose when kernel crashes. That's why it's not recommended. - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Sunday, February 10, 2013 4:40:29 PM Subject: Re: [Engine-devel] Watchdog support Feature Hi Laszlo, Looks very useful. Any thoughts on what engine will do while if the VM is defined as HA while the watchdog is resetting it? - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Wednesday, February 6, 2013 11:25:03 AM Subject: Re: [Engine-devel] Watchdog support Feature Hi everyone, I hope everyone is satisfied with the spec, please read it through for the last time and I will move it to implementation phase soon. Thx, Laszlo - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Thursday, January 24, 2013 3:19:57 PM Subject: Watchdog support Feature Hi, Watchdog support in engine will add watchdog to the ovirt UI and REST API. Watchdog cards will be mainly used in HA vm's. http://www.ovirt.org/Features/Watchdog_engine_support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Trusted Compute Pools
- Original Message - From: Itamar Heim ih...@redhat.com To: Gang Wei gang@intel.com Cc: engine-devel@ovirt.org Sent: Friday, February 8, 2013 2:02:41 PM Subject: Re: [Engine-devel] Trusted Compute Pools On 01/02/2013 10:25, Wei, Gang wrote: Design page updated according to in process Patchset 2. http://wiki.ovirt.org/wiki/Trusted_compute_pools. Hi Wei, in general, looks good. currently, user who creates the VM can choose the value for this field. i wonder if it should be limited to web admin, and not exposed to power users (user portal) another way to think about this is do you want to set it at cluster policy level, and/or if you want to give a specific permission which is required to edit this field (then users with this permission can override the cluster policy maybe. all this can wait of course, and is not blocking the patch for basic functionality to get community feedback (but may be worth documenting as open question / future thoughts in the wiki. two other thoughts/question 1. may be worth allowing to override the setting for this field in run-once dialog. 2. currently, the dialog allows to choose run on trusted node aren't the various values in TrustLevel enum relevant here for the user? (nit: should probably s/node/Host/) Thanks, Itamar Agree on the cluster level use-case. Actually it may be more effective to move attestation decision to cluster level in general, so you will have an attested-cluster. Otherwise, you may have issues during migration and load balancing which may fail other functionalities in the system. Think of a use case where you want to upgrade the current hosts. If you have a mixture of attested on non-attested hosts you may have to take down a trusted VM since there's no suitable host for it. This will be easy when working in attested-cluster where all VMs are free to migrate as needed based on capacity only. If you choose to keep attestation in VM-level, please explain how do you plan to handle the cases I mentioned (migration, etc). Additionally I'll be glad to see a more detailed design with reference to the various parts- RESTful API, Engine and UI. Thanks! Doron Jimmy Wei, Gang wrote on 2012-11-20: Hi, I am an engineer working in Intel Open Source Technology Center, interested in integrating Intel initiated OpenAttestation(OAT) project (https://github.com/OpenAttestation/OpenAttestation.git) into oVirt to provide a way for Administrator to deploy VMs on trusted hosts hardened with H/W-based security features, such as Intel TXT. I made a draft feature page for this: http://wiki.ovirt.org/wiki/Trusted_compute_pools My draft idea is to provide trust_level requirement while doing vm creation like below: curl -v -u vdcad...@qa.lab.tlv.redhat.com -H Content-type: application/xml -d 'vmnamemy_new_vm/name cluster id=99408929-82cf-4dc7-a532-9d998063fa95 / template id=----/ trust_leveltrusted/trust_level/vm' 'http://10.35.1.1/rhevm-api/vms' Then oVirt Engine should query attestation server built with OAT via RESTful API to get all trusted hosts and select one to create the VM. Attestation server performs host verification through following steps: 1. Hosts boot with Intel TXT technology enabled 2. The hosts' BIOS, hypervisor and OS are measured 3. These measured data is sent to Attestation server when challenged by attestation server 4. Attestation server verifies those measurements against good/known database to determine hosts' trustworthiness Hosts need to be installed with OAT host agent to report host integrity to attestation server. By far, I am still in process of getting familiar with oVirt code and not get solid idea yet on how the oVirt Engine should be modified to support this feature. Any kind of comments or suggestions will be highly appreciated. Thanks Gang (Jimmy) Wei ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] cleanup patch review needed
PING? - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, February 5, 2013 3:29:38 PM Subject: [Engine-devel] cleanup patch review needed Hi, I need someone to review this patch http://gerrit.ovirt.org/#/c/11351/ Thx, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Local Authentication Feature
- Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Sunday, February 10, 2013 5:37:10 PM Subject: Re: [Engine-devel] Local Authentication Feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Juan Hernandez jhern...@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, February 10, 2013 5:26:52 PM Subject: Re: [Engine-devel] Local Authentication Feature - Original Message - From: Juan Hernandez jhern...@redhat.com To: engine-devel@ovirt.org Sent: Friday, February 8, 2013 7:50:36 PM Subject: [Engine-devel] Local Authentication Feature Hello, I would like to propose a new feature that allows authentication using the local user database. The details are here: http://www.ovirt.org/Features/Local_Authentication And the proposed change is available for review here: http://gerrit.ovirt.org/11863 I appreciate feedback. Thanks in advance, Juan Hernandez -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. Hi Juan, Very happy to see this one which actually closes an annoying gap! One thing which is missing is user management- add/remove/change users and groups. If we do not plan to handle it within ovirt, the design should state it and explain how user management should work. Shouldn't this be the same as in case of external directory service? i.e - you manage user/group at the directory service, and then you populate engine with it (by adding permissions to users/groups or adding explicitly new users/groups to engine?) Also, what happens when a user is removed from the local DB- will all references to him be removed? Groups? IMHO the behavior in this case should be as in case of current LdapBroker. This could be a decision but it's missing from the design. The diff I see from current supported directory servers are that they actually have their own management tools, which is not the case for local DB. Again, you may state that the various userXXX and groupXXX commandline utilities are the way to manage it, but this is lacking from the design. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Local Authentication Feature
- Original Message - From: Andrew Cathrow acath...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Sunday, February 10, 2013 7:21:32 PM Subject: Re: [Engine-devel] Local Authentication Feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Yair Zaslavsky yzasl...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Sunday, February 10, 2013 11:02:39 AM Subject: Re: [Engine-devel] Local Authentication Feature - Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Sunday, February 10, 2013 5:37:10 PM Subject: Re: [Engine-devel] Local Authentication Feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Juan Hernandez jhern...@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, February 10, 2013 5:26:52 PM Subject: Re: [Engine-devel] Local Authentication Feature - Original Message - From: Juan Hernandez jhern...@redhat.com To: engine-devel@ovirt.org Sent: Friday, February 8, 2013 7:50:36 PM Subject: [Engine-devel] Local Authentication Feature Hello, I would like to propose a new feature that allows authentication using the local user database. The details are here: http://www.ovirt.org/Features/Local_Authentication And the proposed change is available for review here: http://gerrit.ovirt.org/11863 I appreciate feedback. Thanks in advance, Juan Hernandez -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. Hi Juan, Very happy to see this one which actually closes an annoying gap! One thing which is missing is user management- add/remove/change users and groups. If we do not plan to handle it within ovirt, the design should state it and explain how user management should work. Shouldn't this be the same as in case of external directory service? i.e - you manage user/group at the directory service, and then you populate engine with it (by adding permissions to users/groups or adding explicitly new users/groups to engine?) Also, what happens when a user is removed from the local DB- will all references to him be removed? Groups? IMHO the behavior in this case should be as in case of current LdapBroker. This could be a decision but it's missing from the design. The diff I see from current supported directory servers are that they actually have their own management tools, which is not the case for local DB. Again, you may state that the various userXXX and groupXXX commandline utilities are the way to manage it, but this is lacking from the design. Local user support is a feature we certainly need, but somehow ssh'ing into the node feels wrong. A local db is better than the (creative) ssh hack. IIUC it's an internal SSH just for the authentication part. If it succeeds the user is authenticated. Otherwise the user will fail to login. That's the only use of the ssh. everything else should work as it used to so far. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Watchdog support Feature
- Original Message - From: Dave Allan dal...@redhat.com To: Itamar Heim ih...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, January 28, 2013 7:23:25 PM Subject: Re: [Engine-devel] Watchdog support Feature On Fri, Jan 25, 2013 at 08:34:27AM -0800, Itamar Heim wrote: On 24/01/2013 06:19, Laszlo Hornyak wrote: Hi, Watchdog support in engine will add watchdog to the ovirt UI and REST API. Watchdog cards will be mainly used in HA vm's. http://www.ovirt.org/Features/Watchdog_engine_support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel looks like a nice, simple, addition. dave, anyone more intimate with the watchdog worth giving another look? I had a look at the ovirt feature page and asked some libvirt folks as well, and it all looks good to me except that the libvirt documentation says that an event when watchdog fires will be added in the future. That event has been added. The watchdog should work pretty much identically to the physical world: configure the virtual watchdog card in the guest hardware description and install the watchdog daemon in the guest. Dave Thanks Dave. Do you happen do know why the shutdown action is not recommended? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Watchdog support Feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Dave Allan dal...@redhat.com, Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 29, 2013 4:18:53 PM Subject: Re: [Engine-devel] Watchdog support Feature - Original Message - From: Dave Allan dal...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 29, 2013 2:42:42 PM Subject: Re: [Engine-devel] Watchdog support Feature On Tue, Jan 29, 2013 at 08:06:44AM -0500, Doron Fediuck wrote: - Original Message - From: Dave Allan dal...@redhat.com To: Itamar Heim ih...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, January 28, 2013 7:23:25 PM Subject: Re: [Engine-devel] Watchdog support Feature On Fri, Jan 25, 2013 at 08:34:27AM -0800, Itamar Heim wrote: On 24/01/2013 06:19, Laszlo Hornyak wrote: Hi, Watchdog support in engine will add watchdog to the ovirt UI and REST API. Watchdog cards will be mainly used in HA vm's. http://www.ovirt.org/Features/Watchdog_engine_support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel looks like a nice, simple, addition. dave, anyone more intimate with the watchdog worth giving another look? I had a look at the ovirt feature page and asked some libvirt folks as well, and it all looks good to me except that the libvirt documentation says that an event when watchdog fires will be added in the future. That event has been added. The watchdog should work pretty much identically to the physical world: configure the virtual watchdog card in the guest hardware description and install the watchdog daemon in the guest. Dave Thanks Dave. Do you happen do know why the shutdown action is not recommended? Yes--it requires guest cooperation which it's unlikely to get if the guest has gotten into a state where the watchdog fires. Dave Yep, the guest kernel should handle an ACPI signal. I don't know in what circumstances would that work, but vdsm does not support it either, so that decision is already made. Makes sense, thanks. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Watchdog support Feature
- Original Message - From: Michael Pasternak mpast...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Sunday, January 27, 2013 9:27:54 AM Subject: Re: [Engine-devel] Watchdog support Feature On 01/24/2013 04:19 PM, Laszlo Hornyak wrote: Hi, Watchdog support in engine will add watchdog to the ovirt UI and REST API. Watchdog cards will be mainly used in HA vm's. http://www.ovirt.org/Features/Watchdog_engine_support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel Definitely nice feature, but i have a question for [1], won't it cause slow responsive guests (on highly loaded hosts) to constantly reboot? i.e it's understood that 'reboot' is a goal of this feature, but endless reboot of pinned-to-host guests for instance won't do any good right?, my point is: should watchdog 'action' have extra logic for guest-policy like placement for instance? [1] watchdog model=i6300esb action=reset/ -- Michael Pasternak RedHat, ENG-Virtualization RD Michael, good question. The device itself will not be added by default, and there are a few other actions which can be used (default action is none). So this will be in use only by someone who expects it to reboot. Additionally the watchdog is configurable to fire according to the internal drivers settings, so softdog may be configured to trigger the watchdog by several indicators and not only cpu load / time. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Watchdog support Feature
- Original Message - From: Itamar Heim ih...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Thursday, January 24, 2013 5:54:12 PM Subject: Re: [Engine-devel] Watchdog support Feature On 24/01/2013 07:55, Yaniv Kaul wrote: Are all watchdogs available to all operating systems? ask libosinfo :) Ric' Jones has a known post about it[1]. If you look into it[2], you can see a few Linux distro's are supported (Red Hat, Debian Suse) plus I was able to get it in Fedora's default repo. For Microsoft OS's you may download drivers from Intel's web site. [1] http://rwmj.wordpress.com/2010/03/03/what-is-a-watchdog/#comment-4959 [2] http://watchdog.git.sourceforge.net/git/gitweb.cgi?p=watchdog/watchdog;a=tree ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Java code formatting
This should be a jenkins job, not a local one. - Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, January 7, 2013 3:12:55 AM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Eyal Edri ee...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Sunday, January 6, 2013 4:30:23 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Eyal Edri ee...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Friday, January 4, 2013 5:39:52 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: David Jaša dj...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Friday, January 4, 2013 4:37:13 PM Subject: Re: [Engine-devel] Java code formatting Doron Fediuck píše v St 02. 01. 2013 v 03:24 -0500: - Original Message - From: Livnat Peer lp...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:58:15 PM Subject: Re: [Engine-devel] Java code formatting On 01/01/2013 05:39 PM, Doron Fediuck wrote: - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:05:09 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:00:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:53:49 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:48:22 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:33:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:28:15 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:17:18 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:07:53 PM Subject: [Engine-devel] Java code formatting Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 Hi, These automatic conversions are not better than current state, also I don't think that this is that important. If you want machine written code, then also provide commit hook to reformat anything, and probably machines to read it. I, personally, think that this change over the sources I manage did not do any good. Regards, Alon Alon
Re: [Engine-devel] Java code formatting
- Original Message - From: Eyal Edri ee...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Friday, January 4, 2013 5:39:52 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: David Jaša dj...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Friday, January 4, 2013 4:37:13 PM Subject: Re: [Engine-devel] Java code formatting Doron Fediuck píše v St 02. 01. 2013 v 03:24 -0500: - Original Message - From: Livnat Peer lp...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:58:15 PM Subject: Re: [Engine-devel] Java code formatting On 01/01/2013 05:39 PM, Doron Fediuck wrote: - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:05:09 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:00:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:53:49 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:48:22 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:33:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:28:15 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:17:18 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:07:53 PM Subject: [Engine-devel] Java code formatting Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 Hi, These automatic conversions are not better than current state, also I don't think that this is that important. If you want machine written code, then also provide commit hook to reformat anything, and probably machines to read it. I, personally, think that this change over the sources I manage did not do any good. Regards, Alon Alon, there's a formatting convention for the project set long ago. If you feel it needs to be fixed, go ahead and suggest a fix for the xml. Otherwise we end up in the current chaos, where every 2nd or 3rd patch carries unneeded changes. What do you mean unneeded changes? how do you prevent this in future? Alon Unneeded changes is when you get one line of code fixed due to a bug, and many others re-indented. Best prevention is if people would make sure to use the same conventions. We also have a checkstyle which monitors important issues such as trailing white spaces, localization, etc. But how do you prevent this in future, not all working in same editor nor same
Re: [Engine-devel] Java code formatting
- Original Message - From: Livnat Peer lp...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:58:15 PM Subject: Re: [Engine-devel] Java code formatting On 01/01/2013 05:39 PM, Doron Fediuck wrote: - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:05:09 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:00:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:53:49 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:48:22 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:33:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:28:15 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:17:18 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:07:53 PM Subject: [Engine-devel] Java code formatting Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 Hi, These automatic conversions are not better than current state, also I don't think that this is that important. If you want machine written code, then also provide commit hook to reformat anything, and probably machines to read it. I, personally, think that this change over the sources I manage did not do any good. Regards, Alon Alon, there's a formatting convention for the project set long ago. If you feel it needs to be fixed, go ahead and suggest a fix for the xml. Otherwise we end up in the current chaos, where every 2nd or 3rd patch carries unneeded changes. What do you mean unneeded changes? how do you prevent this in future? Alon Unneeded changes is when you get one line of code fixed due to a bug, and many others re-indented. Best prevention is if people would make sure to use the same conventions. We also have a checkstyle which monitors important issues such as trailing white spaces, localization, etc. But how do you prevent this in future, not all working in same editor nor same styles. You can say that once in a while you perform cross over auto re-format... I am not native Java programmer but this sounds very strange thing to do, I don't think I know of any project that does that. The main problem is that people commit stuff they don't touch due to their editor behavior, which tread the whole source as if it was at its disposal, while cation should be taken not to modify extra stuff. So maybe just to reject patches that touches lines which are not belong to the patch it-self. Alon I'm fine with rejecting patches with irrelevant changes. Still sometimes you get new files and this repeats itself. The whole point of a convention is that people keep it. Most major IDEs today can use the same formatting XML we now have. So it's really a matter of self-discipline. OK, I suggest we start rejecting patches with irrelevant changes, and placing these irrelevant changes in their own patch to be relevant. I don't think
Re: [Engine-devel] Engine local configuration
- Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 10:15:37 AM Subject: Re: [Engine-devel] Engine local configuration - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org, Livnat Peer lp...@redhat.com Sent: Tuesday, January 1, 2013 10:08:32 AM Subject: Re: [Engine-devel] Engine local configuration - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org, Livnat Peer lp...@redhat.com Sent: Tuesday, January 1, 2013 9:56:05 AM Subject: Re: [Engine-devel] Engine local configuration - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org, Livnat Peer lp...@redhat.com Sent: Tuesday, January 1, 2013 9:43:50 AM Subject: Re: [Engine-devel] Engine local configuration - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Livnat Peer lp...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 9:30:20 AM Subject: Re: [Engine-devel] Engine local configuration - Original Message - From: Livnat Peer lp...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 9:05:03 AM Subject: Re: [Engine-devel] Engine local configuration On 12/31/2012 05:05 PM, Doron Fediuck wrote: - Original Message - From: Juan Hernandez jhern...@redhat.com To: Roy Golan rgo...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, December 14, 2012 1:25:57 PM Subject: Re: [Engine-devel] Error on starting webadmin On 12/13/2012 03:55 PM, Roy Golan wrote: On 12/13/2012 04:49 PM, Vojtech Szocs wrote: this is making the contribution process more complex. lets think of a lighter way to get a developing setup. I agree, I just wanted to have the local Engine configuration steps documented for reference.. If there's a better way to do it, I'm for it. Vojtech having LocalConfig look for engine.conf.defaults system property before fallback to System.getenv(ENGINE_DEFAULTS); + concatenating -Dengine.conf.defaults=$HOME/.engine.conf.defaults to JAVA_OPTS on standalone.conf How is the system property simpler than the environment variable? I agree that this makes the development process a bit more complex at the moment, but I think that the way to make it simpler is not to continue adding things to standalone.conf. I think that we should move towards a development environment that is closer to the production environment, not the other way around. Ideally the developer should be able to do something like make install to have the engine deployed to a directory structure similar to what he have in the production environment. Then you should be able to go to the bin directory inside that structure and start the engine (and the other tools) using the same script that we use in production environments. If we achieve this goal then we have a simple development environment setup and also we have all developers testing almost the same thing that will go into the production environments. At the moment we don't have that, most times you are testing something quite different (in terms of directory structure, configuration, etc) to what will be installed in production environments. I am working in that direction. Hi Guys, Sorry to resume this discussion, but I find the current situation unfriendly to most developers. I understand there's a need for specific configurations, but it seems to me that this has taken one step too far for most developers. Further more, I expect to see such fundamental concepts being initially discussed here, and not settle with a technical ack, only
Re: [Engine-devel] Engine local configuration
- Original Message -
From: Juan Hernandez jhern...@redhat.com
To: Doron Fediuck dfedi...@redhat.com
Cc: engine-devel@ovirt.org
Sent: Tuesday, January 1, 2013 12:17:33 PM
Subject: Re: [Engine-devel] Engine local configuration
On 01/01/2013 10:51 AM, Alon Bar-Lev wrote:
- Original Message -
From: Doron Fediuck dfedi...@redhat.com
To: Alon Bar-Lev alo...@redhat.com
Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org
Sent: Tuesday, January 1, 2013 10:30:29 AM
Subject: Re: [Engine-devel] Engine local configuration
All we are saying is DEVELOPMENT=1 is the default, since the
code
is built differently for production, and installed accordingly.
Additionally you may have more than one devel setup, for example
for
different versions. So unless you override something (such as
keystore
pass, jboss location) mvn -Pdep should simply work, and make XXX
will
ensure whatever you need to override is done properly.
No, you miss the point.
I want to be able to have a full blown product installed for
development as well.
The only difference is the PREFIX and may be some other stuff I
don't
think of that are specific of development.
I put the DEVELOPMENT=1 to focus the discussion.
I should have written the following steps to be more clear:
$ OEHOME=$HOME/ovirt-engine-root
$ make PREFIX=$OEHOME
$ make PREFIX=$OEHOME DEVELOPMENT=1 install
$ cd ${OEHOME}
$ . ./ovirt-engine.vars
$ engine-setup
snip
$ sbin/engie-service.py start
We probably need more vars on the make install, such as
JBOSS_HOME,
but the idea is that you have fully functional product in
development mode, including PKI and support scripts.
Regards,
Alon
Alon,
most universities today will not give you admin privileges.
We must be able to download the code and run it without additional
requirements. It is possible to run jboss with local configuration
(per user). So we should minimize the restrictions and not impose
new once. Running the installation script is nice to have, but
should be completely optional. Especially if you'd like to see it
being deployed in various distro's. So at least developers should
be able to git clone, compile and run. Everything else is
packaging.
You don't need any admin to install at
PREFIX=$HOME/ovirt-engine-root
We should make sure that we can run all components within PREFIX
without root.
As far as I can tell, we do not actually need root access for 99%
of the implementation, one of the trivial differences is to
replace systemctl start/stop with direct script execution in
non-root mode.
Alon
As I mentioned on my first response to this thread I have been
working
on trying to make the build system able to install the engine to a
directory chosen by the developer, and without root permissions. This
is
the patch that I prepared for that, please take a look at the commit
message:
http://gerrit.ovirt.org/10539
The idea is that the developer can chose a directory and then just
run
the following:
./make.py --build --install --devel --target /that/directory
Even the complexity of this command line can be hidden in a make
target,
or made the default, so the developer will only need to run something
like this:
make install
This will prepare an environment similar to the production
environment,
with the same directory structure (except the /that/directory prefix)
where the engine can run, so to start it the developer only needs to
go
to that directory and execute the same script that we use in
production
environments:
cd /that/directory/usr/bin
./engine-service start
This will also automatically configure the engine for remote
debugging
(adding ENGINE_DEBUG_ADDRESS=0.0.0.0:8787 to the configuration) so
that
the developer can connect to the engine with her/his favorite IDE and
start debugging immediately.
The make.py script also includes options to create Eclipse project
files
and to configure the Eclipse workspace, which I think is very useful,
specially for new developers:
./make.py --eclipse-projects
That creates the .project and .classpath files for Eclipse and adds
the
required variables to the workspace, so after that all what the
developer has to do to start working is open Eclipse and import the
projects.
The patch is not yet ready to merge, as there are a few missing
things,
like changing the ownership of files when running as root, but I
would
appreciate if you can test it.
Alon Juan,
as a one-time setup which can auto-generate /local/ configuration
based on target=`pwd`, this is something which makes sense, assuming
engine-config and other utils will work. Otherwise this is meaningless.
Additionally I'd expect engine developers to be able to perform
mvn clean install -Pdep after the one-time setup and be able to
work as they used to.
Additionally and just as important
[Engine-devel] Java code formatting
Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Java code formatting
- Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:17:18 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:07:53 PM Subject: [Engine-devel] Java code formatting Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 Hi, These automatic conversions are not better than current state, also I don't think that this is that important. If you want machine written code, then also provide commit hook to reformat anything, and probably machines to read it. I, personally, think that this change over the sources I manage did not do any good. Regards, Alon Alon, there's a formatting convention for the project set long ago. If you feel it needs to be fixed, go ahead and suggest a fix for the xml. Otherwise we end up in the current chaos, where every 2nd or 3rd patch carries unneeded changes. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Java code formatting
- Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:33:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:28:15 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:17:18 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:07:53 PM Subject: [Engine-devel] Java code formatting Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 Hi, These automatic conversions are not better than current state, also I don't think that this is that important. If you want machine written code, then also provide commit hook to reformat anything, and probably machines to read it. I, personally, think that this change over the sources I manage did not do any good. Regards, Alon Alon, there's a formatting convention for the project set long ago. If you feel it needs to be fixed, go ahead and suggest a fix for the xml. Otherwise we end up in the current chaos, where every 2nd or 3rd patch carries unneeded changes. What do you mean unneeded changes? how do you prevent this in future? Alon Unneeded changes is when you get one line of code fixed due to a bug, and many others re-indented. Best prevention is if people would make sure to use the same conventions. We also have a checkstyle which monitors important issues such as trailing white spaces, localization, etc. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Java code formatting
- Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:53:49 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:48:22 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:33:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:28:15 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:17:18 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:07:53 PM Subject: [Engine-devel] Java code formatting Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 Hi, These automatic conversions are not better than current state, also I don't think that this is that important. If you want machine written code, then also provide commit hook to reformat anything, and probably machines to read it. I, personally, think that this change over the sources I manage did not do any good. Regards, Alon Alon, there's a formatting convention for the project set long ago. If you feel it needs to be fixed, go ahead and suggest a fix for the xml. Otherwise we end up in the current chaos, where every 2nd or 3rd patch carries unneeded changes. What do you mean unneeded changes? how do you prevent this in future? Alon Unneeded changes is when you get one line of code fixed due to a bug, and many others re-indented. Best prevention is if people would make sure to use the same conventions. We also have a checkstyle which monitors important issues such as trailing white spaces, localization, etc. But how do you prevent this in future, not all working in same editor nor same styles. You can say that once in a while you perform cross over auto re-format... I am not native Java programmer but this sounds very strange thing to do, I don't think I know of any project that does that. The main problem is that people commit stuff they don't touch due to their editor behavior, which tread the whole source as if it was at its disposal, while cation should be taken not to modify extra stuff. So maybe just to reject patches that touches lines which are not belong to the patch it-self. Alon I'm fine with rejecting patches with irrelevant changes. Still sometimes you get new files and this repeats itself. The whole point of a convention is that people keep it. Most major IDEs today can use the same formatting XML we now have. So it's really a matter of self-discipline. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Java code formatting
- Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:05:09 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 5:00:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:53:49 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:48:22 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:33:20 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:28:15 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:17:18 PM Subject: Re: [Engine-devel] Java code formatting - Original Message - From: Doron Fediuck dfedi...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 4:07:53 PM Subject: [Engine-devel] Java code formatting Hi, Recently I saw many patches with multiple code re-formatting. When looking into it, we saw that many people didn't use the project policy, and now we have many files with bad formatting. So I just posted a big ugly fix for this[1], and hopefully if accepted people should start using the right conventions and reduce the amount of non-relevant changes we see in the patches. I'm aware of the fact that this may create some issues when porting patches, but better sooner than later. Doron. [1] http://gerrit.ovirt.org/#/c/10541/1 Hi, These automatic conversions are not better than current state, also I don't think that this is that important. If you want machine written code, then also provide commit hook to reformat anything, and probably machines to read it. I, personally, think that this change over the sources I manage did not do any good. Regards, Alon Alon, there's a formatting convention for the project set long ago. If you feel it needs to be fixed, go ahead and suggest a fix for the xml. Otherwise we end up in the current chaos, where every 2nd or 3rd patch carries unneeded changes. What do you mean unneeded changes? how do you prevent this in future? Alon Unneeded changes is when you get one line of code fixed due to a bug, and many others re-indented. Best prevention is if people would make sure to use the same conventions. We also have a checkstyle which monitors important issues such as trailing white spaces, localization, etc. But how do you prevent this in future, not all working in same editor nor same styles. You can say that once in a while you perform cross over auto re-format... I am not native Java programmer but this sounds very strange thing to do, I don't think I know of any project that does that. The main problem is that people commit stuff they don't touch due to their editor behavior, which tread the whole source as if it was at its disposal, while cation should be taken not to modify extra stuff. So maybe just to reject patches that touches lines which are not belong to the patch it-self. Alon I'm fine with rejecting patches with irrelevant changes. Still sometimes you get new files and this repeats itself. The whole point of a convention is that people keep it. Most major
Re: [Engine-devel] Engine local configuration
- Original Message - From: Alon Bar-Lev alo...@redhat.com To: Livnat Peer lp...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 9:30:20 AM Subject: Re: [Engine-devel] Engine local configuration - Original Message - From: Livnat Peer lp...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Juan Hernandez jhern...@redhat.com, engine-devel@ovirt.org Sent: Tuesday, January 1, 2013 9:05:03 AM Subject: Re: [Engine-devel] Engine local configuration On 12/31/2012 05:05 PM, Doron Fediuck wrote: - Original Message - From: Juan Hernandez jhern...@redhat.com To: Roy Golan rgo...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, December 14, 2012 1:25:57 PM Subject: Re: [Engine-devel] Error on starting webadmin On 12/13/2012 03:55 PM, Roy Golan wrote: On 12/13/2012 04:49 PM, Vojtech Szocs wrote: this is making the contribution process more complex. lets think of a lighter way to get a developing setup. I agree, I just wanted to have the local Engine configuration steps documented for reference.. If there's a better way to do it, I'm for it. Vojtech having LocalConfig look for engine.conf.defaults system property before fallback to System.getenv(ENGINE_DEFAULTS); + concatenating -Dengine.conf.defaults=$HOME/.engine.conf.defaults to JAVA_OPTS on standalone.conf How is the system property simpler than the environment variable? I agree that this makes the development process a bit more complex at the moment, but I think that the way to make it simpler is not to continue adding things to standalone.conf. I think that we should move towards a development environment that is closer to the production environment, not the other way around. Ideally the developer should be able to do something like make install to have the engine deployed to a directory structure similar to what he have in the production environment. Then you should be able to go to the bin directory inside that structure and start the engine (and the other tools) using the same script that we use in production environments. If we achieve this goal then we have a simple development environment setup and also we have all developers testing almost the same thing that will go into the production environments. At the moment we don't have that, most times you are testing something quite different (in terms of directory structure, configuration, etc) to what will be installed in production environments. I am working in that direction. Hi Guys, Sorry to resume this discussion, but I find the current situation unfriendly to most developers. I understand there's a need for specific configurations, but it seems to me that this has taken one step too far for most developers. Further more, I expect to see such fundamental concepts being initially discussed here, and not settle with a technical ack, only to be a part of a thread called: Error on starting webadmin. In this context I expect the verified flag to mean This was discussed and verified with contributers in the relevant list. Hi Doron, Thanks for bringing this on the list, I agree with everything you wrote and could not put it any better myself. I configured an environment from scratch yesterday and the additional step to have this config file in /etc does not feel right, not to mentioned that this is not documented in the wiki installation page. I think one of the guidelines we kept so far for setting a development environment and making it as easy as possible for new developers is that no manual step is needed on top of using the setup profile and this definitely breaks this assumption (at least with the way it is handled today). I strongly reject to have default suitable for DEVELOPMENT mode. This was indeed the situation in the past, and may (and have) cause DEVELOPMENT setting to leak into PRODUCTION. In development mode, the developer should explicitly state that he want to use DEVELOPMENT settings either by configuration or by environment, this way no DEVELOPMENT settings will effect intended PRODUCTION settings. Which can easily be handled when thinking of the difference between (git clone + mvn -Psetup,dep) VS (yum install). The first one can simply work. The latter in handled by the RPM. No need to impose additional burdens on developers which are already handled for production setups. Think of user management, PKI infra, etc. All handled in production, and not needed for the basic (git clone + mvn -Psetup,dep). My use case is exactly what Juan describes, but I think the resolution should be different. ie- contributers should be able to git clone and once they setup jboss
Re: [Engine-devel] [vdsm] CPU Overcommit Feature
- Original Message - From: Dan Kenigsberg dan...@redhat.com To: Itamar Heim ih...@redhat.com Cc: engine-devel engine-devel@ovirt.org, vdsm-de...@fedorahosted.org Sent: Thursday, December 20, 2012 11:55:10 AM Subject: Re: [Engine-devel] [vdsm] CPU Overcommit Feature On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote: On 12/20/2012 09:43 AM, Dan Kenigsberg wrote: On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Greg Padgett gpadg...@redhat.com Cc: engine-devel engine-devel@ovirt.org, vdsm-de...@fedorahosted.org Sent: Wednesday, December 19, 2012 3:59:11 PM Subject: Re: [Engine-devel] CPU Overcommit Feature On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote: Hi, I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation. This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit I've commented about the vdsm/engine API on http://gerrit.ovirt.org/#/c/10144/ but it is probably better to reiterate it here. The suggested API is tightly coupled with an ugly hack we pushed to vdsm in order not to solve the issue properly on the first strike. If we had not have report_host_threads_as_cores, I think we'd have a simpler API reporting only cpuThreads and cpuCores; with no funny boolean flags. Let us strive to that position as much as we can. How about asking whoever used report_host_threads_as_cores to unset it once they install Engine 3.2 ? I think that these are very few people, that would not mind this very much. If this is impossible, I'd add a cpuCores2, always reporting the true number, to be used by new Engines. We may even report it only on the very few cases of report_host_threads_as_cores being set. Dan. Hi Dan, Thanks for the review. I agree simply reporting cores and threads would be the right solution. However, when you have hyperthreading turned off you get cores=threads. This is the same situation you have when hyperthreading turned on, and someone used the vdsm configuration of reporting threads as cores. So the engine won't know the real status of the host. This is not surprising, as report_host_threads_as_cores means in blunt English lie to Engine about the number of cores. The newly suggested flag says don't believe what I said in cpuCores, since I'm lying. Next thing we'd have is another flag saying that the former flag was a lie, and cpuCores is actually trustworthy. Instead of dancing this dance, I suggest to stop lying. report_host_threads_as_cores was a hack to assist a older Engine versions. Engine users that needed it had to set it out-of-band on their hosts. Now if they upgrade their Engine, they can -- as easily -- reset that value. If they forget, nothing devastating happens beyond Engine assuming that hyperthreading is off. Please consider this suggestion. I find it the simplest for all involved parties. the only problem is the new vdsm doesn't know which engine may be using it. if engine would say getVdsCaps engineVersion=3.2, then vdsm could know engine no longer needs lying to and ignore the flag, re-using same field. Note that I do not suggest to drop report_host_threads_as_cores now. I am suggesting to keep on lying even to new Engine. If someone thinks that lying is bad, he should reset report_host_threads_as_cores. It seems to me that the suggested API is being coerced by a very limited use case, that is not going to be really harmed by a straight-forward API. Dan. Dan, Did some further checking, and we can go with it; So basically now we add cpuThreads. Additionally, if the report_host_threads_as_cores is turned on, an additional cpuCoresReal will be reported. Since this is a 3.2 cluster feature, the engine will make sure to update the DB with the real numbers, and older clusters will use the information vdsm reports, while blocking the engine side optimization. An update path is already available here: http://gerrit.ovirt.org/#/c/10144/4 I hope that for cluster 3.3 we'll be able to drop the vdsm configuration. A review would be highly appreciated due to the holidays proximity. Doron ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] CPU Overcommit Feature
- Original Message - From: Dennis Jacobfeuerborn denni...@conversis.de To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org, Andrew Cathrow acath...@redhat.com Sent: Wednesday, December 19, 2012 1:49:24 PM Subject: Re: [Engine-devel] CPU Overcommit Feature On 12/18/2012 07:33 PM, Doron Fediuck wrote: - Original Message - From: Dennis Jacobfeuerborn denni...@conversis.de To: Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 7:59:26 PM Subject: Re: [Engine-devel] CPU Overcommit Feature On 12/18/2012 06:33 PM, Andrew Cathrow wrote: - Original Message - From: Dennis Jacobfeuerborn denni...@conversis.de To: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 12:30:34 PM Subject: Re: [Engine-devel] CPU Overcommit Feature On 12/17/2012 07:13 PM, Simon Grinberg wrote: - Original Message - From: Greg Padgett gpadg...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature Hi, I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation. This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit Basically looking good. Hyperthread though is vendor specific. For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that. in libvirt if I read it right it's attribute name='thread_siblings' So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context. In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere? This is about the maximum number of vCPUs we can give to a VM. If the machine has 32 Physical cores that are hyperthreaded then do we say the max number of vCPUs for a single VM is 32 or 64. If the actual scheduling of vCPUs cannot distinguish between threads and cores then why would you even want to limit yourself to 32 in you example? In that case the scheduling might end up being inefficient no matter how many vCPUs you assign to a guest so why restrict the user? (You might of course want to limit the user for policy reasons but that has nothing to to with the thread/core topic.) On the other hand if KVM does only schedule the vCPUs on distinct cores and extending the count from 32 to 64 implies that this distinction is to be disabled then this will have a performance impact for the guest. In that case I might want to limit the guests to just the 32 physical cores so two vCPUs of a single guest don't get scheduled on two threads of the same core. I've never really looked that closely into the scheduling issue but it might play a role here so I asked if someone had any pointers to infos about how exactly KVM makes its scheduling decisions. Regards, Dennis Dennis, first of all every virtual cpu is basically a qemu-thread which can run on any cpu-thread. The scheduling is done by the kernel scheduler, while we may control it using cpu pinning. ie- you may ask for specific vcpu to run on a specific thread which is from the OS point of view another core. Indeed there are cases where this is not recommended, but other cases where this will actually give you a performance boost, as L1 cache is being shared by the sibling threads. So it's really up to you to test your workload and decide id you wish to utilize it or not. We're giving you powerful tools, and you can decide if and how to use it. What I'm trying to get at is this: Isn't the Count threads as physical cores setting superfluous? If HT is disabled on the node this doesn't do anything anyway but if it is enabled what is to be gained by disabling this option? As far as I can see this makes the UI more complicated for no apparent reason. Regards, Dennis Hi Dennis. Let's take it one at a time; UI wise, this is a cluster level policy, and falls into the optimization tab. So it shouldn't
Re: [Engine-devel] CPU Overcommit Feature
- Original Message - From: Dennis Jacobfeuerborn denni...@conversis.de To: Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 7:59:26 PM Subject: Re: [Engine-devel] CPU Overcommit Feature On 12/18/2012 06:33 PM, Andrew Cathrow wrote: - Original Message - From: Dennis Jacobfeuerborn denni...@conversis.de To: engine-devel@ovirt.org Sent: Tuesday, December 18, 2012 12:30:34 PM Subject: Re: [Engine-devel] CPU Overcommit Feature On 12/17/2012 07:13 PM, Simon Grinberg wrote: - Original Message - From: Greg Padgett gpadg...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature Hi, I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation. This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit Basically looking good. Hyperthread though is vendor specific. For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that. in libvirt if I read it right it's attribute name='thread_siblings' So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context. In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread Does this affect only the number of potential vCpus for the guests or does this also have an impact on the actual scheduling? So far I always disabled HT out of fear that a 2 vCpu guest might actually be scheduled to run in 2 threads of the same core but now I'm not so sure anymore. In the HT case does KVM know that two threads belong to the same core and will it only schedule its vCpus on distinct cores? Is there some documentation about this somewhere? This is about the maximum number of vCPUs we can give to a VM. If the machine has 32 Physical cores that are hyperthreaded then do we say the max number of vCPUs for a single VM is 32 or 64. If the actual scheduling of vCPUs cannot distinguish between threads and cores then why would you even want to limit yourself to 32 in you example? In that case the scheduling might end up being inefficient no matter how many vCPUs you assign to a guest so why restrict the user? (You might of course want to limit the user for policy reasons but that has nothing to to with the thread/core topic.) On the other hand if KVM does only schedule the vCPUs on distinct cores and extending the count from 32 to 64 implies that this distinction is to be disabled then this will have a performance impact for the guest. In that case I might want to limit the guests to just the 32 physical cores so two vCPUs of a single guest don't get scheduled on two threads of the same core. I've never really looked that closely into the scheduling issue but it might play a role here so I asked if someone had any pointers to infos about how exactly KVM makes its scheduling decisions. Regards, Dennis Dennis, first of all every virtual cpu is basically a qemu-thread which can run on any cpu-thread. The scheduling is done by the kernel scheduler, while we may control it using cpu pinning. ie- you may ask for specific vcpu to run on a specific thread which is from the OS point of view another core. Indeed there are cases where this is not recommended, but other cases where this will actually give you a performance boost, as L1 cache is being shared by the sibling threads. So it's really up to you to test your workload and decide id you wish to utilize it or not. We're giving you powerful tools, and you can decide if and how to use it. Doron ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the feature spec. Anyone against it? Laszlo I'm a bit against it. I don't see why it's that complicated: Allow migration - use 'host-model' Do not allow - use 'host-passthrough'. So the actual cpu capabilities would depend not only on the 'host cpu' checkbox, but also on the 'pinned to host' checkbox. I think this logical trick would be both funny and confusing from the user's perspective. The reason of why we need host-passthrough is that otherwise I suspect we depend on libvirt for newer features to be somehow exposed to the guest (not sure about it). Yes, with other words: this is a tuning feature. Let's keep it simple. 1. Please remove AllowMigrateCPUHost. No reason for us to do what libvirt is asking to void. 2. host-passthrough should be available only for non migratable VMs. Y.
Re: [Engine-devel] host cpu feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 6:41:24 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 5:33:35 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would
Re: [Engine-devel] host cpu feature
- Original Message - From: Yaniv Kaul yk...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 6:48:33 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 06:46 PM, Doron Fediuck wrote: - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 6:41:24 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 5:33:35 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from
Re: [Engine-devel] host cpu feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 7:14:46 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 6:10:55 PM Subject: Re: [Engine-devel] host cpu feature Alternative idea, inspired by Thus, if you hit any bugs, you are on your own (http://libvirt.org/formatdomain.html#elementsCPU wrt 'host-passthrough'): A config option to determine if we use host-model or host-passthrough. Y. I do not think the engine should go to this level. ie- it can ask for passthrough as a feature, and the actual implementation is handled by vdsm. If vdsm decides over host-passthrough or host-model, then how will the engine user know what exactly his VM gets. I think vdsm must be told exactly what to do. VDSM maintains some level of independence. This is why it the engine should be able to ask for passthrough as a feature. Otherwize vdsm will handle it. So for now I suggest we stick to passthrough only, and if we get an RFE for advanced mode we'll support the host model. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] MOM [DR for 3.2] Host Power Management Proxy Preferences
- Original Message - From: Eli Mesika emes...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Tuesday, November 13, 2012 6:18:27 PM Subject: [Engine-devel] MOM [DR for 3.2] Host Power Management Proxy Preferences Hi DR MOM : http://wiki.ovirt.org/wiki/Talk:HostPMProxyPreferences Requirements Page : http://wiki.ovirt.org/wiki/Features/HostPMProxyPreferences DR: http://wiki.ovirt.org/wiki/Features/Design/DetailedHostPMProxyPreferences ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel Eli, MOM is an ovirt project: http://wiki.ovirt.org/wiki/Project_Proposal_-_MOM Can you please re-phrase the relevant strings? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Proposal to add Allon Mureinik as maintainer to engine core
I'd like to add Allon is doing a very good reviewing job, giving constructive reviews alongside relevant explanations. So for this and everything Itamar mentioned, Allon gets my +1. - Original Message - From: Yair Zaslavsky yzasl...@redhat.com To: Itamar Heim ih...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, November 14, 2012 6:16:52 AM Subject: Re: [Engine-devel] Proposal to add Allon Mureinik as maintainer to engine core +1 - Original Message - From: Itamar Heim ih...@redhat.com To: engine-devel@ovirt.org Sent: Wednesday, November 14, 2012 6:12:09 AM Subject: [Engine-devel] Proposal to add Allon Mureinik as maintainer to engine core Allon has worked on oVirt for the past 10 months. With 534 patches ranging from cleanups, to complex features like user level api and storage live migration. In addition, Allon has been instrumental in the number of patch reviews he has done. I'd like to propose Allon as a maintainer of engine core. Thanks, Itamar ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Question about hasMemoryToRunVM() in RunVmCommandBase
Hi Ly, So basically just to be clear, indeed memory_over_commit is the solution for memory over commitment. Changing it enabled you to run the amount of VMs you need. - Original Message - From: ly pan ply...@gmail.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, October 31, 2012 12:32:48 PM Subject: Re: [Engine-devel] Question about hasMemoryToRunVM() in RunVmCommandBase Hi Doron, Sorry for the so late reply. These days I looked into the code again and found it's my setting not rightly configured. 1. You opinion about reserved_mem is right. 2. Yes, pending_vmem_size is 0 before the 84th runs. 3. memory_over_commit is where my problem is. In webadmin UI, I set the cluster's memory optimization to none, thus memory_over_commit is 100 in DB(the dafault value is 100 in my env). when I set the cluster's mem_opt to 'for server load', memory_over_commit grows to 150 and when I set to 'for desktop load', memory_over_commit grows to 200. Both 'for server load' and 'for desktop load' have let the vdsMemLimit grows dramaticaly, thus the 84th vm managed to run. So I realized the memory optimization option in cluster is for KSM function. Thank you. ly pan 2012/10/24 Doron Fediuck dfedi...@redhat.com: - Original Message - From: ly pan ply...@gmail.com To: engine-devel@ovirt.org Sent: Thursday, October 18, 2012 2:43:35 PM Subject: [Engine-devel] Question about hasMemoryToRunVM() in RunVmCommandBase Hi: This is my test environment: hardware: Dell PowerEdge 2710 ,Memory 48G Software: OVirt engine 3.0 ,VDSM 4.9.4 ,kernel 2.6.32-279.2.1.el6.x86_64 I create 100 vms from pool(each vm has 512M memory and has 1M memory garanteed, with guest_overhead = 65 and reserved_mem = 256), but only 83 vms turn out to run. When I run the 84th vm, engine says there is not enough memory. However at this time KSM is enabled and 15G free memory is still left on the host. I checked the code and found out that before running a vm, the function hasMemoryToRunVM() in RunVmCommandBase would decide whether there is enough memory in the selected host to run it. The Algorithm in the function is like this: mem_avaliable = mem_required = curVds.getmem_commited() + curVds.getpending_vmem_size() + curVds.getguest_overhead() + curVds.getreserved_mem() + vm.getMinAllocatedMem(); And here is my question: curVds.getmem_commited() is caculated from the memory of vm assigned by user. But when the host is running with KSM enabled, the actual memory of each running vm would be smaller than the memory the user assigned to. In this case, the following may occur: engine says there be no more memroy to run any vm while the host has much free space to run more vms(in my case, 15G). Therefore I think the actual memory size reported by vdsm(The KSM shared memory) should be taken into account when calculating curVds.getmem_commited(). Any ideas are welcome. Hi Ly, Sorry for the delay. I'd like to look into this issue. In the meanwhile a few relevant points; 1. You're saying that reserved_mem=256. This is not accurate, since it's reported by vds as host_mem_reserve(256) + extra_mem_reserve(65) = 321 2. I'm assuming you retried running the 84th vm, just to make sure your pending_vmem_size is 0 once all 83 VMs are running. 3. So assuming (2) is correct, I'd like to ask what's the max_vds_memory_over_commit you have for the cluster? By default it's set to 120, and in such a case the 84th VM should still be running. Here's a calculation sample based on your data, and the above 3 issues: vdsCurrentMem = 83 * (512M+65) = 47,891 0 65 321 1 === 48,278 vdsMemLimit = 120/100 * 47,321.1328125 // reported by vdsm: 'cat /proc/meminfo | grep MemTotal' / 1024 = 56,785.359375 So vdsCurrentMem = vdsMemLimit should be true unless you changed something such as the MemTotal or the memory_over_commit. You can try and change memory_over_commit let us know how it works. Doron ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Java EE 7 roadmap changes
https://blogs.oracle.com/theaquarium/entry/java_ee_7_roadmap ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4
- Original Message - From: Vojtech Szocs vsz...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, September 4, 2012 5:46:01 PM Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4 Hi Doron, interesting use-case for UI plugins! Existing (standard) WebAdmin dialogs could be made extensible for plugin authors, so that additional UI elements (like text boxes) could be added to dialog UI. On one side, extensible dialogs (like the Edit Cluster Policy dialog you mentioned) would provide a way to register new UI elements, controlled and driven through plugin API. On the other side, the code behind extensible dialogs would collect values from extra UI elements, and handle them as appropriate, e.g. pass them along the standard values when communicating with the backend. I think we can add this to post-PoC-rev5 feature list. Cheers, Vojtech Excellent, Vojtech. I'll monitor for news in this thread. Thanks! - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Vojtech Szocs vsz...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, September 4, 2012 4:10:19 PM Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4 - Original Message - From: Vojtech Szocs vsz...@redhat.com To: Juan Hernandez jhern...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Tuesday, September 4, 2012 3:04:50 PM Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4 Hi Juan, thanks for your comments :) Server-side components of UI plugin infrastructure (such as PluginSourcePageServlet) indeed need some more work, I agree with your points. I was thinking that PluginSourcePageServlet and FileServlet are quite similar in their purpose, serving static resources from local filesystem. FileServlet is intended for general use, with 'file' parameter configured as servlet init-param. For example, FileServlet could be used to serve static resources from /usr/share/ovirt-engine/ui-plugins: servlet servlet-namepluginResourceServlet/servlet-name servlet-classorg.ovirt.engine.core.FileServlet/servlet-class init-param param-namefile/param-name param-value/usr/share/ovirt-engine/ui-plugins/param-value /init-param /servlet servlet-mapping servlet-namepluginResourceServlet/servlet-name url-pattern/plugins/*/url-pattern /servlet-mapping Assuming following directory convention for UI plugin descriptors and actual plugin resources: /usr/share/ovirt-engine/ui-plugins/foo.json - Plugin descriptor /usr/share/ovirt-engine/ui-plugins/foo/start.html - Plugin host page /usr/share/ovirt-engine/ui-plugins/foo/foo.js - Actual plugin code (referenced by plugin host page) Such servlet could be used to map http(s)://EngineHost:8700/plugins/foo/start.html to /usr/share/ovirt-engine/ui-plugins/foo/start.html (note that FileServlet is in root WAR context) The purpose of PluginSourcePageServlet is very similar, but in terms of FileServlet, the 'file' parameter is not static (defined in web.xml as init-param), but depends on plugin meta-data (defined in foo.json) for each plugin. PluginSourcePageServlet was meant to map http(s)://EngineHost:8700/webadmin/webadmin/plugin/foo/start.html to /custom/plugin/base/directory/start.html (note that PluginSourcePageServlet is in WebAdmin WAR context) Juan - do you think we could modify/reuse FileServlet for serving UI plugin static resources? As mentioned above, the only difference is that the 'file' parameter (base directory) would be potentially different for each plugin. Please let me know what you think about it. Thanks, Vojtech - Original Message - From: Juan Hernandez jhern...@redhat.com To: Vojtech Szocs vsz...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Thursday, August 30, 2012 8:24:02 PM Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4 Nice work Vojtech, just some comments about the PluginSourcePageServlet: * You can avoid the hardcoded plugin code location with something like this: import org.ovirt.engine.core.utils.LocalConfig; File dataDir = LocalConfig.getInstance().getUsrDir(); File pluginCodeLocation = new File(etcDir, ui-plugins); That will result in /usr/share/ovirt-engine/ui-plugins or whatever directory is configured in the ENGINE_USR parameter in the /etc/sysconfig/ovirt-engine file. * It is very important to check the sanity of the value of the plugin parameter, otherwise an attacker could send you a name with backpaths, and that can result in accessing an unexpected file. In this particular case you are adding the .js extension, so it won't probably result in accessing dangerous files, but anyhow it is a good practice. I
Re: [Engine-devel] Gluster IPTable configuration
- Original Message - From: Andrew Cathrow acath...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: David Jaša dj...@redhat.com, engine-devel@ovirt.org Sent: Friday, August 31, 2012 3:12:34 PM Subject: Re: [Engine-devel] Gluster IPTable configuration - Original Message - From: Alon Bar-Lev alo...@redhat.com To: David Jaša dj...@redhat.com Cc: engine-devel@ovirt.org Sent: Friday, August 31, 2012 5:09:47 AM Subject: Re: [Engine-devel] Gluster IPTable configuration - Original Message - From: David Jaša dj...@redhat.com To: engine-devel@ovirt.org Sent: Friday, August 31, 2012 11:57:11 AM Subject: Re: [Engine-devel] Gluster IPTable configuration Alon Bar-Lev píše v Čt 30. 08. 2012 v 14:40 -0400: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: Shireesh Anjal san...@redhat.com, engine-devel@ovirt.org, Selvasundaram sesub...@redhat.com Sent: Thursday, August 30, 2012 9:37:59 PM Subject: Re: [Engine-devel] Gluster IPTable configuration - Original Message - From: Alon Bar-Lev alo...@redhat.com To: Selvasundaram sesub...@redhat.com Cc: Shireesh Anjal san...@redhat.com, engine-devel@ovirt.org Sent: Thursday, August 30, 2012 2:35:16 PM Subject: Re: [Engine-devel] Gluster IPTable configuration - Original Message - From: Selvasundaram sesub...@redhat.com To: engine-devel@ovirt.org Cc: Shireesh Anjal san...@redhat.com Sent: Thursday, August 30, 2012 4:30:16 PM Subject: [Engine-devel] Gluster IPTable configuration Hi, I want to add gluster specific IPTable configuration in addition to the ovirt IPTable configuration (if it is gluster node). There are two approaches, 1. Having one more gluster specific IP table config in db and merge with ovirt IPTable config (merging NOT appending) [I have the patch engine: Gluster specific firewall configurations #7244] 2. Having two different IP Table config (ovirt and ovirt+gluster) and use either one. Please provide your suggestions or improvements on this. Hello all, The mentioned patch[1], adds hard coded gluster code into the bootstrap code, manipulate the firewall configuration to be gluster specific. It hardcoded search for reject, insert before some other rules. I believe this hardcode approach is obsolete now that we have proper tools for templates. A more robust solution would be defining generic profiles, each profile as a template, each template can refer to different profiles, and assign profile to a node. This way the implementation is not gluster [or any] specific and can be reused for more setups, code is cleaner. or create custom chains ? Can you please elaborate what is custom chains? Thanks! iptables -N my_new_chain iptables -A my_new_chain rule_1 iptables -A my_new_chain ... iptables -A my_new_chain rule_n # if this rule is matched, packet goes through rules in my_new_chain iptables -A INPUT rule -j my_new_chain Hello, How does this solve the original issue? It makes it easier for customers who are adding their own IPTables configuration - when we do rhev-h plugins it'll make things easier. This way we don't wipe out the original rules we just add our own chain. Current default (virt) iptables configuration is kept in the engine's DB. So... 1. We can update the DB value to include Gluster configuration. or 2. Have an additional DB value for virt+Gluster. So (1) will work, but when gluster is not relevant we have redundant ports. But my only reservation from (2) is that it's not scalable for additional services. Either way, I agree with Alon that melding the configurations at runtime is unexpected, and shouldn't be used. After some additional thought... we can have (1), and make all gluster entries commented out (let's say with #GLSTR) by default, so it'll be enabled only when gluster is being used. This will allow introducing future configurations as well without the need for additional DB values. Sample snip of DB value: == :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT # vdsm -A INPUT -p tcp --dport 54321 -j ACCEPT # libvirt tls -A INPUT -p tcp --dport 16514 -j ACCEPT # SSH -A INPUT -p tcp --dport 22 -j ACCEPT # guest consoles -A
[Engine-devel] Unifying (parts of) commit templates.
Hi All, It seems that for commit subjects, vdsm is using a general concept of- BZ#??? some message I'd like to suggest adopting it to the engine template we use today- BZ#??? core | restapi | tools | history | engine | userportal | webadmin: short summary under 50 chars This may help us write some scripts which will work both for vdsm and engine BZs. Doron. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Additional info on patch reviewers
Hi, This may prove to be useful sometime... It is possible to ask gerrit to show a reviewer name, along the verify or code review columns. In order to set it, go to the settings link (up-right corner), and choose Preferences. Check the line Display Person Name In Review Category. See the attached for sample and what to set. -- /d This message will self destruct in the future. Or not. attachment: gerrit-v-r-columns.pngattachment: gerrit-settings-fixed.png___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] gerrit 2.3
On 03/06/12 23:04, Itamar Heim wrote: fyi - gerrit has been upgraded to version 2.3. ___ Infra mailing list in...@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra Adding the devel list. Latest gerrit started a concept of _draft_ branches, which should help all [WIP] users. From the notes: Also adds magic branches refs/drafts/ and refs/publish/ that will handle whether or not a patchset is a draft or goes straight to review. refs/for/ should be deprecated in favor of explicitly marking a patchset as a draft or directly to review. I strongly recommend everyone to take a look here: http://gerrit-documentation.googlecode.com/svn/ReleaseNotes/ReleaseNotes-2.3.html#_drafts (as well as the rest of the new features and updates). -- /d Willyoupleasehelpmefixmykeyboard?Thespacebarisbroken! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Enabling guest memory balloon device
Hi All, In the following wiki, there's a design for enabling the balloon device, which is currently disabled in engine setups. Other than enabling the device, this is also a step forward in the path to vdsm and MoM sub-project integration. More details can be found here: http://www.ovirt.org/wiki/Features/Design/memory-balloon P.S. UI mockups should be updated soon. -- /d “Funny,” he intoned funereally, “how just when you think life can't possibly get any worse it suddenly does.” --Douglas Adams, The Hitchhiker's Guide to the Galaxy ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Maven 3 here we come!
Hi all, As discussed last month[1], we had to deal with some issues which turned out to be a Maven bug. Thanks to Juan and Asaf's work, our current sources now build properly using Maven 3. So you're all invited to migrate into Maven 3. Other than upgrading your local maven package no other action is needed. For now, Maven 2 will also work for you, but I expect in the future we'd like to make use of some advanced features, so migration to 3 is recommended. Talking about advanced features, an interesting challenge is feedback on parallel builds [2]. So whoever wants to try it out and report if it improves run time without breaking anything, will be appreciated. Happy migration! [1]http://lists.ovirt.org/pipermail/arch/2012-April/000490.html [2] https://cwiki.apache.org/MAVEN/parallel-builds-in-maven-3.html -- /d Email returned to sender -- insufficient voltage. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] CPU Pinning @engine
On 21/05/12 12:40, Doron Fediuck wrote: On 21/05/12 12:18, Yaniv Kaul wrote: On 05/21/2012 11:53 AM, Doron Fediuck wrote: On 21/05/12 11:16, Yaniv Kaul wrote: On 05/21/2012 11:08 AM, Doron Fediuck wrote: On 21/05/12 11:01, Dor Laor wrote: On 05/21/2012 10:58 AM, Doron Fediuck wrote: On 21/05/12 01:45, Andrew Cathrow wrote: - Original Message - From: Doron Fediuckdfedi...@redhat.com To: dl...@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine On 20/05/12 21:41, Dor Laor wrote: On 05/17/2012 11:15 PM, Doron Fediuck wrote: On 17/05/12 20:28, Ayal Baron wrote: Live migration will not be supported for such VM's. Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning. I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment Performance may degrade on any migration to loaded host regardless of pinning. There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this. Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2. We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI. We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well. IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope. Safe is letting the user migrate. It may or may not succeed (I'd worry it may put a host into Error state - I hope we've removed this state). This is a pure speculation. If I'm pinning a specific VM to a specific host, and its vCPUS into specific pCPUs, how can live migration be considered a safe thing? If I pinned to host, this is a violation of my request. Now, let's say I didn't pin to host, but I ask for specific vCPUS pinning into specific pCPUs. For now, there's no guarantee the destination host will have a relevant CPU capacity (maybe even topology), so CPU pinning may not work, which is also a violation of my request. Is live migration really the safe thing to do here??? Why not alert me so I could manually migrate to where I think is better? Having that said, I'm reminding you the somewhere earlier in this thread I wrote we'll make the constraint of cpu pinning allowed only on pinned-to-host VMs configurable, so people who wish to walk on wild side, can give it a go. I think the consensus of the thread is that you should allow pin vCPUs to pCPUs even if the VM is not pinned to host. If the user wants to ensure it will not migrate, he should mark the VM as non migratable. Otherwise, it may migrate, and will succeed - if it can meet the same pinning requirements on the destination, or fail - and then no harm done. If migration succeeds without meeting the pinning requirements on the destination, I see this as a libvirt bug. Y. Basically I already said cpu-pin is allowed for migrative VM's once configuring it. So basically what you're saying is that if such migration works, while loosing cpu-pin, it's a libvirt bug. I can live with that. I hope Daniel agrees with you. Design updated, so pinning is not blocked, unless user will block it (pin to host, etc). UI will be updated shortly. Please review- http://www.ovirt.org/wiki/Features/Design/cpu-pinning Sorry is when you wish you had done so initially. ;-) In this case, I disagree as this is configurable. Y. about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues. - Original Message - From: Doron Fediuckdfedi...@redhat.com To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning Please review and comment if needed. Thanks! -- /d Never say OOPS! always say Ah, Interesting
Re: [Engine-devel] CPU Pinning @engine
On 21/05/12 01:45, Andrew Cathrow wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com To: dl...@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine On 20/05/12 21:41, Dor Laor wrote: On 05/17/2012 11:15 PM, Doron Fediuck wrote: On 17/05/12 20:28, Ayal Baron wrote: Live migration will not be supported for such VM's. Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning. I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment Performance may degrade on any migration to loaded host regardless of pinning. There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this. Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2. We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI. We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well. about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues. - Original Message - From: Doron Fediuckdfedi...@redhat.com To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning Please review and comment if needed. Thanks! -- /d Never say OOPS! always say Ah, Interesting! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- /d Never say OOPS! always say Ah, Interesting! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- /d Forty-two, said Deep Thought, with infinite majesty and calm. --Douglas Adams, The Hitchhiker's Guide to the Galaxy ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] CPU Pinning @engine
On 21/05/12 11:01, Dor Laor wrote: On 05/21/2012 10:58 AM, Doron Fediuck wrote: On 21/05/12 01:45, Andrew Cathrow wrote: - Original Message - From: Doron Fediuckdfedi...@redhat.com To: dl...@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine On 20/05/12 21:41, Dor Laor wrote: On 05/17/2012 11:15 PM, Doron Fediuck wrote: On 17/05/12 20:28, Ayal Baron wrote: Live migration will not be supported for such VM's. Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning. I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment Performance may degrade on any migration to loaded host regardless of pinning. There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this. Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2. We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI. We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well. IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope. about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues. - Original Message - From: Doron Fediuckdfedi...@redhat.com To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning Please review and comment if needed. Thanks! -- /d Never say OOPS! always say Ah, Interesting! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- /d Never say OOPS! always say Ah, Interesting! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- /d Air conditioned environment - Do NOT open Windows! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] CPU Pinning @engine
On 21/05/12 12:18, Yaniv Kaul wrote: On 05/21/2012 11:53 AM, Doron Fediuck wrote: On 21/05/12 11:16, Yaniv Kaul wrote: On 05/21/2012 11:08 AM, Doron Fediuck wrote: On 21/05/12 11:01, Dor Laor wrote: On 05/21/2012 10:58 AM, Doron Fediuck wrote: On 21/05/12 01:45, Andrew Cathrow wrote: - Original Message - From: Doron Fediuckdfedi...@redhat.com To: dl...@redhat.com Cc: engine-devel@ovirt.org Sent: Sunday, May 20, 2012 5:49:08 PM Subject: Re: [Engine-devel] CPU Pinning @engine On 20/05/12 21:41, Dor Laor wrote: On 05/17/2012 11:15 PM, Doron Fediuck wrote: On 17/05/12 20:28, Ayal Baron wrote: Live migration will not be supported for such VM's. Migration will work on homogeneous clusters so this should not be enforced (not limited to VMs which are pinned to host) just give a warning. I agree, but if we wish to ping vCPUx to pCPUy in host a, we cannot ensure to have the same in host b, since it may have pCPUy too busy, so performance will degrade. Also, I hope you saw my general comment Performance may degrade on any migration to loaded host regardless of pinning. There is not need to forbid migration. Instead, the SLA assurance policy should verify the dedicated resources and match the target w/ the guarantee and decide according to this. Thanks for the input. We may start with migration blocked in a configurable manner, so people who wish migrate to work will simply set the relevant configuration key. As for looking for a matched CPU, will add it as p2. We shouldn't block migration If a user wants to not migrate a cpu pinned VM then let them use the non-migratable flag in the UI. We're not blocking migration, we're allowing pinning (for now) only for non-migrative VM's. For p2 we'll verify destination host has the relevant pinning capacity, which will allow pinning for migrative VM's as well. IMHO the order should be the opposite - if the user likes to migrate a pinned VM, let him (you can pop some message if you like). I'll check it with UI guys (AFAIK engine UI has no pop-ups). Also, I prefer safe than sorry. So start with a safe working pinning for pinned-to-host VMs, and then push the envelope. Safe is letting the user migrate. It may or may not succeed (I'd worry it may put a host into Error state - I hope we've removed this state). This is a pure speculation. If I'm pinning a specific VM to a specific host, and its vCPUS into specific pCPUs, how can live migration be considered a safe thing? If I pinned to host, this is a violation of my request. Now, let's say I didn't pin to host, but I ask for specific vCPUS pinning into specific pCPUs. For now, there's no guarantee the destination host will have a relevant CPU capacity (maybe even topology), so CPU pinning may not work, which is also a violation of my request. Is live migration really the safe thing to do here??? Why not alert me so I could manually migrate to where I think is better? Having that said, I'm reminding you the somewhere earlier in this thread I wrote we'll make the constraint of cpu pinning allowed only on pinned-to-host VMs configurable, so people who wish to walk on wild side, can give it a go. I think the consensus of the thread is that you should allow pin vCPUs to pCPUs even if the VM is not pinned to host. If the user wants to ensure it will not migrate, he should mark the VM as non migratable. Otherwise, it may migrate, and will succeed - if it can meet the same pinning requirements on the destination, or fail - and then no harm done. If migration succeeds without meeting the pinning requirements on the destination, I see this as a libvirt bug. Y. Basically I already said cpu-pin is allowed for migrative VM's once configuring it. So basically what you're saying is that if such migration works, while loosing cpu-pin, it's a libvirt bug. I can live with that. I hope Daniel agrees with you. Sorry is when you wish you had done so initially. ;-) In this case, I disagree as this is configurable. Y. about starting with a humble solution and gradually improving. In this context (auto)numa can help us, so we better do numa than handle migration for basic mode, risking performance issues. - Original Message - From: Doron Fediuckdfedi...@redhat.com To: engine-devel@ovirt.org Sent: Thursday, May 17, 2012 2:42:46 PM Subject: [Engine-devel] CPU Pinning @engine Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning Please review and comment if needed. Thanks! -- /d Never say OOPS! always say Ah, Interesting! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel
Re: [Engine-devel] Advanced Nfs Options + NFSv4 GUI mockup
On 21/05/12 14:49, Einav Cohen wrote: - Original Message - From: Geert Jansen gjan...@redhat.com Sent: Monday, May 21, 2012 2:44:09 PM On 05/21/2012 12:44 PM, Einav Cohen wrote: i was wondering why you have pulled out retrans and timeout? Are these the most important ones? Sorry - not sure I got you. Can you please clarify your question? Sorry, I meant, why do retrans and timeout have their own input boxes, while the other options need to be set via the 'mount options' box. There's quite a few other important nfs options like tcp, rsize, wsize, and possibly nointr and hard. I was wondering why we took these specific two and gave them their separate boxes but not the other ones. Retrans and Timeout got separate boxes since they have separate parameters in the api (GUI was determined by api). Ayal - any reason for these specific two values having parameters of their own? Reading some of the thread and reviews I can say that these 3 options (version, retrans and timeo) are advanced NFS features, which in most cases should not be used at all. VDSM has very good defaults, and in some specific cases there's a need to change these defaults. Only in such case these options should be modified. Regards, Geert ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- /d Hi, my name is Any Key. Please don't hit me! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Advanced Nfs Options + NFSv4 GUI mockup
On 21/05/12 16:09, Einav Cohen wrote: - Original Message - From: Doron Fediuck dfedi...@redhat.com Sent: Monday, May 21, 2012 3:59:07 PM On 21/05/12 14:49, Einav Cohen wrote: - Original Message - From: Geert Jansen gjan...@redhat.com Sent: Monday, May 21, 2012 2:44:09 PM On 05/21/2012 12:44 PM, Einav Cohen wrote: i was wondering why you have pulled out retrans and timeout? Are these the most important ones? Sorry - not sure I got you. Can you please clarify your question? Sorry, I meant, why do retrans and timeout have their own input boxes, while the other options need to be set via the 'mount options' box. There's quite a few other important nfs options like tcp, rsize, wsize, and possibly nointr and hard. I was wondering why we took these specific two and gave them their separate boxes but not the other ones. Retrans and Timeout got separate boxes since they have separate parameters in the api (GUI was determined by api). Ayal - any reason for these specific two values having parameters of their own? Reading some of the thread and reviews I can say that these 3 options (version, retrans and timeo) are advanced NFS features, which in most cases should not be used at all. VDSM has very good defaults, and in some specific cases there's a need to change these defaults. Only in such case these options should be modified. No question that vdsm has very good defaults and only rarely should they be modified. However, why not change everything via mount options? why do we have separate parameters specifically for retrans and timeout (in addition to mount options)? As Geert said, we have also tcp, rsize, wsize, etc. So why retrans and timeout got parameters of their own and the others didn't (i.e. if there is a need to change tcp, rsize, wsize, the only way to do it is via the generic mount options)? I think it has to do w/ the vdsm implementation of it, maybe also connection management in the backend. But I think Livnat may have some better understandings of that. Regards, Geert ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- /d Hi, my name is Any Key. Please don't hit me! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- /d Why doesn't DOS ever say EXCELLENT command or filename! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] CPU Pinning @engine
Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning Please review and comment if needed. Thanks! -- /d Never say OOPS! always say Ah, Interesting! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] CPU Pinning @engine
On 17/05/12 17:38, Andrew Cathrow wrote: - Original Message - From: Dor Laor dl...@redhat.com To: gkot...@redhat.com Cc: engine-devel@ovirt.org, Eduardo Habkost ehabk...@redhat.com, Gleb Natapov g...@redhat.com Sent: Thursday, May 17, 2012 10:04:10 AM Subject: Re: [Engine-devel] CPU Pinning @engine On 05/17/2012 04:24 PM, Gary Kotton wrote: On 05/17/2012 02:42 PM, Doron Fediuck wrote: Hi All, Currently the VDSM has a CPU pinning hook. We'd like to add better support of it into the engine itself. Here's a design draft to cover it: http://www.ovirt.org/wiki/Features/Design/cpu-pinning Please review and comment if needed. Hi, I have a number of comments: 1. Will the user have any indication for the maximum amount of vCPU's? 2. What happens if a VM is already configured with the same CPU pin info? Will this be shared? Yes 3. Would it possible to extend this to limit the MHz of CPU that can be allocated to a VM? No, that's not supported today in KVM. In the future we should use cgroups to provide prioritization as part of the future SLA feature. 5. If the VM has more than one core, will each core have to be mapped? yes 6. similar to the existing hook Shahar wrote - can you please put a link the the hook that Shahar wrote. :) This will be helpful for those of us unfamiliar with it. http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=tree;f=vdsm_hooks/pincpu;h=fd7a1a85912504d6b0cc8ae4e8123d71bf3c2741;hb=HEAD But basically it's the same syntax as libvirt uses today. 7. Why will live migration not be supported? What is the gap to have this type of support? There's been disagreement here. The argument against it is that if you have heterogeneous hardware then it might not work. eg. If I pin on vCPU 10-11 on host A and and host B only has 8 CPUs I'd argue that if an admin is doing the pinning then it's up to him to make sure he picks a sensible value +1 on the above. In addition, it should be nice to reveal the host cpu topology so the user will be able to match the virtual one. Getting the hyperthread topology is important factor as well. Numa is not mentioned at all. It should be highly desirable to expose the numa topology and add numa pinning using numactl. Since you're refactoring this code, adding options for the new numda daemon would be important too. Another important factor is the ability to pin not only the vcpu threads but also to pin 'service threads' in the host, potential to other physical cpus. Examples for such are the qemu io thread, vhost threads, etc. Regards, Dor Thanks Gary Thanks! +1 on Andrew's answers. Just want to add a general comment- CPU tuning can evolve into something which is much bigger. I choose to start with a humble pinning support, which I know is already being used today using vdsm hook. Having it in the engine (even in a basic mode) will be more friendly and helpful to many users. Going forward we have a lot of grounds to cover in this area, including NUMA and other exciting features, which we'll gradually introduce as the community allows. -- /d All computers wait at the same speed. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [vdsm] RFD: NEW API getAllTasks
On 07/05/12 21:33, Adam Litke wrote: The current APIs for retrieving all task information do not actually return all task information. I would like to introduce a new API that corrects this and other issues with the current API while preserving backwards compatibility with ovirt-engine for as long as is necessary. The current APIs: getAllTasksInfo(spUUID=None, options = None): - Returns a dictionary that maps a task UUID to a task verb. - Despite having 'all' in the name, this API only returns tasks that have an 'spm' tag. - This call returns only one piece of information for each task. - The spUUID parameter is deprecated and ignored. getAllTasksStatuses(spUUID=None, options = None): - Returns a dictionary of task status information. - Despite having 'all' in the name, this API only returns tasks that have an 'spm' tag. - The spUUID parameter is deprecated and ignored. I propose the following new API: getAllTasks(tag=None, options=None): - Returns a dictionary of task information. The info from both of the above functions would be merged into a single result set. - If tag is None, all tasks are returned. otherwise, only tasks matching the tag are returned. - The spUUID parameter is dropped. options is for future extension and is currently not used. This new API includes all functionality that is available in the old calls. In the future, ovirt-engine could switch to this API and preserve the current semantics by passing tag='spm' to getAllTasks. Meanwhile, API users that really want all tasks (gluster and the REST API) can get what they need. Thoughts on this idea? (Adding engine-devel, as this relates to the engine API). AFAIR, in the original design (when a-sync tasks where introduced into vdsm), most (if not all) of the tasks were SPM tasks, and this is the reason for this behavior. Improving the API is welcomed. The suggested design should work. I'd like to verify: - Backwards compatibility works; so running engine's shouldn't be replaced. Dan: any news on this? - Going forward with potential changes in SPM concepts should be supported as well. Dan/Ayal/Livnat: do you think it works? ie- anything else needed than alternate 'spm' tag? -- /d All computers wait at the same speed. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [vdsm] vds_bootstrap.py 's residency
On 03/05/12 17:56, Shu Ming wrote: Hi, I am checking the VDSM and ovirt-engine workspace for vds_bootstrap.py file. It was found that vds_bootstrap.py file was in VDSM workspace and was packaged into vdsm-bootstrap rpm package. Also, it was found that in the host installation process, host node will try to get the vds_bootstrap.py from the engine server. But vds_bootstrap.py is not included in any engine packages. Does that mean we should install vdsm-bootstrap package into engine server? Why not package this file into engine packages instead of vdsm packages? Hi Shu, good questions ;) Let's take it one at a time: 1. The bootstrap RPM should be installed on the engine's machine. 2. The reason behind this odd behavior is as follows; The concept of bootstrapping, means taking a bare Linux installation, and making a Hypervisor out of it. This process includes (amongst other) several delicate operations, such as validations, network configuration, certificate generation and RPM installation. One of the RPM being installed by the bootstrap script, is vdsm itself, so bootstrap cannot be a part of the vdsm. 3. The reason for keeping the bootstrapping scripts in the vdsm repo, is that bootstrapping prepares the hypervisor for vdsm, so if/when something changes in vdsm, is should be modified in bootstrap as well. Hope it helps, Doron -- /d This message will self destruct in the future. Or not. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Fwd: Checkstyle- Webadmin userportal localization
If you're running simple compilation (build) on each patch, it'll be included. So no need for a special job. On 02/05/12 12:24, Eyal Edri wrote: I believe we can add a checkstyle validation job in jenkins for that using checkstyle plugin[1]. [1] https://wiki.jenkins-ci.org/display/JENKINS/Checkstyle+Plugin It can run on each commit, and soon on each patch sent to gerrit (once we'll boost our Jenkins framework for oVirt). Eyal. - Original Message - From: Alona Kaplan alkap...@redhat.com To: de...@redhat.com, gchap...@redhat.com, ac...@redhat.com, vsz...@redhat.com, tjeli...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, May 2, 2012 11:58:12 AM Subject: [Engine-devel] Fwd: Checkstyle- Webadmin userportal localization Hi, Yesterday some changes were made in oVirt's checkstyle (http://gerrit.ovirt.org/#change,3760) 1. Renaming built-tools to build-tools-root and separating it into two packages: * checksyles- for checkstyle xml files. * ovirt-checkstyle-extension- for adding new (not built-in) checkstyle checks. 2. Adding to the checkstyle.xml a new check- NlsCheck, which fails the compilation in case non-externalized strings appear in the code. [To understand more about the need of this check, see http://gerrit.ovirt.org/#change,3612] From now on, all strings in the web-admin/user-portal java code should be one of the following: - Externalized (in case the string should be localized) - Have a //$NON-NLS-N$ comment next to them (in case the string shouldn't be externalized, e.g. HashName of a component) If there is a non externalized string in a web-admin/user-portal java file and there is no //$NON-NLS-N$ comment next to it, mvn compilation will fail. Note: NlsCheck is currently configured to run only on web-admin/user-portal projects. If you want NlsCheck check to run on another project, please add the following to the configuration of the checkstyle in the project's pom file: propertyExpansionrunNlsCheck=true/propertyExpansion Alona. -- /d Common sense is not so common. --Voltaire, Dictionnaire Philosophique (1764) ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [vdsm] vdsm on openSuSE
On 30/04/12 19:23, Itamar Heim wrote: On 04/30/2012 06:45 PM, Sascha Littel wrote: Am Montag, 30. April 2012, 16:45:12 schrieben Sie: Hi Sasha, This may be an issue of SSH authentication method. Can you please check you SSH server in the host- Password auth should be password and not Keyboard-interactive. This may lead to SSH auth failure as you engine log indicates. Thanks dude this was the hint I need. I changed the PasswordAuthentication in /etc/ssh/sshd_config. Now I can add the vdsm into the oVirt engine host. Now the real work can beginn. Doron - can we catch this error and give this hint to users as something worth checking? (added engine-devel, as this extends to the engine side). AFAICT, we get auth failure, with no reason. In order to handle it we can go in to ways (need to decide)- 1. Add the keyboard-interactive auth to Mina SSHD. There's a guy who added it[a] and we may try and ask for hints from him. I know that patches are welcomed there as well ;) 2. Try to diagnose the failure we get, or scan Mina's err / debug stream. I suspect we should be able to see something like: debug1: Authentications that can continue: password,publickey ... debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password So if server does not report 'password' as an option we could give a better auth-failure message. It will be nice if someone from our community could pick this up, and if not this would be a nice feature for one of the coming versions. [a] http://mail-archives.apache.org/mod_mbox/mina-dev/201112.mbox/%3ccacpdtxmmweqtq+as+fqzwpgxcxday4hzxk0jarvczkyntfw...@mail.gmail.com%3E Am Montag, 30. April 2012, 13:09:25 schrieben Sie: On 04/30/2012 02:07 PM, Sascha Littel wrote: Am Montag, 30. April 2012, 05:04:09 schrieben Sie: On 04/29/2012 10:24 PM, S. Littel wrote: Hi everybody, I'm working currently on a running version of vdsm 4.9.1 for openSuSE 11.3. I'm changing many lines in the start/stop scripts e.g. paths, rc commands. Most of this work looks fine but if I try to get a connection between the oVirt engine (runs on a openSuSE 12.1) and the vdsm host I get a ssl error. Also after setting ssl in vdsm.conf to false and changing the settings in oVirt engine database I still get this error. which settings are you changing in the db? I changed the seetings in the database with this 2 commands: did you restart engine after changing these? Yes. I found this page in the oVirt Wiki: http://ovirt.org/w/index.php?title=OVirt_- _disable_SSL_in_VDSMdiff=3036oldid=prev psql engine -U postgres -c UPDATE vdc_options set option_value = 'false' where option_name = 'SSLEnabled' psql engine -U postgres -c UPDATE vdc_options set option_value = 'false' where option_name = 'UseSecureConnectionWithServers' UseSecureConnectionWithServers? Yes. So the general question, is there someone working on a openSuSE 11.3 or 11.4 version of vdsm? Or someone who has experience how to get it work? Regards Sascha Littel Here is the failure massage from the vdsm-reg.log I get on the vdsm host: SSLError: [Errno 185090050] _ssl.c:328: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib MainThread::DEBUGdeployUtil::1413::root::getRemoteFile end. MainThread::DEBUGdeployUtil::621::root::handleSSHKey start MainThread::ERRORdeployUtil::614::root::restorecon /root/.ssh/authorized_keys failed And this is the failure message from engine.log on the oVirt engine host: ERROR [org.ovirt.engine.core.utils.hostinstall.MinaInstallWrapper] (http--0.0.0.0-8443-1) Could not connect to server xen007.f1.aiges.net: Failed connecting to xen007.f1.aiges.net using given password! Please verify your password is correct and that the host accepts password-based authentication WARN [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8443-1) CanDoAction of action AddVds failed. Reasons:VDS_CANNOT_CONNECT_TO_SERVER,VAR__ACTION __ADD,VAR__TYPE__HOST Regards Sascha Littel -- /d Never say OOPS! always say Ah, Interesting! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Jar versions in ovirt-engine
On 18/04/12 14:04, Juan Hernandez wrote: On 04/18/2012 09:51 AM, Ofer Schreiber wrote: Ever wondered why the version of oVirt's first release is 3.0.0_0001? The answer is simple - We use ovirt-engine jar's version as our main release version. Personally, I think the current versioning scheme is ugly. Actually, I can't name even one open-source project using _ in it's version. What can we do about it? We have couple of options: 1. Leave the engine alone, and use a separate versioning scheme (e.g - use just 3.1.0 as the main version for next release) 2. Remove _ from engine jars 3. Do nothing. I'd like to hear your thoughts, as well as the reasons to use such an unusual versioning scheme. --- Ofer Schreiber oVirt Release Manager ___ Arch mailing list a...@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch From my point of view using the 0001 suffix in the names of the jar files is not a big problem, but I agree that using it in the release number is ugly, and it produces issues/discussions during packaging. I vote for option #1: use 3.1.0 for the next main version. The original versioning scheme was due to a bug in maven2. Juan, I've read some of the Java packaging concepts, but didn't see (or missed) thoughts about modular versioning (ie- artifacts). Here are the things to consider here; - Current RPM's are using the version declared in the POM files. Should this concept remain? * I think it should remain, as other packaging systems should be able to use it as well and end-up is the similar project version. - Do we want to expose oVirt engine artifacts in a Maven repo for others to consume? * If we do, we'll need to make sure we have a scheme that works both for Maven and for packaging (RPM) comparisons. One last thing... I know most of the packaging work now is RPM based. Still, I'm asking you to leave enough room for non-RPM distro's to join in. -- /d Forty-two, said Deep Thought, with infinite majesty and calm. --Douglas Adams, The Hitchhiker's Guide to the Galaxy ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Jar versions in ovirt-engine
On 19/04/12 13:26, Juan Hernandez wrote: On 04/19/2012 12:00 PM, Doron Fediuck wrote: On 18/04/12 14:04, Juan Hernandez wrote: On 04/18/2012 09:51 AM, Ofer Schreiber wrote: Ever wondered why the version of oVirt's first release is 3.0.0_0001? The answer is simple - We use ovirt-engine jar's version as our main release version. Personally, I think the current versioning scheme is ugly. Actually, I can't name even one open-source project using _ in it's version. What can we do about it? We have couple of options: 1. Leave the engine alone, and use a separate versioning scheme (e.g - use just 3.1.0 as the main version for next release) 2. Remove _ from engine jars 3. Do nothing. I'd like to hear your thoughts, as well as the reasons to use such an unusual versioning scheme. --- Ofer Schreiber oVirt Release Manager ___ Arch mailing list a...@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch From my point of view using the 0001 suffix in the names of the jar files is not a big problem, but I agree that using it in the release number is ugly, and it produces issues/discussions during packaging. I vote for option #1: use 3.1.0 for the next main version. The original versioning scheme was due to a bug in maven2. Juan, I've read some of the Java packaging concepts, but didn't see (or missed) thoughts about modular versioning (ie- artifacts). Here are the things to consider here; - Current RPM's are using the version declared in the POM files. Should this concept remain? * I think it should remain, as other packaging systems should be able to use it as well and end-up is the similar project version. I can talk from the Fedora point of view only, as that is what I know a bit. In Fedora there can be only one version of a given jar file installed in the system, so there is no point in adding a version number to the name of that jar file: the version number is already in the package containing that jar file. In fact if the build generates jar files with version numbers in the name the RPM should remove those jar files. That is why I say that having any kind of numbers in the names of the jars is not important: we have to remove them anyway. Packaging guidelines (see [1]) recommend to avoid version numbers in the jar files, and I think that makes sense. This would be the easy solution. What happens when you have more than a single Java app, and both using different versions of the same jar file? This means that one of the app's will need to bring it along and use it locally, rather than system-level usage. I'm guessing if we assume such a constraint the solution will be to force all app's to use latest jar version, which isn't trivial. So some distro's will allow of concept of slotted installation. This means I currently /have/ 2 working versions of postgres in my laptop (using Gentoo)- equery l postgresql-server * Searching for postgresql-server ... [IP-] [ ] dev-db/postgresql-server-8.4.11:8.4 [IP-] [ ] dev-db/postgresql-server-9.1.3:9.1 The same works on my laptop for Maven, Java, Python and many others. If you think about it, Fedora supports slotted installation for kernels, and then added alternatives to do that with other packages as well (mta, Java..). So there's a need and a way to handle several versions of the same library (regardless of the language), and we should be careful when taking such assumptions. At least try to be as flexible as possible, to allow others to join in. So learning from Fedora I'd say- let the RPM install in a versioned folder (ie- /usr/lib/jvm/jre-1.5.0-gcj/..), and leave the jar files without versions for now. In the future we may need to change it as some disrto's may use sym links to indicate the latest jar. In such a case the RPM will stripdown the version from the artifact. On the other hand adding that _0001 suffix to the project itself complicates (just a bit) the packaging. It has to be discussed where that postrelease number has to go: in the version tag of the package, in the release tag? You can see an example of that discussion in the review bug of the ovirt-engine package (see [2]). I agree. As I said- it was meant to be for maven2 bug solution. Not needed when we move to maven3. - Do we want to expose oVirt engine artifacts in a Maven repo for others to consume? * If we do, we'll need to make sure we have a scheme that works both for Maven and for packaging (RPM) comparisons. I think that we don't need to make the artifacts available for others now. Is anyone consuming those artifacts? However I agree that is good to have numbering that works for Maven and RPM. If we won't allow artifact consumption, we won't have it ;) The idea is to be able to share artifacts between sibling projects as well as external ones. As I see it, modular programming is integrating several sub-projects into a working project, based on an API. So each sub
Re: [Engine-devel] Jar versions in ovirt-engine
On 19/04/12 16:53, Juan Hernandez wrote: On 04/19/2012 03:22 PM, Doron Fediuck wrote: On 19/04/12 13:26, Juan Hernandez wrote: On 04/19/2012 12:00 PM, Doron Fediuck wrote: On 18/04/12 14:04, Juan Hernandez wrote: On 04/18/2012 09:51 AM, Ofer Schreiber wrote: Ever wondered why the version of oVirt's first release is 3.0.0_0001? The answer is simple - We use ovirt-engine jar's version as our main release version. Personally, I think the current versioning scheme is ugly. Actually, I can't name even one open-source project using _ in it's version. What can we do about it? We have couple of options: 1. Leave the engine alone, and use a separate versioning scheme (e.g - use just 3.1.0 as the main version for next release) 2. Remove _ from engine jars 3. Do nothing. I'd like to hear your thoughts, as well as the reasons to use such an unusual versioning scheme. --- Ofer Schreiber oVirt Release Manager ___ Arch mailing list a...@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch From my point of view using the 0001 suffix in the names of the jar files is not a big problem, but I agree that using it in the release number is ugly, and it produces issues/discussions during packaging. I vote for option #1: use 3.1.0 for the next main version. The original versioning scheme was due to a bug in maven2. Juan, I've read some of the Java packaging concepts, but didn't see (or missed) thoughts about modular versioning (ie- artifacts). Here are the things to consider here; - Current RPM's are using the version declared in the POM files. Should this concept remain? * I think it should remain, as other packaging systems should be able to use it as well and end-up is the similar project version. I can talk from the Fedora point of view only, as that is what I know a bit. In Fedora there can be only one version of a given jar file installed in the system, so there is no point in adding a version number to the name of that jar file: the version number is already in the package containing that jar file. In fact if the build generates jar files with version numbers in the name the RPM should remove those jar files. That is why I say that having any kind of numbers in the names of the jars is not important: we have to remove them anyway. Packaging guidelines (see [1]) recommend to avoid version numbers in the jar files, and I think that makes sense. This would be the easy solution. Again talking only about Fedora: Having just one version of every jar is not simple at all, in fact it requires a lot of work to make sure that the selected versions work properly together. See below, we actually share the same view... What happens when you have more than a single Java app, and both using different versions of the same jar file? This means that one of the app's will need to bring it along and use it locally, rather than system-level usage. What happens is that both applications have to be patched so that they work correctly with the same version of that jar file. If possible the patches are pushed upstream, if not they applied as part of the package. Embedding another version of that jar file in one of the applications is not allowed, in fact that is something that packagers have to undo quite often. See below... converging into the latest jar is what I figured that will happen. Still, as I see it such constraints are not really needed. I'm guessing if we assume such a constraint the solution will be to force all app's to use latest jar version, which isn't trivial. I agree completely, it is not trivial at all, that is where packagers expend most of their time. So some distro's will allow of concept of slotted installation. This means I currently /have/ 2 working versions of postgres in my laptop (using Gentoo)- equery l postgresql-server * Searching for postgresql-server ... [IP-] [ ] dev-db/postgresql-server-8.4.11:8.4 [IP-] [ ] dev-db/postgresql-server-9.1.3:9.1 The same works on my laptop for Maven, Java, Python and many others. If you think about it, Fedora supports slotted installation for kernels, and then added alternatives to do that with other packages as well (mta, Java..). So there's a need and a way to handle several versions of the same library (regardless of the language), and we should be careful when taking such assumptions. At least try to be as flexible as possible, to allow others to join in. In Fedora that is allowed only for major versions: java-1.7.0 and java-1.6.0, maven 2 and maven 3, so on, but not usually for minor versions (there are exceptions). It's a good start. So learning from Fedora I'd say- let the RPM install in a versioned folder (ie- /usr/lib/jvm/jre-1.5.0-gcj/..), and leave the jar files without versions for now. In the future we may need to change it as some disrto's may use sym links to indicate the latest jar
Re: [Engine-devel] Jar versions in ovirt-engine
On 19/04/12 17:17, Juan Hernandez wrote: On 04/19/2012 04:10 PM, Doron Fediuck wrote: On 19/04/12 16:53, Juan Hernandez wrote: On 04/19/2012 03:22 PM, Doron Fediuck wrote: On 19/04/12 13:26, Juan Hernandez wrote: On 04/19/2012 12:00 PM, Doron Fediuck wrote: On 18/04/12 14:04, Juan Hernandez wrote: On 04/18/2012 09:51 AM, Ofer Schreiber wrote: Ever wondered why the version of oVirt's first release is 3.0.0_0001? The answer is simple - We use ovirt-engine jar's version as our main release version. Personally, I think the current versioning scheme is ugly. Actually, I can't name even one open-source project using _ in it's version. What can we do about it? We have couple of options: 1. Leave the engine alone, and use a separate versioning scheme (e.g - use just 3.1.0 as the main version for next release) 2. Remove _ from engine jars 3. Do nothing. I'd like to hear your thoughts, as well as the reasons to use such an unusual versioning scheme. --- Ofer Schreiber oVirt Release Manager ___ Arch mailing list a...@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch From my point of view using the 0001 suffix in the names of the jar files is not a big problem, but I agree that using it in the release number is ugly, and it produces issues/discussions during packaging. I vote for option #1: use 3.1.0 for the next main version. The original versioning scheme was due to a bug in maven2. Juan, I've read some of the Java packaging concepts, but didn't see (or missed) thoughts about modular versioning (ie- artifacts). Here are the things to consider here; - Current RPM's are using the version declared in the POM files. Should this concept remain? * I think it should remain, as other packaging systems should be able to use it as well and end-up is the similar project version. I can talk from the Fedora point of view only, as that is what I know a bit. In Fedora there can be only one version of a given jar file installed in the system, so there is no point in adding a version number to the name of that jar file: the version number is already in the package containing that jar file. In fact if the build generates jar files with version numbers in the name the RPM should remove those jar files. That is why I say that having any kind of numbers in the names of the jars is not important: we have to remove them anyway. Packaging guidelines (see [1]) recommend to avoid version numbers in the jar files, and I think that makes sense. This would be the easy solution. Again talking only about Fedora: Having just one version of every jar is not simple at all, in fact it requires a lot of work to make sure that the selected versions work properly together. See below, we actually share the same view... What happens when you have more than a single Java app, and both using different versions of the same jar file? This means that one of the app's will need to bring it along and use it locally, rather than system-level usage. What happens is that both applications have to be patched so that they work correctly with the same version of that jar file. If possible the patches are pushed upstream, if not they applied as part of the package. Embedding another version of that jar file in one of the applications is not allowed, in fact that is something that packagers have to undo quite often. See below... converging into the latest jar is what I figured that will happen. Still, as I see it such constraints are not really needed. I'm guessing if we assume such a constraint the solution will be to force all app's to use latest jar version, which isn't trivial. I agree completely, it is not trivial at all, that is where packagers expend most of their time. So some distro's will allow of concept of slotted installation. This means I currently /have/ 2 working versions of postgres in my laptop (using Gentoo)- equery l postgresql-server * Searching for postgresql-server ... [IP-] [ ] dev-db/postgresql-server-8.4.11:8.4 [IP-] [ ] dev-db/postgresql-server-9.1.3:9.1 The same works on my laptop for Maven, Java, Python and many others. If you think about it, Fedora supports slotted installation for kernels, and then added alternatives to do that with other packages as well (mta, Java..). So there's a need and a way to handle several versions of the same library (regardless of the language), and we should be careful when taking such assumptions. At least try to be as flexible as possible, to allow others to join in. In Fedora that is allowed only for major versions: java-1.7.0 and java-1.6.0, maven 2 and maven 3, so on, but not usually for minor versions (there are exceptions). It's a good start. So learning from Fedora I'd say- let the RPM install in a versioned folder (ie- /usr/lib/jvm/jre-1.5.0-gcj/..), and leave the jar files without versions for now. In the future we may need
