Re: [Engine-devel] Adding Authentication mechanism to oVirt
- Original Message - > From: "Itamar Heim" > To: "Alon Bar-Lev" > Cc: "Thierry Kauffmann" , "Yaniv Kaul" > , "engine-devel" > > Sent: Thursday, December 13, 2012 1:43:19 AM > Subject: Re: Adding Authentication mechanism to oVirt > > On 12/12/2012 06:53 PM, Alon Bar-Lev wrote: > > Hello Thierry, > > > > If I understand correctly you wish to help in modifying the engine > > to support non GSSAPI authentication methods. > > > > Following is a quick design goals for this implementation. > > > > I will be glad to improve this. > > thanks for writing this part. > what about making the LDAP part plugable, or at least > scheme-configurable to cover openLDAP, or any other source (atlassian > crowd), or well, anything? I thought our primary goal is to reduce the noise around the kerberos. Doing a complete re-write (aka modular) will consume far more resources and time. It also touches many part of the implementation instead of only the LDAP context and authentication. > > thanks, > Itamar > > > Alon > > > > --- > > > > Implementation should support the following transports: > > > > 1. LDAP (plain). > > 2. LDAP over TLS. > > 3. LDAP with StartTLS. > > > > Implementation should support the following authentication methods: > > > > 1. Simple. > > 2. Digest-MD5 (plain and strong). > > > > I believe the GSSAPI can be dropped, I see no advantage of using > > it. > > > > A sample of low level implementation for transport and > > authentication is attached. > > > > When adding a domain the following facts should be provided: > > > > 1. Search user name. > > 2. Search user password. > > 3. Transport type (ldap, ldaps, ldap+startTLS) > > 4. Authentication (simple, Digest-MD5) > > 5. Sever selection policy (failover, round-robin, random). > > 6. Server address type (explicit, DNS record) > > 7. Server address set. > > 8. Optional base DN. > > 9. Optional root certificate. > > 10. Optional certificate chain. > > 11. Search page size. > > 10. Query timeout. > > etc... > > > > Within product there are two separate components that perform LDAP > > authentication: > > > > 1. User password validation. > > 2. User permission fetch. > > > > These two components needs to work in share-nothing mode, meaning > > that each should communicate with directory independently with the > > other. > > > > USER PASSWORD VALIDATION > > > > Input: user > > Input: domain > > Input: password > > Output: DN of user > > Output: success/failure > > Credentials used: user/password provided. > > Notes: LDAP session should not be cached. > > Logic: Perform LDAP bind. > > > > USER PERMISSION FETCH > > > > Input: DN of user (passed by user password validation) > > Input: domain (passed by user password validation) > > Output: A set of permissions > > Credentials used: search user and password configured within > > system. > > Notes: LDAP context can be cached. > > Logic: Perform LDAP searches, this is most of current logic. > > > > > ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Adding Authentication mechanism to oVirt
On 12/12/2012 06:53 PM, Alon Bar-Lev wrote: Hello Thierry, If I understand correctly you wish to help in modifying the engine to support non GSSAPI authentication methods. Following is a quick design goals for this implementation. I will be glad to improve this. thanks for writing this part. what about making the LDAP part plugable, or at least scheme-configurable to cover openLDAP, or any other source (atlassian crowd), or well, anything? thanks, Itamar Alon --- Implementation should support the following transports: 1. LDAP (plain). 2. LDAP over TLS. 3. LDAP with StartTLS. Implementation should support the following authentication methods: 1. Simple. 2. Digest-MD5 (plain and strong). I believe the GSSAPI can be dropped, I see no advantage of using it. A sample of low level implementation for transport and authentication is attached. When adding a domain the following facts should be provided: 1. Search user name. 2. Search user password. 3. Transport type (ldap, ldaps, ldap+startTLS) 4. Authentication (simple, Digest-MD5) 5. Sever selection policy (failover, round-robin, random). 6. Server address type (explicit, DNS record) 7. Server address set. 8. Optional base DN. 9. Optional root certificate. 10. Optional certificate chain. 11. Search page size. 10. Query timeout. etc... Within product there are two separate components that perform LDAP authentication: 1. User password validation. 2. User permission fetch. These two components needs to work in share-nothing mode, meaning that each should communicate with directory independently with the other. USER PASSWORD VALIDATION Input: user Input: domain Input: password Output: DN of user Output: success/failure Credentials used: user/password provided. Notes: LDAP session should not be cached. Logic: Perform LDAP bind. USER PERMISSION FETCH Input: DN of user (passed by user password validation) Input: domain (passed by user password validation) Output: A set of permissions Credentials used: search user and password configured within system. Notes: LDAP context can be cached. Logic: Perform LDAP searches, this is most of current logic. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] UI plugins - next steps
On 12/12/2012 02:57 PM, Vojtech Szocs wrote:
1. Pass proper (restapi-definition) entities to UI plugins, instead of simple
"{entityId:}" objects
- using restapi-types mappers to map backend business entities to
restapi-definition entities (Java POJOs generated from api.xsd via JAXB)
- exporting restapi-definition entities for use with JavaScript (UI
plugins), e.g. using gwt-exporter [1] (alternatively, we could use GWT deferred
binding and do this ourselves)
I think its time we revisit the implications/scope of moving the UI to
work on top of the REST API or just the REST entities.
until then, i think just adding specific fields that make sense/are
asked for for these entities should be good enough.
if we see the "moving UI on top of REST API" is still too far, then
doing the type mapping for just the plugins would make sense.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Adding Authentication mechanism to oVirt
- Original Message - > From: "Itamar Heim" > To: "Alon Bar-Lev" > Cc: "Thierry Kauffmann" , "Yaniv Kaul" > , "engine-devel" > > Sent: Thursday, December 13, 2012 1:27:15 AM > Subject: Re: Adding Authentication mechanism to oVirt > > On 12/12/2012 06:53 PM, Alon Bar-Lev wrote: > > I believe the GSSAPI can be dropped, I see no advantage of using > > it. > > how would upgrade/backward compatibility look like? > > We have the user/password, so there should be no problem we can test if authentication succeeds in methods based on some order... but we cross this when we reach the point. Asking the user for transport/authentication scheme during upgrade is something that can also be done. We can keep the GSSAPI around of course... Not something I would like to do. Alon ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Adding Authentication mechanism to oVirt
On 12/12/2012 06:53 PM, Alon Bar-Lev wrote: I believe the GSSAPI can be dropped, I see no advantage of using it. how would upgrade/backward compatibility look like? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] ThreadPoolUtil with 500 threads and no queue?
- Original Message - > From: "Juan Hernandez" > To: "Michael Kublin" > Cc: engine-devel@ovirt.org > Sent: Wednesday, December 12, 2012 6:31:44 PM > Subject: Re: [Engine-devel] ThreadPoolUtil with 500 threads and no queue? > > On 12/12/2012 05:21 PM, Michael Kublin wrote: > > > > > > - Original Message - > >> From: "Juan Hernandez" > >> To: engine-devel@ovirt.org > >> Sent: Wednesday, December 12, 2012 6:06:30 PM > >> Subject: [Engine-devel] ThreadPoolUtil with 500 threads and no > >> queue? > >> > >> Hello all, > >> > >> What is the reasoning behind the decision to have a pool with a > >> maximum > >> of 500 threads and no job queue (see ThreadPoolUtil.java)? > >> Wouldn't > >> it > >> make more sense to have a much smaller thread pool and a > >> potentially > >> large queue of jobs? > >> > >> Regards, > >> Juan Hernandez > > > > There are three general strategies for queuing: > > > > 1) Direct handoffs. A good default choice for a work queue is a > > SynchronousQueue that hands off tasks to threads without > > otherwise holding them. Here, an attempt to queue a task will > > fail if no threads are immediately available to run it, so a new > > thread will be constructed. This policy avoids lockups when > > handling sets of requests that might have internal dependencies. > > Direct handoffs generally require unbounded maximumPoolSizes to > > avoid rejection of new submitted tasks. This in turn admits the > > possibility of unbounded thread growth when commands continue to > > arrive on average faster than they can be processed. > > 2) Unbounded queues. Using an unbounded queue (for example a > > LinkedBlockingQueue without a predefined capacity) will cause > > new tasks to wait in the queue when all corePoolSize threads are > > busy. Thus, no more than corePoolSize threads will ever be > > created. (And the value of the maximumPoolSize therefore doesn't > > have any effect.) This may be appropriate when each task is > > completely independent of others, so tasks cannot affect each > > others execution; for example, in a web page server. While this > > style of queuing can be useful in smoothing out transient bursts > > of requests, it admits the possibility of unbounded work queue > > growth when commands continue to arrive on average faster than > > they can be processed. > > 3) Bounded queues. A bounded queue (for example, an > > ArrayBlockingQueue) helps prevent resource exhaustion when used > > with finite maximumPoolSizes, but can be more difficult to tune > > and control. Queue sizes and maximum pool sizes may be traded > > off for each other: Using large queues and small pools minimizes > > CPU usage, OS resources, and context-switching overhead, but can > > lead to artificially low throughput. If tasks frequently block > > (for example if they are I/O bound), a system may be able to > > schedule time for more threads than you otherwise allow. Use of > > small queues generally requires larger pool sizes, which keeps > > CPUs busier but may encounter unacceptable scheduling overhead, > > which also decreases throughput. > > > > Why not? we are using 1). > > Actually 500 threads should be enough for very big applications > > I think that 500 are maybe too much, even for a very big application, > the reasons you explain very well in 3). A resource that we can very > easily overload is the database. If those 500 threads happen to need > database connections we have a problem. > > I think we should use a bounded queue and have both the number of > threads and the size of the queue configurable. Does that make sense? > Actually I think that it is more complicated. Today most of the threads are used to perform xml rpc calls and in order to paralyse some operations (for example connect all host to some storage domain), most of the action from users are not opening a new thread in order to retrieve some information from DB. I think today most of the load on DB comes from scheduled monitoring jobs, which are running every couple seconds and performing enormous number of queries to DB. I think if we want to make some kind of queue it should be done at the business logic level and if we decided that we can not run some action because of load (user action or internal action) we can reject it, I think that thread pool is internal mechanism which can not solve our design problems, we can tune it but it is not main cause of problem. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Adding Authentication mechanism to oVirt
- Original Message - > From: "Yaniv Kaul" > To: "Alon Bar-Lev" > Cc: "Thierry Kauffmann" , "Itamar Heim" > , "engine-devel" > > Sent: Wednesday, December 12, 2012 7:18:18 PM > Subject: Re: Adding Authentication mechanism to oVirt > > On 12/12/2012 06:53 PM, Alon Bar-Lev wrote: > > Hello Thierry, > > > > If I understand correctly you wish to help in modifying the engine > > to support non GSSAPI authentication methods. > > > > Following is a quick design goals for this implementation. > > > > I will be glad to improve this. > > Alon > > > > --- > > > > Implementation should support the following transports: > > > > 1. LDAP (plain). > > 2. LDAP over TLS. > > 3. LDAP with StartTLS. > > > > Implementation should support the following authentication methods: > > > > 1. Simple. > > 2. Digest-MD5 (plain and strong). > > > > I believe the GSSAPI can be dropped, I see no advantage of using > > it. > > Except that Active Directory usually does not work (especially not > out > of the box) with certificates. > Y. Active Directory support SASL with Diget-MD5, no need to support GSSAPI. > > > > > A sample of low level implementation for transport and > > authentication is attached. > > > > When adding a domain the following facts should be provided: > > > > 1. Search user name. > > 2. Search user password. > > 3. Transport type (ldap, ldaps, ldap+startTLS) > > 4. Authentication (simple, Digest-MD5) > > 5. Sever selection policy (failover, round-robin, random). > > 6. Server address type (explicit, DNS record) > > 7. Server address set. > > 8. Optional base DN. > > 9. Optional root certificate. > > 10. Optional certificate chain. > > 11. Search page size. > > 10. Query timeout. > > etc... > > > > Within product there are two separate components that perform LDAP > > authentication: > > > > 1. User password validation. > > 2. User permission fetch. > > > > These two components needs to work in share-nothing mode, meaning > > that each should communicate with directory independently with the > > other. > > > > USER PASSWORD VALIDATION > > > > Input: user > > Input: domain > > Input: password > > Output: DN of user > > Output: success/failure > > Credentials used: user/password provided. > > Notes: LDAP session should not be cached. > > Logic: Perform LDAP bind. > > > > USER PERMISSION FETCH > > > > Input: DN of user (passed by user password validation) > > Input: domain (passed by user password validation) > > Output: A set of permissions > > Credentials used: search user and password configured within > > system. > > Notes: LDAP context can be cached. > > Logic: Perform LDAP searches, this is most of current logic. > > ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Adding Authentication mechanism to oVirt
On 12/12/2012 06:53 PM, Alon Bar-Lev wrote: Hello Thierry, If I understand correctly you wish to help in modifying the engine to support non GSSAPI authentication methods. Following is a quick design goals for this implementation. I will be glad to improve this. Alon --- Implementation should support the following transports: 1. LDAP (plain). 2. LDAP over TLS. 3. LDAP with StartTLS. Implementation should support the following authentication methods: 1. Simple. 2. Digest-MD5 (plain and strong). I believe the GSSAPI can be dropped, I see no advantage of using it. Except that Active Directory usually does not work (especially not out of the box) with certificates. Y. A sample of low level implementation for transport and authentication is attached. When adding a domain the following facts should be provided: 1. Search user name. 2. Search user password. 3. Transport type (ldap, ldaps, ldap+startTLS) 4. Authentication (simple, Digest-MD5) 5. Sever selection policy (failover, round-robin, random). 6. Server address type (explicit, DNS record) 7. Server address set. 8. Optional base DN. 9. Optional root certificate. 10. Optional certificate chain. 11. Search page size. 10. Query timeout. etc... Within product there are two separate components that perform LDAP authentication: 1. User password validation. 2. User permission fetch. These two components needs to work in share-nothing mode, meaning that each should communicate with directory independently with the other. USER PASSWORD VALIDATION Input: user Input: domain Input: password Output: DN of user Output: success/failure Credentials used: user/password provided. Notes: LDAP session should not be cached. Logic: Perform LDAP bind. USER PERMISSION FETCH Input: DN of user (passed by user password validation) Input: domain (passed by user password validation) Output: A set of permissions Credentials used: search user and password configured within system. Notes: LDAP context can be cached. Logic: Perform LDAP searches, this is most of current logic. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Adding Authentication mechanism to oVirt
Hello Thierry,
If I understand correctly you wish to help in modifying the engine to support
non GSSAPI authentication methods.
Following is a quick design goals for this implementation.
I will be glad to improve this.
Alon
---
Implementation should support the following transports:
1. LDAP (plain).
2. LDAP over TLS.
3. LDAP with StartTLS.
Implementation should support the following authentication methods:
1. Simple.
2. Digest-MD5 (plain and strong).
I believe the GSSAPI can be dropped, I see no advantage of using it.
A sample of low level implementation for transport and authentication is
attached.
When adding a domain the following facts should be provided:
1. Search user name.
2. Search user password.
3. Transport type (ldap, ldaps, ldap+startTLS)
4. Authentication (simple, Digest-MD5)
5. Sever selection policy (failover, round-robin, random).
6. Server address type (explicit, DNS record)
7. Server address set.
8. Optional base DN.
9. Optional root certificate.
10. Optional certificate chain.
11. Search page size.
10. Query timeout.
etc...
Within product there are two separate components that perform LDAP
authentication:
1. User password validation.
2. User permission fetch.
These two components needs to work in share-nothing mode, meaning that each
should communicate with directory independently with the other.
USER PASSWORD VALIDATION
Input: user
Input: domain
Input: password
Output: DN of user
Output: success/failure
Credentials used: user/password provided.
Notes: LDAP session should not be cached.
Logic: Perform LDAP bind.
USER PERMISSION FETCH
Input: DN of user (passed by user password validation)
Input: domain (passed by user password validation)
Output: A set of permissions
Credentials used: search user and password configured within system.
Notes: LDAP context can be cached.
Logic: Perform LDAP searches, this is most of current logic.
/*
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import java.io.*;
import java.net.*;
import java.util.*;
import java.security.cert.*;
import javax.naming.*;
import javax.naming.directory.*;
import javax.naming.ldap.*;
import javax.net.*;
import javax.net.ssl.*;
public class LDAPSearch {
private boolean _doStartTLS = false;
private boolean _doSASL_digestMD5 = false;
private boolean _doSASL_strong = false;
private boolean _verifyPeerCertificate = true;
private String _securityProtocol = "TLSv1";
private String _user;
private String _password;
private int _searchPageSize;
LdapContext _ctx;
StartTlsResponse _tls;
String _nameContext = "";
public static interface ResultHandler {
public boolean handle(NamingEnumeration results) throws NamingException;
}
public void useStartTLS(boolean startTLS) {
_doStartTLS = startTLS;
}
public void useDigestMD5(boolean strong) {
_doSASL_digestMD5 = true;
_doSASL_strong = strong;
}
public void setSecurityProtocol(String securityProtocol) {
_securityProtocol = securityProtocol;
}
public void setCredentials(String user, String password) {
_user = user;
_password = password;
}
public void setSearchPageSize(int searchPageSize) {
_searchPageSize = searchPageSize;
}
public void setVerifyPeerCertificate(boolean verify) {
_verifyPeerCertificate = verify;
}
public void createContext(String url)
throws NamingException, IOException {
boolean doTLS = url.startsWith("ldaps:");
Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
env.put(Context.PROVIDER_URL, url);
if (doTLS) {
env.put(Context.SECURITY_PROTOCOL, _securityProtocol);
if (!_verifyPeerCertificate) {
UnsecureSSLSocketFactory.setDefaultProtocol(_securityProtocol);
env.put("java.naming.ldap.factory.socket", UnsecureSSLSocketFactory.class.getName());
}
}
_ctx = new InitialLdapContext(env, null);
if (!doTLS && _doStartTLS) {
_tls = (StartTlsResponse)_ctx.extendedOperation(
new StartTlsRequest()
);
SSLSession session = _tls.negotiate(
_verifyPeerCertificate ?
(SSLSocketFactory)SSLSocketFactory.getDefault() :
UnsecureSSLSocketFactory.createSocketFact
Re: [Engine-devel] ThreadPoolUtil with 500 threads and no queue?
On 12/12/2012 05:21 PM, Michael Kublin wrote: > > > - Original Message - >> From: "Juan Hernandez" >> To: engine-devel@ovirt.org >> Sent: Wednesday, December 12, 2012 6:06:30 PM >> Subject: [Engine-devel] ThreadPoolUtil with 500 threads and no queue? >> >> Hello all, >> >> What is the reasoning behind the decision to have a pool with a >> maximum >> of 500 threads and no job queue (see ThreadPoolUtil.java)? Wouldn't >> it >> make more sense to have a much smaller thread pool and a potentially >> large queue of jobs? >> >> Regards, >> Juan Hernandez > > There are three general strategies for queuing: > > 1) Direct handoffs. A good default choice for a work queue is a > SynchronousQueue that hands off tasks to threads without otherwise holding > them. Here, an attempt to queue a task will fail if no threads are > immediately available to run it, so a new thread will be constructed. This > policy avoids lockups when handling sets of requests that might have internal > dependencies. Direct handoffs generally require unbounded maximumPoolSizes to > avoid rejection of new submitted tasks. This in turn admits the possibility > of unbounded thread growth when commands continue to arrive on average faster > than they can be processed. > 2) Unbounded queues. Using an unbounded queue (for example a > LinkedBlockingQueue without a predefined capacity) will cause new tasks to > wait in the queue when all corePoolSize threads are busy. Thus, no more than > corePoolSize threads will ever be created. (And the value of the > maximumPoolSize therefore doesn't have any effect.) This may be appropriate > when each task is completely independent of others, so tasks cannot affect > each others execution; for example, in a web page server. While this style of > queuing can be useful in smoothing out transient bursts of requests, it > admits the possibility of unbounded work queue growth when commands continue > to arrive on average faster than they can be processed. > 3) Bounded queues. A bounded queue (for example, an ArrayBlockingQueue) > helps prevent resource exhaustion when used with finite maximumPoolSizes, but > can be more difficult to tune and control. Queue sizes and maximum pool sizes > may be traded off for each other: Using large queues and small pools > minimizes CPU usage, OS resources, and context-switching overhead, but can > lead to artificially low throughput. If tasks frequently block (for example > if they are I/O bound), a system may be able to schedule time for more > threads than you otherwise allow. Use of small queues generally requires > larger pool sizes, which keeps CPUs busier but may encounter unacceptable > scheduling overhead, which also decreases throughput. > > Why not? we are using 1). > Actually 500 threads should be enough for very big applications I think that 500 are maybe too much, even for a very big application, the reasons you explain very well in 3). A resource that we can very easily overload is the database. If those 500 threads happen to need database connections we have a problem. I think we should use a bounded queue and have both the number of threads and the size of the queue configurable. Does that make sense? -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] ThreadPoolUtil with 500 threads and no queue?
- Original Message - > From: "Juan Hernandez" > To: engine-devel@ovirt.org > Sent: Wednesday, December 12, 2012 6:06:30 PM > Subject: [Engine-devel] ThreadPoolUtil with 500 threads and no queue? > > Hello all, > > What is the reasoning behind the decision to have a pool with a > maximum > of 500 threads and no job queue (see ThreadPoolUtil.java)? Wouldn't > it > make more sense to have a much smaller thread pool and a potentially > large queue of jobs? > > Regards, > Juan Hernandez There are three general strategies for queuing: 1) Direct handoffs. A good default choice for a work queue is a SynchronousQueue that hands off tasks to threads without otherwise holding them. Here, an attempt to queue a task will fail if no threads are immediately available to run it, so a new thread will be constructed. This policy avoids lockups when handling sets of requests that might have internal dependencies. Direct handoffs generally require unbounded maximumPoolSizes to avoid rejection of new submitted tasks. This in turn admits the possibility of unbounded thread growth when commands continue to arrive on average faster than they can be processed. 2) Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without a predefined capacity) will cause new tasks to wait in the queue when all corePoolSize threads are busy. Thus, no more than corePoolSize threads will ever be created. (And the value of the maximumPoolSize therefore doesn't have any effect.) This may be appropriate when each task is completely independent of others, so tasks cannot affect each others execution; for example, in a web page server. While this style of queuing can be useful in smoothing out transient bursts of requests, it admits the possibility of unbounded work queue growth when commands continue to arrive on average faster than they can be processed. 3) Bounded queues. A bounded queue (for example, an ArrayBlockingQueue) helps prevent resource exhaustion when used with finite maximumPoolSizes, but can be more difficult to tune and control. Queue sizes and maximum pool sizes may be traded off for each other: Using large queues and small pools minimizes CPU usage, OS resources, and context-switching overhead, but can lead to artificially low throughput. If tasks frequently block (for example if they are I/O bound), a system may be able to schedule time for more threads than you otherwise allow. Use of small queues generally requires larger pool sizes, which keeps CPUs busier but may encounter unacceptable scheduling overhead, which also decreases throughput. Why not? we are using 1). Actually 500 threads should be enough for very big applications ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] ThreadPoolUtil with 500 threads and no queue?
Hello all, What is the reasoning behind the decision to have a pool with a maximum of 500 threads and no job queue (see ThreadPoolUtil.java)? Wouldn't it make more sense to have a much smaller thread pool and a potentially large queue of jobs? Regards, Juan Hernandez -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Request power-user status for mom project in jenkins
On Tue, 2012-12-11 at 20:58 +0200, Dan Kenigsberg wrote: > On Tue, Dec 11, 2012 at 11:24:07AM -0500, Eyal Edri wrote: > > [adding engine-devel] > > > > usually there is an official vote on each new power user for jenkins, > > and as long as there are no objections it is approved. > > > > since i know you're a mom maintainer and familiar with jenkins, i vote +1. > > I trust Adam (who is mom's maintainer), and trust that he is capable of > becoming familiar with jenkins without causing pain to other users. > > +1 +1 > ___ > Infra mailing list > in...@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] UI plugins patches merged, moving out of proof-of-concept stage
Great work Vojtech. Thank you. -Original Message- From: engine-devel-boun...@ovirt.org [mailto:engine-devel-boun...@ovirt.org] On Behalf Of Vojtech Szocs Sent: Wednesday, December 12, 2012 7:28 AM To: engine-devel Subject: [Engine-devel] UI plugins patches merged, moving out of proof-of-concept stage Hi everyone, I'm happy to announce that oVirt UI plugins feature just moved out of proof-of-concept stage, with relevant patches [1,2] merged into ovirt-engine git repository. I'll be working on related documentation (wiki) in near future, along with contributing some sample plugins into samples-uiplugins [3] git repository. I'd like to thank everyone involved in pushing this feature forward. I hope we'll see some interesting plugins soon! Regards, Vojtech [1] http://gerrit.ovirt.org/#/c/8120/ [2] http://gerrit.ovirt.org/#/c/9250/ [3] git://gerrit.ovirt.org/samples-uiplugins ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] UI plugins - next steps
Nice work!
See some comments inline.
- Original Message -
> From: "Vojtech Szocs"
> To: "engine-devel"
> Sent: Wednesday, December 12, 2012 2:57:11 PM
> Subject: [Engine-devel] UI plugins - next steps
>
> Hi guys,
>
> here's my list of tasks that didn't make it into UI plugins yet.
> These tasks should be revisited and implemented in future patches.
> The list isn't sorted in any way, so feel free to comment or
> highlight specific tasks to raise their priority, or add your own
> tasks/ideas.
>
> 1. Pass proper (restapi-definition) entities to UI plugins, instead
> of simple "{entityId:}" objects
>- using restapi-types mappers to map backend business entities to
>restapi-definition entities (Java POJOs generated from api.xsd
>via JAXB)
>- exporting restapi-definition entities for use with JavaScript
>(UI plugins), e.g. using gwt-exporter [1] (alternatively, we
>could use GWT deferred binding and do this ourselves)
>
> 2. showDialog API function should integrate with WebAdmin dialog
> infrastructure
>- shouldn't be too hard, need to create custom dialog
>PresenterWidget/View and bind it as non-singleton (just like
>other popups in oVirt GUI)
>
> 3. Extend existing WebAdmin dialogs - prototype Doron's proposal to
> extend Edit Cluster Policy dialog with custom options that would be
> passed to backend
>
> 4. Introduce form-based tabs - custom tab that would display
> key/value pairs organized in columns, just like in standard
> "General" sub tabs
>
> 5. Introduce table-based tabs - custom tab that would display grid
> (table) just like in standard main tabs, with API to add columns,
> set data, etc.
>
> 6. Improve api.addMainTabActionButton to allow different button vs.
> context menu item representations (only button, only menu item,
> both)
>
> 7. Add api.currentUserName and api.currentUserId API functions (these
> would return same data as with UserLogin event parameters)
>
Pushing this now.
http://gerrit.ovirt.org/#/c/9998/
> 8. Minor things: consider renaming UiInit to PluginInit (one-time
> plugin initialization, called only once, can do non-UI init stuff
> there as well), consider renaming api.ready to api.initialize
> (plugin indicates that infrastructure can proceed with initializing
> itself)
>
I think we should also consider adding an option to add a button in a more
general scope, for things that aren't entity-specific, but system-specific.
Maybe next to the "configure" button, in the upper right corner of the screen?
> Regards,
> Vojtech
>
> [1] http://code.google.com/p/gwt-exporter/wiki/GettingStarted
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
On 12/12/2012 03:10 PM, Vojtech Szocs wrote: > Thanks Michael, in that case we can just use XML when talking with REST API. > > Also, one idea worth implementing in future is to have some sort of debug > console on GUI (I've seen this idea implemented in other applications). When > activated, users could see REST API XML communication (request/response) > between WebAdmin and Engine, which could be useful for someone interested in > writing his own REST API client. right, i always do that in all clients i've wrote (including SDKs & CLI). > > Vojtech > > > - Original Message - > From: "Michael Pasternak" > To: "Vojtech Szocs" > Cc: "Itamar Heim" , "engine-devel" > Sent: Wednesday, December 12, 2012 1:33:56 PM > Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here > > On 12/12/2012 01:51 PM, Vojtech Szocs wrote: >> Hi, sorry for my late response, >> Our first iteration could be: a1, rewrite UiCommon (UI business logic) layer to work with API types a2, write adapter (e.g. modify Frontend class) between UiCommon using API types and Generic API (GWT RPC) using internal entities >>> >>> does this mean converting three times? >> >> Not sure what you mean, but here's my idea of the first iteration - what >> would happen on frontend side (GWT): >> 1. UiCommon, using API types, invokes query/command via RPC bridge >> (org.ovirt.engine.ui.frontend.Frontend class) >> 2. Frontend class, still using Generic API (GWT RPC), does the actual >> query/command invocation >> 3. Frontend class receives query/command result and translates it from BE >> type to API type (converter callback) >> >> My idea was to isolate BE type usage into Frontend class, so that when we >> decide to use REST API, we just need to update our RPC bridge (Frontend >> class). >> Our second iteration could be: b1, remove adapter from step a2, drop Generic API (GWT RPC) usage b2, write API type <-> JSON mapper, possibly using some existing framework >>> >>> do you mean talking with api in JSON? >> >> Yes, that was my original idea. I'm not against using XML when talking with >> REST API either. GWT supports parsing both JSON and XML. > > infra. is there, but officially we not supporting json yet, > (resteasy-json-provider has issues with jaxb, - we waiting for fix) > >> >> Vojtech >> >> >> - Original Message - >> From: "Michael Pasternak" >> To: "Vojtech Szocs" >> Cc: "Itamar Heim" , "engine-devel" >> Sent: Sunday, November 25, 2012 9:27:59 AM >> Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here >> >> On 11/20/2012 12:57 PM, Vojtech Szocs wrote: >>> Hi Michael, >>> if we plan moving UI on top of API, you should be: 1. importing restapi-types project 2. writing intermediate layer to translate BE entities to API's using #1 3. using public entities from #2 this way your future migration to API (instead of native BE) will be much more easier. >>> >>> Indeed, this is very useful for GWT RPC -> REST API transition in general, >>> many thanks for pointing this out, Michael. As you suggest, we can use >>> restapi-types mappers to translate between internal entities and API types. >>> >>> Our first iteration could be: >>> a1, rewrite UiCommon (UI business logic) layer to work with API types >>> a2, write adapter (e.g. modify Frontend class) between UiCommon using API >>> types and Generic API (GWT RPC) using internal entities >> >> does this mean converting three times? >> >>> >>> Our second iteration could be: >>> b1, remove adapter from step a2, drop Generic API (GWT RPC) usage >>> b2, write API type <-> JSON mapper, possibly using some existing framework >> >> do you mean talking with api in JSON? >> >>> b3, write RPC layer that talks REST API with Engine >>> >>> However, in order to use REST API types on client (GWT), we need their >>> source code. restapi-interface-definition uses Maven JAXB plugin to >>> generate API types from XSD schema (src/main/resources/api.xsd). On client, >>> we need restapi-definition--sources.jar that includes those >>> generated API types (target/generated-sources/xjc). >> >>> >>> As for UI plugins (short term), we can just use restapi-types mappers and >>> implement step b2, (API type -> JSON mapper). >>> >>> Thanks, >>> Vojtech >>> >>> >>> - Original Message - >>> From: "Michael Pasternak" >>> To: "Vojtech Szocs" >>> Cc: "Itamar Heim" , "engine-devel" >>> >>> Sent: Tuesday, November 20, 2012 9:48:27 AM >>> Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here >>> >>> On 11/20/2012 12:13 AM, Itamar Heim wrote: On 11/19/2012 02:07 PM, Vojtech Szocs wrote: > Hi Itamar, > > UI plugin infrastructure translates internal business entities into > JSON-like representations and passes those representations to UI plugins. > (Internal entities are NOT > exposed to UI plugins directly.) > > Currently, all entities supp
Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
Thanks Michael, in that case we can just use XML when talking with REST API.
Also, one idea worth implementing in future is to have some sort of debug
console on GUI (I've seen this idea implemented in other applications). When
activated, users could see REST API XML communication (request/response)
between WebAdmin and Engine, which could be useful for someone interested in
writing his own REST API client.
Vojtech
- Original Message -
From: "Michael Pasternak"
To: "Vojtech Szocs"
Cc: "Itamar Heim" , "engine-devel"
Sent: Wednesday, December 12, 2012 1:33:56 PM
Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
On 12/12/2012 01:51 PM, Vojtech Szocs wrote:
> Hi, sorry for my late response,
>
>>> Our first iteration could be:
>>> a1, rewrite UiCommon (UI business logic) layer to work with API types
>>> a2, write adapter (e.g. modify Frontend class) between UiCommon using API
>>> types and Generic API (GWT RPC) using internal entities
>>
>> does this mean converting three times?
>
> Not sure what you mean, but here's my idea of the first iteration - what
> would happen on frontend side (GWT):
> 1. UiCommon, using API types, invokes query/command via RPC bridge
> (org.ovirt.engine.ui.frontend.Frontend class)
> 2. Frontend class, still using Generic API (GWT RPC), does the actual
> query/command invocation
> 3. Frontend class receives query/command result and translates it from BE
> type to API type (converter callback)
>
> My idea was to isolate BE type usage into Frontend class, so that when we
> decide to use REST API, we just need to update our RPC bridge (Frontend
> class).
>
>>> Our second iteration could be:
>>> b1, remove adapter from step a2, drop Generic API (GWT RPC) usage
>>> b2, write API type <-> JSON mapper, possibly using some existing framework
>>
>> do you mean talking with api in JSON?
>
> Yes, that was my original idea. I'm not against using XML when talking with
> REST API either. GWT supports parsing both JSON and XML.
infra. is there, but officially we not supporting json yet,
(resteasy-json-provider has issues with jaxb, - we waiting for fix)
>
> Vojtech
>
>
> - Original Message -
> From: "Michael Pasternak"
> To: "Vojtech Szocs"
> Cc: "Itamar Heim" , "engine-devel"
> Sent: Sunday, November 25, 2012 9:27:59 AM
> Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
>
> On 11/20/2012 12:57 PM, Vojtech Szocs wrote:
>> Hi Michael,
>>
>>> if we plan moving UI on top of API, you should be:
>>>
>>> 1. importing restapi-types project
>>> 2. writing intermediate layer to translate BE entities to API's using #1
>>> 3. using public entities from #2
>>>
>>> this way your future migration to API (instead of native BE) will be much
>>> more easier.
>>
>> Indeed, this is very useful for GWT RPC -> REST API transition in general,
>> many thanks for pointing this out, Michael. As you suggest, we can use
>> restapi-types mappers to translate between internal entities and API types.
>>
>> Our first iteration could be:
>> a1, rewrite UiCommon (UI business logic) layer to work with API types
>> a2, write adapter (e.g. modify Frontend class) between UiCommon using API
>> types and Generic API (GWT RPC) using internal entities
>
> does this mean converting three times?
>
>>
>> Our second iteration could be:
>> b1, remove adapter from step a2, drop Generic API (GWT RPC) usage
>> b2, write API type <-> JSON mapper, possibly using some existing framework
>
> do you mean talking with api in JSON?
>
>> b3, write RPC layer that talks REST API with Engine
>>
>> However, in order to use REST API types on client (GWT), we need their
>> source code. restapi-interface-definition uses Maven JAXB plugin to generate
>> API types from XSD schema (src/main/resources/api.xsd). On client, we need
>> restapi-definition--sources.jar that includes those generated API
>> types (target/generated-sources/xjc).
>
>>
>> As for UI plugins (short term), we can just use restapi-types mappers and
>> implement step b2, (API type -> JSON mapper).
>>
>> Thanks,
>> Vojtech
>>
>>
>> - Original Message -
>> From: "Michael Pasternak"
>> To: "Vojtech Szocs"
>> Cc: "Itamar Heim" , "engine-devel"
>> Sent: Tuesday, November 20, 2012 9:48:27 AM
>> Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
>>
>> On 11/20/2012 12:13 AM, Itamar Heim wrote:
>>> On 11/19/2012 02:07 PM, Vojtech Szocs wrote:
Hi Itamar,
UI plugin infrastructure translates internal business entities into
JSON-like representations and passes those representations to UI plugins.
(Internal entities are NOT
exposed to UI plugins directly.)
Currently, all entities supported by UI plugin infrastructure (as per
org.ovirt.engine.ui.webadmin.plugin.entity.EntityType) are transformed
into following representation:
{ entityId: "[BusinessEntityGuidAsString]" }
For example, a VM entity with entity ID "
Re: [Engine-devel] UI plugins patches merged, moving out of proof-of-concept stage
Congrats! - Original Message - > From: "Vojtech Szocs" > To: "engine-devel" > Sent: Wednesday, December 12, 2012 2:27:41 PM > Subject: [Engine-devel] UI plugins patches merged,moving out of > proof-of-concept stage > > Hi everyone, > > I'm happy to announce that oVirt UI plugins feature just moved out of > proof-of-concept stage, with relevant patches [1,2] merged into > ovirt-engine git repository. > > I'll be working on related documentation (wiki) in near future, along > with contributing some sample plugins into samples-uiplugins [3] git > repository. > > I'd like to thank everyone involved in pushing this feature forward. > I hope we'll see some interesting plugins soon! > > Regards, > Vojtech > > [1] http://gerrit.ovirt.org/#/c/8120/ > [2] http://gerrit.ovirt.org/#/c/9250/ > [3] git://gerrit.ovirt.org/samples-uiplugins > ___ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel > ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] UI plugins - next steps
Hi guys,
here's my list of tasks that didn't make it into UI plugins yet. These tasks
should be revisited and implemented in future patches. The list isn't sorted in
any way, so feel free to comment or highlight specific tasks to raise their
priority, or add your own tasks/ideas.
1. Pass proper (restapi-definition) entities to UI plugins, instead of simple
"{entityId:}" objects
- using restapi-types mappers to map backend business entities to
restapi-definition entities (Java POJOs generated from api.xsd via JAXB)
- exporting restapi-definition entities for use with JavaScript (UI
plugins), e.g. using gwt-exporter [1] (alternatively, we could use GWT deferred
binding and do this ourselves)
2. showDialog API function should integrate with WebAdmin dialog infrastructure
- shouldn't be too hard, need to create custom dialog PresenterWidget/View
and bind it as non-singleton (just like other popups in oVirt GUI)
3. Extend existing WebAdmin dialogs - prototype Doron's proposal to extend Edit
Cluster Policy dialog with custom options that would be passed to backend
4. Introduce form-based tabs - custom tab that would display key/value pairs
organized in columns, just like in standard "General" sub tabs
5. Introduce table-based tabs - custom tab that would display grid (table) just
like in standard main tabs, with API to add columns, set data, etc.
6. Improve api.addMainTabActionButton to allow different button vs. context
menu item representations (only button, only menu item, both)
7. Add api.currentUserName and api.currentUserId API functions (these would
return same data as with UserLogin event parameters)
8. Minor things: consider renaming UiInit to PluginInit (one-time plugin
initialization, called only once, can do non-UI init stuff there as well),
consider renaming api.ready to api.initialize (plugin indicates that
infrastructure can proceed with initializing itself)
Regards,
Vojtech
[1] http://code.google.com/p/gwt-exporter/wiki/GettingStarted
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] [SOLVED] Re: Failed to start service jboss.deployment.subunit."engine.ear"
turns out there were few issues with dbscripts, commits [1] fixed that. [1] 30d68953b89b5b0a78564075acb9f69f25c4ffc1, 4fe8562e24008c415dda2e046596257586127cf2 On 12/12/2012 01:02 PM, Laszlo Hornyak wrote: > Looks like your DB is destroyed, or JBoss is connecting to the wrong DB. > > - Original Message - >> From: "Michael Pasternak" >> To: "engine-devel" >> Sent: Wednesday, December 12, 2012 11:56:35 AM >> Subject: [Engine-devel] Failed to start service >> jboss.deployment.subunit."engine.ear" >> >> >> any clue what it might be? (latest sources + db) >> >> 2012-12-12 12:51:27,658 INFO [org.ovirt.engine.core.bll.Backend] >> (MSC service thread 1-7) Start time: 12/12/12 12:51 PM >> 2012-12-12 12:51:27,685 ERROR [org.jboss.msc.service.fail] (MSC >> service thread 1-7) MSC1: Failed to start service >> jboss.deployment.subunit."engine.ear"."engine-bll.jar".component.Backend.START: >> org.jboss.msc.service.StartException in service >> jboss.deployment.subunit."engine.ear"."engine-bll.jar".component.Backend.START: >> Failed to start service >> at >> >> org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) >> [jboss-msc-1.0.2.GA.jar:1.0.2.GA] >> at >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >> [rt.jar:1.6.0_20] >> at >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >> [rt.jar:1.6.0_20] >> at java.lang.Thread.run(Thread.java:636) [rt.jar:1.6.0_20] >> Caused by: java.lang.IllegalStateException: JBAS011048: Failed to >> construct component instance >> at >> >> org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:163) >> at >> >> org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) >> at >> >> org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) >> at >> >> org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) >> at >> >> org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) >> at >> >> org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) >> [jboss-msc-1.0.2.GA.jar:1.0.2.GA] >> at >> >> org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) >> [jboss-msc-1.0.2.GA.jar:1.0.2.GA] >> ... 3 more >> Caused by: javax.ejb.EJBException: >> org.springframework.jdbc.BadSqlGrammarException: >> PreparedStatementCallback; bad SQL grammar [select * from >> get_all_commands()]; nested >> exception is org.postgresql.util.PSQLException: ERROR: function >> get_all_commands() does not exist >> Hint: No function matches the given name and argument types. You >> might need to add explicit type casts. >> Position: 16 >> >> >> ... >> >> >> -- >> >> Michael Pasternak >> RedHat, ENG-Virtualization R&D >> ___ >> Engine-devel mailing list >> Engine-devel@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >> -- Michael Pasternak RedHat, ENG-Virtualization R&D ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] upgrade script
Hi all, Please pay attention - when you send patches with upgrade script the prefix should be 03_02__ and not 03_01__ For example - this script was originally 03_01_ once, and it's in correct backend/manager/dbscripts/upgrade/03_02_0020_add_libvirt_version_col.sql Yair ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
On 12/12/2012 01:51 PM, Vojtech Szocs wrote:
> Hi, sorry for my late response,
>
>>> Our first iteration could be:
>>> a1, rewrite UiCommon (UI business logic) layer to work with API types
>>> a2, write adapter (e.g. modify Frontend class) between UiCommon using API
>>> types and Generic API (GWT RPC) using internal entities
>>
>> does this mean converting three times?
>
> Not sure what you mean, but here's my idea of the first iteration - what
> would happen on frontend side (GWT):
> 1. UiCommon, using API types, invokes query/command via RPC bridge
> (org.ovirt.engine.ui.frontend.Frontend class)
> 2. Frontend class, still using Generic API (GWT RPC), does the actual
> query/command invocation
> 3. Frontend class receives query/command result and translates it from BE
> type to API type (converter callback)
>
> My idea was to isolate BE type usage into Frontend class, so that when we
> decide to use REST API, we just need to update our RPC bridge (Frontend
> class).
>
>>> Our second iteration could be:
>>> b1, remove adapter from step a2, drop Generic API (GWT RPC) usage
>>> b2, write API type <-> JSON mapper, possibly using some existing framework
>>
>> do you mean talking with api in JSON?
>
> Yes, that was my original idea. I'm not against using XML when talking with
> REST API either. GWT supports parsing both JSON and XML.
infra. is there, but officially we not supporting json yet,
(resteasy-json-provider has issues with jaxb, - we waiting for fix)
>
> Vojtech
>
>
> - Original Message -
> From: "Michael Pasternak"
> To: "Vojtech Szocs"
> Cc: "Itamar Heim" , "engine-devel"
> Sent: Sunday, November 25, 2012 9:27:59 AM
> Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
>
> On 11/20/2012 12:57 PM, Vojtech Szocs wrote:
>> Hi Michael,
>>
>>> if we plan moving UI on top of API, you should be:
>>>
>>> 1. importing restapi-types project
>>> 2. writing intermediate layer to translate BE entities to API's using #1
>>> 3. using public entities from #2
>>>
>>> this way your future migration to API (instead of native BE) will be much
>>> more easier.
>>
>> Indeed, this is very useful for GWT RPC -> REST API transition in general,
>> many thanks for pointing this out, Michael. As you suggest, we can use
>> restapi-types mappers to translate between internal entities and API types.
>>
>> Our first iteration could be:
>> a1, rewrite UiCommon (UI business logic) layer to work with API types
>> a2, write adapter (e.g. modify Frontend class) between UiCommon using API
>> types and Generic API (GWT RPC) using internal entities
>
> does this mean converting three times?
>
>>
>> Our second iteration could be:
>> b1, remove adapter from step a2, drop Generic API (GWT RPC) usage
>> b2, write API type <-> JSON mapper, possibly using some existing framework
>
> do you mean talking with api in JSON?
>
>> b3, write RPC layer that talks REST API with Engine
>>
>> However, in order to use REST API types on client (GWT), we need their
>> source code. restapi-interface-definition uses Maven JAXB plugin to generate
>> API types from XSD schema (src/main/resources/api.xsd). On client, we need
>> restapi-definition--sources.jar that includes those generated API
>> types (target/generated-sources/xjc).
>
>>
>> As for UI plugins (short term), we can just use restapi-types mappers and
>> implement step b2, (API type -> JSON mapper).
>>
>> Thanks,
>> Vojtech
>>
>>
>> - Original Message -
>> From: "Michael Pasternak"
>> To: "Vojtech Szocs"
>> Cc: "Itamar Heim" , "engine-devel"
>> Sent: Tuesday, November 20, 2012 9:48:27 AM
>> Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
>>
>> On 11/20/2012 12:13 AM, Itamar Heim wrote:
>>> On 11/19/2012 02:07 PM, Vojtech Szocs wrote:
Hi Itamar,
UI plugin infrastructure translates internal business entities into
JSON-like representations and passes those representations to UI plugins.
(Internal entities are NOT
exposed to UI plugins directly.)
Currently, all entities supported by UI plugin infrastructure (as per
org.ovirt.engine.ui.webadmin.plugin.entity.EntityType) are transformed
into following representation:
{ entityId: "[BusinessEntityGuidAsString]" }
For example, a VM entity with entity ID "vm123" will translate to:
{ entityId: "vm123" }
Translation is currently based on
org.ovirt.engine.core.common.businessentities.BusinessEntity interface,
like so: "BusinessEntity" (we expect ID type
parameter to be NGuid-compatible). However, I've found that there are some
entities (like Pool -
org.ovirt.engine.core.common.businessentities.vm_pools) that don't
implement BusinessEntity interface.
>>>
>>> ok, so we only pass the ID for now. good.
>>>
Quick question to backend folks - IIRC all entities extend
org.ovirt.engine.core.common.businessentities.IVdcQueryable, but not all
[Engine-devel] UI plugins patches merged, moving out of proof-of-concept stage
Hi everyone, I'm happy to announce that oVirt UI plugins feature just moved out of proof-of-concept stage, with relevant patches [1,2] merged into ovirt-engine git repository. I'll be working on related documentation (wiki) in near future, along with contributing some sample plugins into samples-uiplugins [3] git repository. I'd like to thank everyone involved in pushing this feature forward. I hope we'll see some interesting plugins soon! Regards, Vojtech [1] http://gerrit.ovirt.org/#/c/8120/ [2] http://gerrit.ovirt.org/#/c/9250/ [3] git://gerrit.ovirt.org/samples-uiplugins ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
Hi, sorry for my late response,
>> Our first iteration could be:
>> a1, rewrite UiCommon (UI business logic) layer to work with API types
>> a2, write adapter (e.g. modify Frontend class) between UiCommon using API
>> types and Generic API (GWT RPC) using internal entities
>
> does this mean converting three times?
Not sure what you mean, but here's my idea of the first iteration - what would
happen on frontend side (GWT):
1. UiCommon, using API types, invokes query/command via RPC bridge
(org.ovirt.engine.ui.frontend.Frontend class)
2. Frontend class, still using Generic API (GWT RPC), does the actual
query/command invocation
3. Frontend class receives query/command result and translates it from BE type
to API type (converter callback)
My idea was to isolate BE type usage into Frontend class, so that when we
decide to use REST API, we just need to update our RPC bridge (Frontend class).
>> Our second iteration could be:
>> b1, remove adapter from step a2, drop Generic API (GWT RPC) usage
>> b2, write API type <-> JSON mapper, possibly using some existing framework
>
> do you mean talking with api in JSON?
Yes, that was my original idea. I'm not against using XML when talking with
REST API either. GWT supports parsing both JSON and XML.
Vojtech
- Original Message -
From: "Michael Pasternak"
To: "Vojtech Szocs"
Cc: "Itamar Heim" , "engine-devel"
Sent: Sunday, November 25, 2012 9:27:59 AM
Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
On 11/20/2012 12:57 PM, Vojtech Szocs wrote:
> Hi Michael,
>
>> if we plan moving UI on top of API, you should be:
>>
>> 1. importing restapi-types project
>> 2. writing intermediate layer to translate BE entities to API's using #1
>> 3. using public entities from #2
>>
>> this way your future migration to API (instead of native BE) will be much
>> more easier.
>
> Indeed, this is very useful for GWT RPC -> REST API transition in general,
> many thanks for pointing this out, Michael. As you suggest, we can use
> restapi-types mappers to translate between internal entities and API types.
>
> Our first iteration could be:
> a1, rewrite UiCommon (UI business logic) layer to work with API types
> a2, write adapter (e.g. modify Frontend class) between UiCommon using API
> types and Generic API (GWT RPC) using internal entities
does this mean converting three times?
>
> Our second iteration could be:
> b1, remove adapter from step a2, drop Generic API (GWT RPC) usage
> b2, write API type <-> JSON mapper, possibly using some existing framework
do you mean talking with api in JSON?
> b3, write RPC layer that talks REST API with Engine
>
> However, in order to use REST API types on client (GWT), we need their source
> code. restapi-interface-definition uses Maven JAXB plugin to generate API
> types from XSD schema (src/main/resources/api.xsd). On client, we need
> restapi-definition--sources.jar that includes those generated API
> types (target/generated-sources/xjc).
>
> As for UI plugins (short term), we can just use restapi-types mappers and
> implement step b2, (API type -> JSON mapper).
>
> Thanks,
> Vojtech
>
>
> - Original Message -
> From: "Michael Pasternak"
> To: "Vojtech Szocs"
> Cc: "Itamar Heim" , "engine-devel"
> Sent: Tuesday, November 20, 2012 9:48:27 AM
> Subject: Re: [Engine-devel] UI Plugins: PoC patch revision 7 is here
>
> On 11/20/2012 12:13 AM, Itamar Heim wrote:
>> On 11/19/2012 02:07 PM, Vojtech Szocs wrote:
>>> Hi Itamar,
>>>
>>> UI plugin infrastructure translates internal business entities into
>>> JSON-like representations and passes those representations to UI plugins.
>>> (Internal entities are NOT
>>> exposed to UI plugins directly.)
>>>
>>> Currently, all entities supported by UI plugin infrastructure (as per
>>> org.ovirt.engine.ui.webadmin.plugin.entity.EntityType) are transformed into
>>> following representation:
>>>
>>> { entityId: "[BusinessEntityGuidAsString]" }
>>>
>>> For example, a VM entity with entity ID "vm123" will translate to:
>>>
>>> { entityId: "vm123" }
>>>
>>> Translation is currently based on
>>> org.ovirt.engine.core.common.businessentities.BusinessEntity interface,
>>> like so: "BusinessEntity" (we expect ID type
>>> parameter to be NGuid-compatible). However, I've found that there are some
>>> entities (like Pool -
>>> org.ovirt.engine.core.common.businessentities.vm_pools) that don't
>>> implement BusinessEntity interface.
>>
>> ok, so we only pass the ID for now. good.
>>
>>>
>>> Quick question to backend folks - IIRC all entities extend
>>> org.ovirt.engine.core.common.businessentities.IVdcQueryable, but not all
>>> entities implement
>>> org.ovirt.engine.core.common.businessentities.BusinessEntity interface.
>>> What is the precise relation between IVdcQueryable and BusinessEntity?
>>>
>>> As for UI plugins, currently all entities get translated to above mentioned
>>> basic JSON-like representation. You can see the relevant code in
>
Re: [Engine-devel] Failed to start service jboss.deployment.subunit."engine.ear"
Looks like your DB is destroyed, or JBoss is connecting to the wrong DB. - Original Message - > From: "Michael Pasternak" > To: "engine-devel" > Sent: Wednesday, December 12, 2012 11:56:35 AM > Subject: [Engine-devel] Failed to start service > jboss.deployment.subunit."engine.ear" > > > any clue what it might be? (latest sources + db) > > 2012-12-12 12:51:27,658 INFO [org.ovirt.engine.core.bll.Backend] > (MSC service thread 1-7) Start time: 12/12/12 12:51 PM > 2012-12-12 12:51:27,685 ERROR [org.jboss.msc.service.fail] (MSC > service thread 1-7) MSC1: Failed to start service > jboss.deployment.subunit."engine.ear"."engine-bll.jar".component.Backend.START: > org.jboss.msc.service.StartException in service > jboss.deployment.subunit."engine.ear"."engine-bll.jar".component.Backend.START: > Failed to start service > at > > org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) > [jboss-msc-1.0.2.GA.jar:1.0.2.GA] > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > [rt.jar:1.6.0_20] > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > [rt.jar:1.6.0_20] > at java.lang.Thread.run(Thread.java:636) [rt.jar:1.6.0_20] > Caused by: java.lang.IllegalStateException: JBAS011048: Failed to > construct component instance > at > > org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:163) > at > > org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) > at > > org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) > at > > org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) > at > > org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) > at > > org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > [jboss-msc-1.0.2.GA.jar:1.0.2.GA] > at > > org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > [jboss-msc-1.0.2.GA.jar:1.0.2.GA] > ... 3 more > Caused by: javax.ejb.EJBException: > org.springframework.jdbc.BadSqlGrammarException: > PreparedStatementCallback; bad SQL grammar [select * from > get_all_commands()]; nested > exception is org.postgresql.util.PSQLException: ERROR: function > get_all_commands() does not exist > Hint: No function matches the given name and argument types. You > might need to add explicit type casts. > Position: 16 > > > ... > > > -- > > Michael Pasternak > RedHat, ENG-Virtualization R&D > ___ > Engine-devel mailing list > Engine-devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel > ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Failed to start service jboss.deployment.subunit."engine.ear"
any clue what it might be? (latest sources + db) 2012-12-12 12:51:27,658 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Start time: 12/12/12 12:51 PM 2012-12-12 12:51:27,685 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-7) MSC1: Failed to start service jboss.deployment.subunit."engine.ear"."engine-bll.jar".component.Backend.START: org.jboss.msc.service.StartException in service jboss.deployment.subunit."engine.ear"."engine-bll.jar".component.Backend.START: Failed to start service at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.2.GA.jar:1.0.2.GA] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.6.0_20] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.6.0_20] at java.lang.Thread.run(Thread.java:636) [rt.jar:1.6.0_20] Caused by: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:163) at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA] at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA] ... 3 more Caused by: javax.ejb.EJBException: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [select * from get_all_commands()]; nested exception is org.postgresql.util.PSQLException: ERROR: function get_all_commands() does not exist Hint: No function matches the given name and argument types. You might need to add explicit type casts. Position: 16 ... -- Michael Pasternak RedHat, ENG-Virtualization R&D ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
