We are experiencing authentication issues with LDAP since upgrade to 4.5.1.
After some time (...), users can not authenticate anymore, however, authentication in other services using ldap works during this time. The issue is only related to cloudstack login it seems. We haven't found the root cause yet, a network setup issue or openldap config issue can not be excluded. Stacktrace: 2016-02-29 10:05:36,375 DEBUG [cloudstack.ldap.LdapContextFactory] (catalina-exec-4:ctx-9ffa7c60) initializing ldap with provider url: ldap://ldap.example.com:389 2016-02-29 10:05:42,382 DEBUG [cloudstack.ldap.LdapManagerImpl] (catalina-exec-4:ctx-9ffa7c60) ldap Exception: javax.naming.NamingException: LDAP response read timed out, timeout used:6000ms.; remaining name 'dc=foo,dc=bar' at com.sun.jndi.ldap.Connection.readReply(Connection.java:485) at com.sun.jndi.ldap.LdapClient.getSearchReply(LdapClient.java:639) at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:562) at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1985) at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1847) at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1772) at org.apache.cloudstack.ldap.LdapUserManager.searchUsers(LdapUserManager.java:206) at org.apache.cloudstack.ldap.LdapUserManager.getUser(LdapUserManager.java:122) at org.apache.cloudstack.ldap.LdapManagerImpl.getUser(LdapManagerImpl.java:173) at org.apache.cloudstack.ldap.LdapManagerImpl.canAuthenticate(LdapManagerImpl.java:97) at org.apache.cloudstack.ldap.LdapAuthenticator.authenticate(LdapAuthenticator.java:61) 2016-02-29 10:05:42,383 DEBUG [cloudstack.ldap.LdapManagerImpl] (catalina-exec-4:ctx-9ffa7c60) Exception while doing an LDAP bind for user johndoe org.apache.cloudstack.ldap.NoLdapUserMatchingQueryException: No users matching: No Ldap User found for username: johndoe As I understand there is a username lookup (bind with top reader credentials) to see if a user exists in the ldap. if found a new connection will be etablished for auth. In the above stacktrace it seem that the username lookup fails. Further we see on the ACS management server however, is that LDAP connection are not going to be closed at any time. For _every_ successful auth, the tcp connection remains established forever. In my understanding of http://docs.oracle.com/javase/jndi/tutorial/ldap/connect/config.html these connections will become idle after successful authentication and reused for new authentication. However, the reuse for the auth doesn't seem to work. _Every_ new successful auth of a user _creates_ a new ldap connection. We don't know if this is related to our problem, but at least it doesn't look like a wanted behavior. In the docs we read: "By default, idle connections remain in the pool indefinitely until they are garbage-collected" But as said, they seem never be gc-ed. After we added -Dcom.sun.jndi.ldap.connect.pool.timeout=60000 to the /etc/cloudstack/management/tomcat6.conf which resulted in the connections beeing gc-ed and we didn't have any report about missing login since then. Has anyone also see such an issue? Any thoughts? René
