Larry, Thank you. The reason we want to filter by group in authentication level is to avoid performance, we have 1000s of groups.
I will also try with Authorization. Hoping to have memberOf working within authentication. Thank you again for suggestions. Regards Raja From: larry mccay <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Tuesday, November 27, 2018 at 8:56 AM To: "[email protected]" <[email protected]> Subject: Re: Knox LDAP group filer is not working Hi Raja - I need to better understand why you have a need to do the filtering within the authentication provider. This is more easily done within the authorization provider and leaves you with more options for doing group lookup. At some point, Instead of doing it from LDAP you may want to use local OS account groups for instance. My preference these days is to keep the authentication provider config very simple to establish the authenticated user, use Hadoop Group Lookup Provider [1] to look up groups exactly the way is done within Hadoop itself. Then use the AclsAuthzProvider to provide service level authorization based on group membership, username and/or ip address [2]. By decoupling these things, you have the most flexibility to compose and even extend providers to meet your needs. thanks, --larry 1. https://knox.apache.org/books/knox-1-1-0/user-guide.html#Hadoop+Group+Lookup+Provider 2. https://knox.apache.org/books/knox-1-1-0/user-guide.html#Authorization On Tue, Nov 27, 2018 at 8:45 AM Kevin Risden <[email protected]<mailto:[email protected]>> wrote: The username you used to login doesn't match the regex you gave. <param name="main.ldapRealm.principalRegex" value="(.*?)\\(.*?)"/> If you are going to only specify the username you need to modify the regex to only have one capture group. Kevin Risden On Mon, Nov 26, 2018 at 9:08 PM Raja Marimuthu <[email protected]<mailto:[email protected]>> wrote: Kevin, Thank you so much. When I have tried with prinicipalrgeex, I get below error User DN : CN=Len,OU=US02P01,OU=mmc_users,DC=ds,DC=nb,DC=com Group DN : CN=m_powerusers,OU=Applications,OU=Groups,DC=ds,DC=nb,DC=com I need to provide. Access only to m_powerusers. group Setting 1: <param name="main.ldapRealm.userSearchBase" value="dc=ds,dc=nb,dc=com"/> <param name="main.ldapRealm.principalRegex" value="(.*?)\\(.*?)"/> <param name="main.ldapRealm.userSearchFilter" value="(&(objectclass=person)(memberOf=cn={1},OU=Applications,OU=Groups,DC=ds,DC=nb,DC=com)(sAMAccountName={2}))"/> <param> Setting 2 : <param name="main.ldapRealm.userSearchBase" value="dc=ds,dc=nb,dc=com"/> <param name="main.ldapRealm.principalRegex" value="(.*?)\\(.*?)"/> <param name="main.ldapRealm.userSearchFilter" value="(&(objectclass=person)(memberOf=cn=m_powerusers,OU=Applications,OU=Groups,DC=ds,DC=nb,DC=com)(sAMAccountName={2}))"/> <param> User DN : CN=Len,OU=US02P01,OU=mmc_users,DC=ds,DC=nb,DC=com Group DN : CN=m_powerusers,OU=Applications,OU=Groups,DC=ds,DC=nb,DC=com Error : 2018-11-27 02:02:13,678 WARN authc.AbstractAuthenticator (AbstractAuthenticator.java:authenticate(216)) - Authentication failed for token submission [org.apache.shiro.authc.UsernamePasswordToken - len, rememberMe=false (73.230.13.102)]. Possible unexpected error? (Typical or expected login exceptions should extend from AuthenticationException). java.lang.IllegalArgumentException: Principal len does not match (.*?)\\(.*?) at org.apache.knox.gateway.shirorealm.KnoxLdapRealm.matchPrincipal(KnoxLdapRealm.java:658) at org.apache.knox.gateway.shirorealm.KnoxLdapRealm.getUserDn(KnoxLdapRealm.java:681) at org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm.getUserDn(KnoxLdapRealm.java:98) at org.apache.shiro.realm.ldap.JndiLdapRealm.getLdapPrincipal(JndiLdapRealm.java:342) at org.apache.shiro.realm.ldap.JndiLdapRealm.queryForAuthenticationInfo(JndiLdapRealm.java:371) at org.apache.shiro.realm.ldap.JndiLdapRealm.doGetAuthenticationInfo(JndiLdapRealm.java:295) at org.apache.knox.gateway.shirorealm.KnoxLdapRealm.doGetAuthenticationInfo(KnoxLdapRealm.java:200) at org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm.doGetAuthenticationInfo(KnoxLdapRealm.java:54) at org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:568) at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doSingleRealmAuthentication(ModularRealmAuthenticator.java:180) at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:267) at org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198) at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106) at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:270) at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:256) at org.apache.shiro.web.filter.authc.AuthenticatingFilter.executeLogin(AuthenticatingFilter.java:53) at org.apache.shiro.web.filter.authc.BasicHttpAuthenticationFilter.onAccessDenied(BasicHttpAuthenticationFilter.java:190) at org.apache.shiro.web.filter.AccessControlFilter.onAccessDenied(AccessControlFilter.java:133) at org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(AccessControlFilter.java:162) at org.apache.shiro.web.filter.PathMatchingFilter.isFilterChainContinued(PathMatchingFilter.java:203) at org.apache.shiro.web.filter.PathMatchingFilter.preHandle(PathMatchingFilter.java:178) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:131) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) at org.apache.knox.gateway.filter.ResponseCookieFilter.doFilter(ResponseCookieFilter.java:50) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:171) at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:94) at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:141) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:201) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41) at org.eclipse.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:152) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:748) 2018-11-27 02:02:13,688 DEBUG servlet.SimpleCookie (SimpleCookie.java:addCookieHeader(226)) - Added HttpServletResponse Cookie [rememberMe=deleteMe; Path=/gateway/gf; Max-Age=0; Expires=Mon, 26-Nov-2018 02:02:13 GMT] 2018-11-27 02:02:13,688 DEBUG authc.BasicHttpAuthenticationFilter (BasicHttpAuthenticationFilter.java:sendChallenge(274)) - Authentication required: sending 401 Authentication challenge response. From: Kevin Risden <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, November 26, 2018 at 7:22 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Knox LDAP group filer is not working >From [1], the userSearchFilter needs to have a reference to the user who is >logged in. Basically what you are trying to do with the userSearchFilter is >only allow the user to login if the user matches the query. When you do the >search filter without a reference to the user who is trying to login you are >basically just grabbing all users that match the query. This is not what you >want. The principalRegex has capture groups that you can use in the >userSearchFilter to build out the query to match for the given username and >filter. 1. https://knox.apache.org/books/knox-1-1-0/user-guide.html#Advanced+LDAP+configuration+parameters Kevin Risden On Mon, Nov 26, 2018 at 3:53 PM Raja Marimuthu <[email protected]<mailto:[email protected]>> wrote: Anyone have experienced this issue ? Using LDAP group filter with AD ? Logged in user is different, but it’s. taking first user from the group as computer userDN , and throws. Null pointer . Raja Marimuthu | Solutions Architect (AWS – Big Data) NorthBay Solutions Direct: 717-808-6966 [email protected]<mailto:[email protected]> www.northbaysolutions.com<http://www.northbaysolutions.com/> From: Raja Marimuthu <[email protected]<mailto:[email protected]>> Date: Wednesday, November 14, 2018 at 4:37 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Knox LDAP group filer is not working Kevin, I have setup AD and configured in gateway xml, but. I am having this issue… - 2018-11-14 21:08:26,993 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /ganglia/ 2018-11-14 21:08:27,016 DEBUG knox.gateway (KnoxLdapRealm.java:getUserDn(718)) - Searching from dc=ds,dc=nb,dc=com where (&(objectclass=*)(memberOf=cn=marsh-prd-global-bld-powerusers,OU=Applications,OU=Groups,DC=ds,DC=nb,DC=com)) scope subtree 2018-11-14 21:08:27,022 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(724)) - Computed userDn: CN=Len,OU=US02P01,OU=mmc_users,DC=ds,DC=nb,DC=com using ldapSearch for principal: len 2018-11-14 21:08:27,045 ERROR knox.gateway (AbstractGatewayFilter.java:doFilter(66)) - Failed to execute filter: javax.servlet.ServletException: java.lang.NullPointerException javax.servlet.ServletException: java.lang.NullPointerException at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196) at org.apache.shiro.web.filter.authc.AuthenticatingFilter.cleanup(AuthenticatingFilter.java:155) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) Thanks Raja From: Kevin Risden <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, November 8, 2018 at 5:41 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Knox LDAP group filer is not working If you are using the demo LDAP server then memberOf isn't available. It is an LDAP extension that only exists in AD. Apache DS LDAP doesn't support virtual attributes. If you want to emulate it you would need to update the user object with the attributes. Kevin Risden On Thu, Nov 8, 2018 at 5:38 PM Raja Marimuthu <[email protected]<mailto:[email protected]>> wrote: Kevin, I have tried. But its. Not working, Here’s my gateway xml LDAP config <param name="main.ldapRealm" value="org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm" /> <param name="main.ldapContextFactory" value="org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory" /> <param name="main.ldapRealm.contextFactory" value="$ldapContextFactory" /> <param> <name>main.ldapRealm.authorizationEnabled</name> <value>true</value> </param> <param name="main.ldapRealm.contextFactory.url" value="ldap://localhost:33389"/> <param name="main.ldapRealm.contextFactory.systemUsername" value="uid=admin,ou=people,dc=hadoop,dc=apache,dc=org"/> <param name="main.ldapRealm.contextFactory.systemPassword" value="admin-password"/> <param name="main.ldapRealm.userSearchBase" value="ou=people,dc=hadoop,dc=apache,dc=org"/> <param name="main.ldapRealm.userSearchFilter" value="(&(objectclass=person)(sAMAccountName={2})(|(memberOf=cn=contractor,dc=hadoop,dc=apache,dc=org)(memberOf=cn=scientist,ou=grouds,dc=hadoop,dc=apache,dc=org))"/> <param name="main.ldapRealm.userObjectClass" value="person"/> users.ldif # Please replace with site specific values dn: dc=hadoop,dc=apache,dc=org objectclass: organization objectclass: dcObject o: Hadoop dc: hadoop # Entry for a sample people container # Please replace with site specific values dn: ou=people,dc=hadoop,dc=apache,dc=org objectclass:top objectclass:organizationalUnit ou: people # Entry for a sample contractor container # Please replace with site specific values dn: ou=contractor,dc=hadoop,dc=apache,dc=org objectclass:top objectclass:organizationalUnit ou: contractor # entry for sample user jerry dn: uid=jerry,ou=contractor,dc=hadoop,dc=apache,dc=org objectclass:top objectclass:person objectclass:organizationalPerson objectclass:inetOrgPerson cn: jerry sn: jerry uid: jerry userPassword:jerry-password # entry for sample user sam dn: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org objectclass:top objectclass:person objectclass:organizationalPerson objectclass:inetOrgPerson cn: sam sn: sam uid: sam userPassword:sam-password # entry for sample user tom dn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org objectclass:top objectclass:person objectclass:organizationalPerson objectclass:inetOrgPerson cn: tom sn: tom uid: tom userPassword:tom-password # create FIRST Level groups branch dn: ou=groups,dc=hadoop,dc=apache,dc=org objectclass:top objectclass:organizationalUnit ou: groups description: generic groups branch # create the analyst group under groups dn: cn=analyst,ou=groups,dc=hadoop,dc=apache,dc=org objectclass:top objectclass: groupofnames cn: analyst description:analyst group member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org member: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org # create the scientist group under groups dn: cn=scientist,ou=groups,dc=hadoop,dc=apache,dc=org objectclass:top objectclass: groupofnames cn: scientist description: scientist group member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org On Nov 7, 2018, at 4:45 PM, Kevin Risden <[email protected]<mailto:[email protected]>> wrote: Assuming you are referring to something like KNOX-1307 [1]? The user search filter you can create can filter by groups depending on what you are trying to do. memberOf is one way for AD to limit users to only ones in a certain group. 1. https://issues.apache.org/jira/browse/KNOX-1307 Kevin Risden On Wed, Nov 7, 2018 at 4:24 PM Raja Marimuthu <[email protected]<mailto:[email protected]>> wrote: Hi, We are trying to filter users by specific. LDAP groups, tried several options provided in the documentation : https://knox.apache.org/books/knox-1-1-0/user-guide.html#Advanced+LDAP+Authentication User Search by Filter • userSearchBase (Required) • userSearchFilter (Required) • userSearchScope (Optional) • principalRegex (Optional) Group filter is supported ? Do we have any working alternative to filter. Users by group ? Thanks Raja
