Hi Steven, A few things to help your investigations:
As you used to use basic auth, user credentials were sent along with every request, so having a non-sticky load balancer wasn't an issue. However, with form auth, there is a dialog that needs to go on between the client and the server. WIthout a sticky load balancer, each roundtrip of that conversation could occur on a different node, eg: node1 receives original request that decides auth is necessary, does redirect to login page; node 2 serves login page; node 3 receives credentials, does authentication, redirects to the original uri; node 4 receives request for the original uri and checks the authentication. Ideally, the same node would process the whole conversation. If you have multiple simultaneous requests - even going to the same node - with the NullSessionCache, each one will operate on their own copy of the session. As its a race, one request might see the session fully authenticated, another might not. As you have protected "/*" if you have subordinate urls that you have not specifically excluded (I note you've excluded favico etc) , for eg images, the browser might well issue multiple simultaneous requests. When using form authentication, the user name and credentials are serialized into the stored session, but not the UserIdentity. This is re-created when the session is deserialized by calling LoginService.login(name, credentials, null), so by the time the SecurityHandler calls FormAuthenticator.validateRequest(request, response, isAuthMandatory) the UserIdentity should be present. So I'd check to make sure that your LoginService is doing the right thing. Finally, turn on DEBUG for org.eclipse.jetty.security and maybe org.eclipse.jetty.server.session too, and you will see more information about the security client/server dialog that might help you to debug what is going on. On Fri, 20 Aug 2021 at 04:47, Steven Schlansker <[email protected]> wrote: > Hi jetty-users, > > We are trying to replace an existing, working Basic authentication > scheme with the built in Form Jetty authentication coupled with > sessions to improve user experience. > I configured the JDBC session store and got things basically working, > except that sessions have really bad behavior when multiple service > instances are deployed. > I think since we have a load balancer without sticky sessions, the > DefaultSessionCache is interfering. I tried to disable it by calling > server.addBean(new NullSessionCacheFactory()) but this leads me to > users always hitting an exception: > > java.lang.IllegalStateException: !UserIdentity > at > org.eclipse.jetty.security.authentication.SessionAuthentication.getUserIdentity(SessionAuthentication.java:68) > at > org.eclipse.jetty.security.authentication.FormAuthenticator.validateRequest(FormAuthenticator.java:331) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:537) > > I'm not sure how we're triggering this !UserIdentity assertion. With > the default session cache, this exception does show up occasionally, > and I'm not sure exactly why / when. With the null cache it happens > seemingly always. > > We're running embedded Jetty 9.4.43. I'm including my setup code > below, as well as the full stack trace. Thank you in advance for any > advice as to where to go next! > > final var securityHandler = new ConstraintSecurityHandler(); > securityHandler.setLoginService(loginService); > securityHandler.addRole("ws"); > > final var constraintMapping = new ConstraintMapping(); > final var constraint = new Constraint(Constraint.__FORM_AUTH, "ws"); > constraint.setAuthenticate(true); > constraintMapping.setConstraint(constraint); > constraintMapping.setPathSpec("/*"); > > securityHandler.addConstraintMapping(constraintMapping); > > final var noAuth = new Constraint(); > noAuth.setName(Constraint.NONE); > final String loginPath = "/login"; > final String loginErrPath = loginPath + "/error"; > for (final var exclude : new String[] { "/favicon.ico", "/health", > "/health/*", loginPath, loginErrPath }) { > final var noAuthMapping = new ConstraintMapping(); > noAuthMapping.setConstraint(noAuth); > noAuthMapping.setPathSpec(exclude); > securityHandler.addConstraintMapping(noAuthMapping); > } > securityHandler.setHandler(/* webapp context handler */); > > securityHandler.setAuthenticator(new FormAuthenticator(loginPath, > loginErrPath, false)); > > final var sessionHandler = new SessionHandler(); > sessionHandler.setHandler(securityHandler); > final int duration = (int) Duration.ofDays(7).toSeconds(); > sessionHandler.setMaxInactiveInterval(duration); > sessionHandler.new CookieConfig().setMaxAge(duration); > > final var sessionSchema = new SessionTableSchema(); > sessionSchema.setTableName("JettySessions_myapp"); > > final var dbAdapt = new DatabaseAdaptor(); > dbAdapt.setDatasource(ds); > > final var dataStoreFactory = new JDBCSessionDataStoreFactory(); > dataStoreFactory.setGracePeriodSec((int) Duration.ofDays(1).toSeconds()); > dataStoreFactory.setSessionTableSchema(sessionSchema); > dataStoreFactory.setDatabaseAdaptor(dbAdapt); > server.addBean(dataStoreFactory); > > final var sessionIdMgr = new DefaultSessionIdManager(server); > sessionIdMgr.setWorkerName(kubernetesPodId); > server.setSessionIdManager(sessionIdMgr); > > final var sessionCache = new NullSessionCacheFactory(); > sessionCache.setSaveOnCreate(true); > sessionCache.setFlushOnResponseCommit(true); > server.addBean(sessionCache); > > for (final var connector : server.getConnectors()) { > if (connector instanceof HttpConfiguration.ConnectionFactory) { > final var httpConf = ((HttpConfiguration.ConnectionFactory) > connector).getHttpConfiguration(); > httpConf.setSecurePort(443); > httpConf.setSecureScheme("https"); > } > } > > And this is our LoginService: > > public class WSUser implements Principal { > private final UserView userView; > > WSUser(final UserView userView) { > this.userView = userView; > } > > @Override > public String getName() { > return userView.user().name(); > } > > public UserView getUserView() { > return userView; > } > } > > public class WsLoginService implements LoginService { > private static final Logger LOG = > LoggerFactory.getLogger(WsLoginService.class); > > private final Cache<Entry<String, String>, UserIdentity> > loginCache = Caffeine > .newBuilder() > .expireAfterWrite(Duration.ofHours(1)) > .maximumSize(1000) > .build(); > > private final WSFlexLoginManager loginMgr; > private IdentityService identityService = new DefaultIdentityService(); > > @Inject > WsLoginService( > final WSFlexLoginManager loginMgr, > final Provider<UserClient> user, > final Provider<PermissionChecker> checker) { > this.loginMgr = loginMgr; > } > > @Override > public String getName() { > return REALM; > } > > @Override > public UserIdentity login(final String username, final Object > credentials, final ServletRequest request) { > final var password = Objects.toString(credentials); > return loginCache.get( > Map.entry(username, password), > x -> doLogin(username, password)); > } > > private UserIdentity doLogin(final String username, final String > password) { > try { > final LoginResponse resp = loginMgr.login(username, password); > if (!resp.mayLogin()) > { > LOG.warn("login '{}' disallowed" username); > return null; > } > final WSUser userPrincipal = new WSUser(resp.userView()); > final Subject subject = new Subject(true, > Set.of(userPrincipal), Set.of(), Set.of(resp)); > return getIdentityService().newUserIdentity(subject, > userPrincipal, new String[] { "ws" }); > } catch (final Exception e) { > // don't show errors to unauthenticated users > LOG.warn("login '{}' failed", username, e); > return null; > } > } > > @Override > public boolean validate(final UserIdentity identity) { > if (! (identity.getUserPrincipal() instanceof WSUser)) { > return false; > } > final Set<Object> pc = > identity.getSubject().getPrivateCredentials(); > if (pc.size() != 1) { > return false; > } > try { > return loginMgr.validate((LoginResponse) pc.iterator().next()); > } catch (final Exception e) { > LOG.warn("while trying to check session '{}'", > identity.getUserPrincipal().getName(), e); > return false; > } > } > > @Override > public IdentityService getIdentityService() { > return identityService; > } > > @Override > public void setIdentityService(final IdentityService identityService) { > this.identityService = identityService; > } > > @Override > public void logout(final UserIdentity identity) { > // no-op > } > } > > And here's the full error: > > HTTP ERROR 500 java.lang.IllegalStateException: !UserIdentity > > URI:/ > STATUS:500 > MESSAGE:java.lang.IllegalStateException: !UserIdentity > SERVLET:org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-5d7f1e59 > CAUSED BY:java.lang.IllegalStateException: !UserIdentity > > Caused by: > > java.lang.IllegalStateException: !UserIdentity > at > org.eclipse.jetty.security.authentication.SessionAuthentication.getUserIdentity(SessionAuthentication.java:68) > at > org.eclipse.jetty.security.authentication.FormAuthenticator.validateRequest(FormAuthenticator.java:331) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:537) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > com.paywholesail.components.server.metrics.OTInstrumentedHandler.handle(OTInstrumentedHandler.java:278) > at > org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at org.eclipse.jetty.server.Server.handle(Server.java:516) > at > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) > at org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) > at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) > at java.base/java.lang.Thread.run(Thread.java:831) > _______________________________________________ > jetty-users mailing list > [email protected] > To unsubscribe from this list, visit > https://www.eclipse.org/mailman/listinfo/jetty-users > -- Jan Bartel <[email protected]> www.webtide.com *Expert assistance from the creators of Jetty and CometD*
_______________________________________________ jetty-users mailing list [email protected] To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jetty-users
