Hi all,
We are trying to deploy our large CRM web-application migrated to
Portal(Jetspeed 2.1.3) - to Production, and expierencing a strange
non-regular exception with Portal session (we are using Spring Portlet MVC
also):
Inner exceptions:
org.springframework.web.portlet.handler.PortletSessionRequiredException:
Could not obtain portlet session
Stack trace:
org.springframework.web.portlet.mvc.BaseCommandController.getRenderCommand(BaseCommandController.java:617)
org.springframework.web.portlet.mvc.AbstractFormController.handleRenderRequestInternal(AbstractFormController.java:436)
com.db.gm.sales.aspen.web.portlet.controllers.LoginController.handleRenderRequestInternal(LoginController.java:143)
org.springframework.web.portlet.mvc.AbstractController.handleRenderRequest(AbstractController.java:219)
org.springframework.web.portlet.mvc.SimpleControllerHandlerAdapter.handleRender(SimpleControllerHandlerAdapter.java:52)
org.springframework.web.portlet.DispatcherPortlet.doRenderService(DispatcherPortlet.java:811)
org.springframework.web.portlet.FrameworkPortlet.processRequest(FrameworkPortlet.java:483)
org.springframework.web.portlet.FrameworkPortlet.doDispatch(FrameworkPortlet.java:453)
javax.portlet.GenericPortlet.render(GenericPortlet.java:175)
org.apache.jetspeed.factory.JetspeedPortletInstance.render(JetspeedPortletInstance.java:103)
org.apache.jetspeed.container.JetspeedContainerServlet.doGet(JetspeedContainerServlet.java:277)
javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:594)
org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:505)
org.apache.jetspeed.container.invoker.ServletPortletInvoker.invoke(ServletPortletInvoker.java:273)
org.apache.jetspeed.container.invoker.ServletPortletInvoker.render(ServletPortletInvoker.java:140)
org.apache.pluto.PortletContainerImpl.renderPortlet(PortletContainerImpl.java:119)
org.apache.jetspeed.container.JetspeedPortletContainerWrapper.renderPortlet(JetspeedPortletContainerWrapper.java:121)
org.apache.jetspeed.aggregator.impl.RenderingJobImpl.execute(RenderingJobImpl.java:271)
org.apache.jetspeed.aggregator.impl.PortletRendererImpl.renderNow(PortletRendererImpl.java:228)
org.apache.jetspeed.aggregator.impl.PageAggregatorImpl.aggregateAndRender(PageAggregatorImpl.java:147)
org.apache.jetspeed.aggregator.impl.PageAggregatorImpl.aggregateAndRender(PageAggregatorImpl.java:143)
org.apache.jetspeed.aggregator.impl.PageAggregatorImpl.build(PageAggregatorImpl.java:78)
org.apache.jetspeed.aggregator.AggregatorValve.invoke(AggregatorValve.java:46)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.aggregator.HeaderAggregatorValve.invoke(HeaderAggregatorValve.java:53)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.decoration.DecorationValve.invoke(DecorationValve.java:144)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.resource.ResourceValveImpl.invoke(ResourceValveImpl.java:130)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.pipeline.valve.impl.ActionValveImpl.invoke(ActionValveImpl.java:207)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.container.ContainerValve.invoke(ContainerValve.java:109)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.container.PageHistoryValve.invoke(PageHistoryValve.java:108)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.profiler.impl.ProfilerValveImpl.invoke(ProfilerValveImpl.java:248)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.security.impl.LoginValidationValveImpl.invoke(LoginValidationValveImpl.java:159)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.security.impl.PasswordCredentialValveImpl.invoke(PasswordCredentialValveImpl.java:150)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.localization.impl.LocalizationValveImpl.invoke(LocalizationValveImpl.java:170)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.security.impl.AbstractSecurityValve$1.run(AbstractSecurityValve.java:138)
java.security.AccessController.doPrivileged(Native Method)
javax.security.auth.Subject.doAsPrivileged(Subject.java:454)
org.apache.jetspeed.security.JSSubject.doAsPrivileged(JSSubject.java:179)
org.apache.jetspeed.security.impl.AbstractSecurityValve.invoke(AbstractSecurityValve.java:132)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.container.url.impl.PortalURLValveImpl.invoke(PortalURLValveImpl.java:67)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.capabilities.impl.CapabilityValveImpl.invoke(CapabilityValveImpl.java:126)
org.apache.jetspeed.pipeline.JetspeedPipeline$Invocation.invokeNext(JetspeedPipeline.java:167)
org.apache.jetspeed.pipeline.JetspeedPipeline.invoke(JetspeedPipeline.java:146)
org.apache.jetspeed.engine.JetspeedEngine.service(JetspeedEngine.java:227)
org.apache.jetspeed.engine.JetspeedServlet.doGet(JetspeedServlet.java:242)
javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
org.apache.jetspeed.engine.servlet.XXSUrlAttackFilter.doFilter(XXSUrlAttackFilter.java:52)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
com.db.gm.sales.aspen.web.portlet.filters.SessionCheckFilter.doFilter(SessionCheckFilter.java:139)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
com.db.gm.sales.aspen.web.portlet.filters.SessionCheckFilter.doFilter(SessionCheckFilter.java:139)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
org.apache.catalina.valves.FastCommonAccessLogValve.invoke(FastCommonAccessLogValve.java:482)
org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:393)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
java.lang.Thread.run(Thread.java:595)
Looking at Spring sources, we can see this happens at 617 line
(underscored):
protected final Object getRenderCommand(RenderRequest request) throws
PortletException {
PortletSession session = request.getPortletSession(false);
if (session == null) {
throw new PortletSessionRequiredException("Could
not obtain portlet session");
}
Object command =
session.getAttribute(getRenderCommandSessionAttributeName());
if (command == null) {
throw new PortletSessionRequiredException("Could
not obtain command object from portlet session");
}
return command;
}
This exception happens accidentally at one of the production nodes (we
have a load-balancer there, but without cluster facilities) some time
after Jetspeed start and some load (5-10 simultaneous users), we can't
find an exact test-case to reproduce it. All we can state is that it
happens sometimes between Action and Render phase, when Render phase can't
find session (to get command and errors object from it) after Action phase
is completed.
But after it happened once at one of the node - it comes to inconsistent
state and this exception is thrown at any further request, untill we
restart the cluster.
We are using oracle database for Jetspeed. Spring Portlet MVC - is the
lastest, 2.5.xx version (we had 2.xx earlier and updated but it didn't
help)
So in other words the issue is that Jetspeed deployed to multiple nodes
(just load-balancing and all of them point to the same Jetspeed Oracle
scheme) after start and some load - looses the Portal Session between
Action and Render phases during the same Action request processing, i.e. :
`request.getPortletSession(false)` from the subsequent Render phase -
fails returing null.
We haven't experienced it at our local development environments (i.e. at
single node environment). And we haven't got anything similar at Tomcat
(in production) before when we have our CRM as ordinary web-application,
so it can't be an issue with our code session logic.
Could you investigated it and tell why it can happen? Or what steps do you
suggest to do in order to discover the reason for it?
Thanks,
Regards,
Mikhail
---
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and delete this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this e-mail is strictly forbidden.
Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional
EU corporate and regulatory disclosures.
