Hi, There seems to be a problem in working with a spring based CXF server-side endpoint in OSGi that can lead to a serious stability issue. I would like to bring up this problem here for discussion so that we can find a solution to it.
We can expect that an endpoint bundle (or a CXF scenario bundle, whatever you call it) does not get automatically stopped when the cxf bundle is stopped, if the scenario bundle is configured using spring instead of blueprint. But whichever configuration is used, the scenario bundle should be releasing all its resources when it is stopped while the cxf bundle is already stopped. However, it appears to be that some resources are not released in this case. And I have been thinking about a way to fix this issue. This problem is only relevant when the cxf-bundle is just stopped without a uninstall/install cycle. Basically, what appears to be happening is, when the cxf bundle is started, it instantiates a DestinationRegistryImpl instance and a CXFNonSpringServlet servlet instance. And they are exposed as OSGi services. When a scenario bundle is started, it instantiates an AbstractHTTPDestination, registers it in the servlet’s destination registry, and stores additional server-side bookkeeping information in the CXFBusLifeCycleManager. When the scenario bundle is stopped, the CXFBusLifeCycleManager executes a series of cleanup actions which are grouped in the preShutdown and postShutdown phases. In the preShutdown phase, the corresponding server instance, ServerImpl, is stopped and this in turn calls the destination’s (AbstractHTTPDestinations’) shutdown method. This method accesses the registry object and removes the corresponding path from it. This registry object is exposed as a proxy to the scenario bundle and accessible while the cxf-bundle is active. When the cxf-bundle is no longer active (e.g., stopped before), the thread accessing this object simply gets blocked and never comes back alive. Consequently, the clean up code placed in the subsequent phases is not executed and this prevents some objects such as the bus and endpoint instances to be released from memory. The destination registry is probably also in some strange state as it goes stale before getting emptied. The very bad side of this problem is that you don’t see any errors in the log nor in the console. You only notice that your console gets slow after a few start/stop cycles. You can observe the manifestation of this problem when you use JMX or some analytical tool. As I mentioned earlier, I have thought about a way to prevent this failed clean up step and tried the following approach that seems to solve this issue. In short, I added the appropriate cleanup code in the destroy method of CXFNonSpringServlet and slightly modified AbstractHTTPDestination. The long version follows. First of all, the destination registry object is held in CXFNonSpringServlet. I think this registry can get cleaned up when this servlet itself is no longer in use (when the cxf-bundle is stopped and the web container unregisters this servlet). Therefore, I added in its destroy method the appropriate cleanup code for its destination registry. And as I needed a way to mark the corresponding destination (AbstractHTTPDestination) as “done/unregistered” so that when the scenario bundle is stopped and the destination object is to be cleaned up, the executing thread does not access the stale registry reference attached to this destination object, I added method releaseRegistry() in AbstractHTTPDestnation to set its registry to null and changed the shutdown method of AbstractHTTPDestination to accesses the registry reference and uses it only when the registry reference is not null. With these changes, I am observing that the scenario bundle can complete its clean up steps even when the cxf-bundle is already stopped, thereby avoiding the problem described. I am not sure if there is some problem with this approach or if there are better approaches. I would appreciate your comments. Thanks. Regards, aki
