One more strange thing I noticed. I started Servicemix up, everything deployed fine. I then had to shut it down and restart it. when I did it started undeploying some of my components and SAs. I noticed some debug messages like this:
21:25:33,671 | DEBUG | xtenderThread-28 | XBeanNamespaceHandler | ontext.v2c.XBeanNamespaceHandler 839 | Could not find resource: META-INF/services/org/apache/xbean/spring/http/ service.com/definitions/taskDescription I see the reference that it's talking about in one of the jars in the SE component that is deployed. Though, it's part defined in a different jar than the jar that is created for the SE. Is that now a limitation in Servicemix 4.0? I didn't have any issue like this in Servicemix 3.3.1. I'm thoroughly confused. That looks to be the only thing I can find that remotely looks like an error. On Tue, Sep 1, 2009 at 8:52 PM, Ryan Moquin <[email protected]> wrote: > Just for some extra info, on that last run where only some stuff started, > for one of the SAs that only made it to the resolved state, this is the last > bit of the log that mentions that SA: > > 19:33:26,937 | DEBUG | pool-1-thread-1 | Activator > | org.apache.camel.osgi.Activator 73 | Bundle resolved: > tasking-service-sa > 19:33:26,937 | DEBUG | lixDispatchQueue | tasking-service-sa | > ? ? | BundleEvent RESOLVED > 19:33:26,937 | DEBUG | lixDispatchQueue | tasking-service-sa | > ? ? | BundleEvent INSTALLED > > It doesn't mention that SA anywhere else in the log after that. > > > On Tue, Sep 1, 2009 at 7:41 PM, Ryan Moquin <[email protected]>wrote: > >> Also, I would say the most consistent situation I seem to hit is where all >> of the Servicemix components install, my components install, and only 2 or 3 >> of my SAs install and the other 4 or 5 don't install. >> >> I didn't notice you said osgi/list, I was doing jbi/list. I just deleted >> the data directory and restarted, I see all my stuff as listed Active in >> osgi/list and the Spring column is blank. jbi/list shows all my stuff as >> started. I'll go ahead and try it again since it decided to deploy >> everything this time. >> >> This time, I deleted the data directory and started servicemix back up. >> This time when it stops deploying things, all 4 of my components show as >> Active, 5 of my SAs show as Active and 3 of my SAs show as resolved. If I >> look at jbi/list, I see all 4 of my components and only 2 SAs out of the 8 >> total show as started. The other 6 aren't even listed. This time when I >> was doing it, I was emptying my recycle bin. It seems to me like the >> majority of the time when not everything finishes, some other process was >> working on my machine as well. Could some of the deployments be timing >> out? It's been sitting for a while and none of the statuses have changed. >> >> I'm not really sure what's going on but it seems like some of the >> components just get abandoned ... >> >> Ryan >> >> >> On Tue, Sep 1, 2009 at 7:13 PM, Ryan Moquin <[email protected]>wrote: >> >>> I didn't set it to debug, since I figured if there was an error it would >>> tell me but like I said, it non-deterministically will sometimes only start >>> 4 total components and no SA's or it will start all of them... or it will >>> start any number in between. If I delete the data directory and restart, >>> then a different number of components/SAs will start without me modifying >>> anything other than deleting the data directory. >>> >>> The worst case I've seen so far, was 4 of my components all started and >>> non of the servicemix components started. I was getting all kinds of weird >>> activemq errors about not being able to connect to 0.0.0. I should have >>> saved some of the log. I was getting weird JAXB errors as well.. basically >>> everything went nuts. I then deleted the data directory and everything >>> started up without any issues. >>> >>> Anyhow, I turned on debug, deleted the data directory and started up. 3 >>> of my SA's didn't start, but all of my components and servicemix components >>> started. Now, I don't see any errors, this is a snippet from the log >>> regarding one of my SAs which rely on servicemix-cxf-se (which hadn't been >>> installed as of this point): >>> >>> 18:38:21,828 | DEBUG | pool-1-thread-1 | >>> BundleWatcher | .swissbox.extender.BundleWatcher 176 | >>> Scanning bundle [process-service-sa] >>> 18:38:21,828 | DEBUG | pool-1-thread-1 | >>> Deployer | cemix.jbi.deployer.impl.Deployer 301 | >>> Checking bundle: 'null (process-service-sa)' >>> 18:38:21,859 | INFO | pool-1-thread-1 | >>> Deployer | cemix.jbi.deployer.impl.Deployer 343 | >>> Deploying bundle 'null (process-service-sa)' as a JBI service assembly >>> 18:38:21,906 | WARN | pool-1-thread-1 | >>> Deployer | cemix.jbi.deployer.impl.Deployer 359 | >>> Requirements not met for JBI artifact in bundle null (process-service-sa). >>> Installation pending. >>> org.apache.servicemix.jbi.deployer.impl.PendingException: Component not >>> installed: servicemix-cxf-se >>> 18:38:21,921 | DEBUG | pool-1-thread-1 | >>> ContextLoaderListener | .activator.ContextLoaderListener 709 | >>> Scanning bundle [null (process-service-sa)] for configurations... >>> 18:38:21,921 | DEBUG | pool-1-thread-1 | >>> ContextLoaderListener | .activator.ContextLoaderListener 715 | >>> Creating an application context for bundle [null (process-service-sa)] >>> 18:38:21,921 | DEBUG | pool-1-thread-1 | >>> ContextLoaderListener | .activator.ContextLoaderListener 726 | >>> No application context created for bundle [null (process-service-sa)] >>> 18:38:21,921 | INFO | pool-1-thread-1 | >>> FileMonitor | x.kernel.filemonitor.FileMonitor 550 | >>> Started: process-service-sa [119] >>> 18:38:21,921 | DEBUG | lixDispatchQueue | >>> process-service-sa | ? ? >>> | BundleEvent STARTED >>> >>> After this point in the log, there isn't a single mention of that SA >>> again anywhere in the rest of the log.... About 4000 lines down from the >>> above segment in the log file, the servicemix-cxf-se component bundle gets >>> deployed, but it never finishes installing process-service-sa. >>> >>> 18:38:29,343 | DEBUG | Thread-5 | >>> BundleWatcher | .swissbox.extender.BundleWatcher 176 | >>> Scanning bundle [servicemix-cxf-se] >>> 18:38:29,359 | DEBUG | Thread-5 | >>> Deployer | cemix.jbi.deployer.impl.Deployer 301 | >>> Checking bundle: 'ServiceMix :: CXF Service Engine (servicemix-cxf-se)' >>> 18:38:29,359 | DEBUG | Thread-5 | >>> Deployer | cemix.jbi.deployer.impl.Deployer 304 | >>> Bundle 'ServiceMix :: CXF Service Engine (servicemix-cxf-se)' does not >>> contain any JBI descriptor. >>> 18:38:29,359 | DEBUG | Thread-5 | >>> ContextLoaderListener | .activator.ContextLoaderListener 709 | >>> Scanning bundle [ServiceMix :: CXF Service Engine (servicemix-cxf-se)] for >>> configurations... >>> 18:38:29,359 | DEBUG | Thread-5 | >>> ContextLoaderListener | .activator.ContextLoaderListener 715 | >>> Creating an application context for bundle [ServiceMix :: CXF Service Engine >>> (servicemix-cxf-se)] >>> 18:38:29,359 | INFO | Thread-5 | >>> ultOsgiApplicationContextCreator | ultOsgiApplicationContextCreator 67 | >>> Discovered configurations {osgibundle:/META-INF/spring/*.xml} in bundle >>> [ServiceMix :: CXF Service Engine (servicemix-cxf-se)] >>> 18:38:29,359 | DEBUG | Thread-5 | >>> ContextLoaderListener | .activator.ContextLoaderListener 735 | >>> Bundle [ServiceMix :: CXF Service Engine (servicemix-cxf-se)] is Spring type >>> compatible with Spring-DM >>> 18:38:29,359 | DEBUG | Thread-5 | >>> ContextLoaderListener | .activator.ContextLoaderListener 784 | >>> Asynchronous context creation for bundle [ServiceMix :: CXF Service Engine >>> (servicemix-cxf-se)] >>> 18:38:29,359 | DEBUG | lixDispatchQueue | >>> servicemix-cxf-se | ? ? | >>> BundleEvent STARTED >>> 18:38:29,359 | DEBUG | xtenderThread-49 | >>> WaiterApplicationContextExecutor | WaiterApplicationContextExecutor 162 | >>> Starting first stage of refresh for >>> OsgiBundleXmlApplicationContext(bundle=servicemix-cxf-se, >>> config=osgibundle:/META-INF/spring/*.xml) >>> 18:38:29,359 | DEBUG | xtenderThread-49 | >>> WaiterApplicationContextExecutor | WaiterApplicationContextExecutor 202 | >>> Calling preRefresh on >>> .... >>> >>> Sometimes rather than just not being resumed once the dependencies are >>> installed, sometimes the SA is installed correctly but later is says it's >>> undeploying it. >>> >>> About 9000 more lines down in the log, it finishes up installing another >>> one of my SAs. This one starts deploying after the cxf-se, which may be the >>> reason it actually goes through with installing it. I don't see any errors >>> installing any of the SUs in the SA, and it appears that it finishes >>> installing it but I don't see it listed in the console with jbi/list. Other >>> SAs of mine install but don't show up in the console either. >>> >>> This is the last bit of the log where it creates the service for my SA >>> after processing all of the SU's. After that I don't see anything about >>> that SA anymore, or anything saying it's started... >>> >>> 18:39:00,343 | INFO | pool-1-thread-1 | >>> ReflectionServiceFactoryBean | .support.JaxWsServiceFactoryBean 163 | >>> Creating Service >>> {http://service.com/salina/workflow}workflow-service<http://service.com/salina/workflow%7Dworkflow-service>from >>> class com.service.WorkflowServicePort >>> 18:39:00,343 | DEBUG | xtenderThread-64 | >>> SaxonComponent | icemix.common.AsyncBaseLifeCycle 296 | >>> Starting component >>> 18:39:00,375 | DEBUG | xtenderThread-64 | >>> SaxonComponent | icemix.common.AsyncBaseLifeCycle 304 | >>> Component started >>> >>> It just seems like SAs just randomly die or never get installed. If I >>> stop servicemix, delete the data directory, restart, then sometimes >>> everything will fully install... I don't see any errors or any reason that >>> those SAs don't install and like I said it's not consistent on what it >>> installs or doesn't install. Sometimes it doesn't install all the >>> Servicemix components either. >>> >>> >>> On Tue, Sep 1, 2009 at 12:42 AM, Gert Vanthienen < >>> [email protected]> wrote: >>> >>>> Ryan, >>>> >>>> I guess you already enabled DEBUG logging for this? What does >>>> osgi/list say about the bundles/SA/XML files you are using? Does it >>>> mark the bundles as Active -- if not, something might have gone wrong >>>> at file install time? >>>> >>>> Regards, >>>> >>>> Gert Vanthienen >>>> ------------------------ >>>> Open Source SOA: http://fusesource.com >>>> Blog: http://gertvanthienen.blogspot.com/ >>>> >>>> >>>> >>>> 2009/9/1 Ryan Moquin <[email protected]>: >>>> > I wanted to reask about SAs in Servicemix 4.0. When I start >>>> Servicemix 4.0 >>>> > fresh (no data directory), certain SAs that it deploys, will end up in >>>> the >>>> > stopped state. There are no errors, and it seems to be random. If I >>>> stop >>>> > Servicemix, delete the data directory and start it back up, then >>>> different >>>> > SAs will be stopped, or sometimes they'll all start up and be fine. >>>> It >>>> > appears that if they do all start up, then they seem to start >>>> everytime if I >>>> > restart the server. It just seems like on a fresh start, sometimes an >>>> SA >>>> > won't start up properly. Is there anyway to find out the reason why >>>> an SA >>>> > doesn't start? I don't see anything in the log, is there something I >>>> can >>>> > turn on to see what might be causing the intermittent problem? >>>> > >>>> > If I try to access an external endpoint on one of the SAs that don't >>>> show up >>>> > in jbi/list (such as a CXF endpoint), I'll see this error (it's the >>>> only >>>> > error I see). >>>> > >>>> > Interceptor has thrown exception, unwinding now >>>> > org.apache.cxf.interceptor.Fault: Endpoint is stopped >>>> > at >>>> > >>>> org.apache.servicemix.cxfbc.CxfBcConsumer$1.handleMessage(CxfBcConsumer.java:440) >>>> > at >>>> > >>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:236) >>>> > at >>>> > >>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:89) >>>> > at >>>> > >>>> org.apache.servicemix.cxfbc.CxfBcConsumer$JbiChainInitiationObserver.onMessage(CxfBcConsumer.java:646) >>>> > at >>>> > >>>> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:302) >>>> > at >>>> > >>>> org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:266) >>>> > at >>>> > >>>> org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:70) >>>> > at >>>> > >>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) >>>> > at >>>> > >>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) >>>> > at >>>> org.mortbay.jetty.handler.HandlerList.handle(HandlerList.java:49) >>>> > at >>>> > >>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) >>>> > at org.mortbay.jetty.Server.handle(Server.java:324) >>>> > at >>>> > >>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) >>>> > at >>>> > >>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864) >>>> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) >>>> > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) >>>> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) >>>> > at >>>> > >>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) >>>> > at >>>> > >>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522) >>>> > Caused by: java.lang.Exception: Endpoint is stopped >>>> > ... 19 more >>>> > >>>> >>> >>> >> >
