You are right Robert, the error is seen at shutdown, even if it's similar. I continued doing a bit of debugging to understand how sling is loading up the different resources and had a quesiton: do configurations from the repoinit files get installed in a specific order with relation to the artifacts? do they get installed in different threads? This might idicate if we are seeing a slow thread in there. Thanks again for all the help. Carlos
On Thu, Jan 30, 2020 at 4:47 AM Robert Munteanu <romb...@apache.org> wrote: > Hi Carlos, > > Yes, this may be a timing issue. > > I could not follow the link you sent me for some reason. I think the > build log is the one from [1]. If that is the case, the error is > visible at shutdown, and probably does not have the same root cause. > > I'd still like to get some steps to reproduce - even if it's a change > of 1 in 3, even if it depends on containers. > > Thanks, > Robert > > > [1]: > https://builds.apache.org/job/Sling/job/sling-org-apache-sling-starter/job/master/104/console > > On Wed, 2020-01-29 at 21:27 -0500, Carlos Munoz wrote: > > Robert, I checked the latest (master) pipeline build logs for the > > starter > > project: > > > > > https://builds.apache.org/blue/organizations/jenkins/Sling%2Fsling-org-apache-sling-starter/detail/master/104/pipeline/24 > > > > and found that there is a very similar error being reported > > (different > > principal and bundle), but same type of exception nonetheless. > > > > Carlos > > > > > > On Wed, Jan 29, 2020 at 7:47 PM Carlos Munoz <camu...@redhat.com> > > wrote: > > > > > Robert, I wonder if this is a timing issue. I’m not sure I > > > understand how > > > Sling is loading bundles and configurations, but is it possible > > > that it > > > could load up a bundle which needs a specific configuration before > > > said > > > configuration has finished loading? > > > > > > I mention this because we are seeing the error now on a > > > containerized > > > environment where resources may be more virtualized than in a local > > > environment, where the application seems to run without any issues. > > > > > > Regards, > > > > > > Carlos > > > > > > On Tue, Jan 28, 2020 at 10:11 PM Carlos Munoz <camu...@redhat.com> > > > wrote: > > > > > > > Hi Robert, I'm picking up this thread again since we briefly > > > > talked about > > > > this problem; allow me to recap: > > > > We are attempting to migrate bundle versions for a Sling > > > > application from > > > > their Sling 11 versions to the latest stable versions. The > > > > application is > > > > running against an already populated mongo database and we are > > > > seeing the > > > > following exception when deploying. > > > > > > > > 29.01.2020 02:58:59.571 *ERROR* [Apache Sling Repository Startup > > > > Thread > > > > #4] ERROR: Bundle '160' EventDispatcher: Error during dispatch. > > > > (org.apache.sling.api.SlingException: Can't create the JCR event > > > > listener.) > > > > org.apache.sling.api.SlingException: Can't create the JCR event > > > > listener. > > > > at > > > > org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProv > > > > ider.registerListeners(JcrResourceProvider.java:227) > > > > > > > > at > > > > org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProv > > > > ider.start(JcrResourceProvider.java:182) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Handler.activate(ResourceProviderHandler.java:74) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker.activate(ResourceProviderTracker.java:360) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker.register(ResourceProviderTracker.java:192) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker.access$200(ResourceProviderTracker.java:59) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker$1.addingService(ResourceProviderTracker.java:130) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker$1.addingService(ResourceProviderTracker.java:106) > > > > > > > > at > > > > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(Ser > > > > viceTracker.java:943) > > > > > > > > at > > > > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(Ser > > > > viceTracker.java:871) > > > > > > > > at > > > > org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked > > > > .java:256) > > > > at > > > > org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java: > > > > 229) > > > > at > > > > org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(Servi > > > > ceTracker.java:903) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.invokeServiceListenerC > > > > allback(EventDispatcher.java:990) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireEventImmediately(E > > > > ventDispatcher.java:838) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireServiceEvent(Event > > > > Dispatcher.java:545) > > > > > > > > at > > > > org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4833 > > > > ) > > > > at > > > > org.apache.felix.framework.Felix.registerService(Felix.java:3804) > > > > at > > > > org.apache.felix.framework.BundleContextImpl.registerService(Bund > > > > leContextImpl.java:328) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager$3.regi > > > > ster(AbstractComponentManager.java:906) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager$3.regi > > > > ster(AbstractComponentManager.java:892) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.RegistrationManager.changeRegis > > > > tration(RegistrationManager.java:128) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager.regist > > > > erService(AbstractComponentManager.java:959) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager.activa > > > > teInternal(AbstractComponentManager.java:732) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticC > > > > ustomizer.addedService(DependencyManager.java:1045) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticC > > > > ustomizer.addedService(DependencyManager.java:999) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customiz > > > > erAdded(ServiceTracker.java:1216) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customiz > > > > erAdded(ServiceTracker.java:1137) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked. > > > > trackAdding(ServiceTracker.java:944) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked. > > > > track(ServiceTracker.java:880) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceC > > > > hanged(ServiceTracker.java:1168) > > > > > > > > at > > > > org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.s > > > > erviceChanged(BundleComponentActivator.java:125) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.invokeServiceListenerC > > > > allback(EventDispatcher.java:990) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireEventImmediately(E > > > > ventDispatcher.java:838) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireServiceEvent(Event > > > > Dispatcher.java:545) > > > > > > > > at > > > > org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4833 > > > > ) > > > > at > > > > org.apache.felix.framework.Felix.registerService(Felix.java:3804) > > > > at > > > > org.apache.felix.framework.BundleContextImpl.registerService(Bund > > > > leContextImpl.java:328) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager.register > > > > Service(AbstractSlingRepositoryManager.java:218) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager.initiali > > > > zeAndRegisterRepositoryService(AbstractSlingRepositoryManager.jav > > > > a:541) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager.access$3 > > > > 00(AbstractSlingRepositoryManager.java:92) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager$4.run(Ab > > > > stractSlingRepositoryManager.java:496) > > > > > > > > Caused by: javax.jcr.LoginException: Can neither derive user name > > > > nor > > > > principal names for bundle org.apache.sling.jcr.resource [154] > > > > and sub > > > > service observation > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepository2.loginService(A > > > > bstractSlingRepository2.java:387) > > > > > > > > at > > > > org.apache.sling.jcr.resource.internal.JcrListenerBaseConfig.<ini > > > > t>(JcrListenerBaseConfig.java:62) > > > > > > > > at > > > > org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProv > > > > ider.registerListeners(JcrResourceProvider.java:218) > > > > > > > > ... 41 more > > > > > > > > > > > > The application deploys fine when not running against mongo, or > > > > when > > > > running against a clean mongo instance. > > > > > > > > The changes are located here for reference: > > > > > > > > > https://github.com/redhataccess/pantheon/pull/219/files#diff-e93a9e4b7b62ab20d546f78f9ac775c8L33 > > > > > > > > Any ideas on what could be going wrong? > > > > > > > > Regards, > > > > > > > > Carlos > > > > > > > > > > > > > > > > On Mon, Jan 27, 2020 at 4:57 AM Robert Munteanu < > > > > romb...@apache.org> > > > > wrote: > > > > > > > > > Happy to hear that you got it sorted out! Feel free to come > > > > > back with > > > > > more questions if you have any. > > > > > > > > > > Thanks, > > > > > Robert > > > > > > > > > > On Fri, 2020-01-24 at 10:58 -0500, Carlos Munoz wrote: > > > > > > Thanks Robert. I think we actually found out what was going > > > > > > on: it > > > > > > seems we > > > > > > have a poorly defined index which was being deployed as part > > > > > > of our > > > > > > bundle > > > > > > and which was interfering with some of the other indexes. As > > > > > > soon as > > > > > > we > > > > > > removed it everything started working once again. We are > > > > > > working on a > > > > > > better index for the query right now. > > > > > > > > > > > > Really appreciate your willingness to help here... ++ > > > > > > > > > > > > On Fri, Jan 24, 2020 at 5:03 AM Robert Munteanu < > > > > > > romb...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > I tried building the app from source code but did not > > > > > > > reproduce the > > > > > > > problem. I guess this matches your experience - this > > > > > > > happens only > > > > > > > during an 'upgrade'. > > > > > > > > > > > > > > Can you please give me a set of steps to reproduce? Ideally > > > > > > > without > > > > > > > MongoDB, but if that's required leave it in :-) > > > > > > > > > > > > > > Thanks, > > > > > > > Robert > > > > > > > > > > > > > > On Wed, 2020-01-22 at 22:08 -0500, Carlos Munoz wrote: > > > > > > > > I double checked and we do have the mapping. We copied > > > > > > > > all the > > > > > > > > provisioning > > > > > > > > files from the commit you recommended earlier [1] and > > > > > > > > deployed > > > > > > > > like > > > > > > > > that. > > > > > > > > > > > > > > > > In fact, you can see our provisioning files here: [2] We > > > > > > > > are only > > > > > > > > adding a > > > > > > > > single file with our own bundle and configurations. > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > https://github.com/apache/sling-org-apache-sling-starter/commit/c4f6e3b > > > > > > > > [2] > > > > > > > > > > > > > > https://github.com/redhataccess/pantheon/tree/upgrade-sling-bundles/pantheon-slingstart/src/main/provisioning > > > > > > > > On Wed, Jan 22, 2020 at 4:54 PM Robert Munteanu < > > > > > > > > romb...@apache.org> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > On Wed, 2020-01-22 at 16:16 -0500, Carlos Munoz wrote: > > > > > > > > > > Thanks for the tip Daniel! > > > > > > > > > > > > > > > > > > > > Robert - we were able to successfully package the > > > > > > > > > > sling > > > > > > > > > > starter > > > > > > > > > > with > > > > > > > > > > the > > > > > > > > > > latest definitions as you pointed, but when deploying > > > > > > > > > > on top > > > > > > > > > > of > > > > > > > > > > an > > > > > > > > > > existing > > > > > > > > > > database we started getting a JCR error: > > > > > > > > > > > > > > > > > > > > javax.jcr.LoginException: Can neither derive user > > > > > > > > > > name nor > > > > > > > > > > principal > > > > > > > > > > names > > > > > > > > > > for bundle org.apache.sling.jcr.resource [152] and > > > > > > > > > > sub > > > > > > > > > > service > > > > > > > > > > observation > > > > > > > > > > > > > > > > > > > > We don't get the same error when deploying on a fresh > > > > > > > > > > database. > > > > > > > > > > > > > > > > > > It seems that you have some missing service user > > > > > > > > > mappings. > > > > > > > > > Those > > > > > > > > > might > > > > > > > > > be required by newer versions of the bundles that you > > > > > > > > > just > > > > > > > > > consumed. In > > > > > > > > > the Sling Starter the current mapping is defined at > > > > > > > > > [1]. > > > > > > > > > > > > > > > > > > Does adding that as a configuration to your application > > > > > > > > > help? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Robert > > > > > > > > > > > > > > > > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > https://github.com/apache/sling-org-apache-sling-starter/blob/7eac121fc3f00c95ef5b8ac38133f6796a4a6c08/src/main/provisioning/sling.txt#L199-L202 > > > > > > > > > > -- > > > > > > Carlos A. Muñoz > > > > > > Manager, Software Engineering - Customer Platform > > > > > > Red Hat <https://www.redhat.com> > > > <https://red.ht/sig> > > > > >
