Sure thing. Steps to reproduce are fairly simple, we actually saw the same result when building a vanilla sling starter from master [1]. But if you want to be closer to our specific problem you can build from our application's starter module [2].
We containerize the application using the Dockerfile in the same repo [3], and we run in a Kubernetes orchestrated server against a Mongo DB cluster with 3 nodes (note that there is some latency due to the database). We run the application from the resulting container image and we see the error as sling is coming up. Let me know if this is enough to go by. As a side question, I was wondering if the order of the provisioning files makes any difference to the sequence in which bundles and configurations are installed. I was thinking if this is the case we might be able to reorganize those files to ensure all configurations are loaded before the bundles using them. Regards, Carlos [1] https://github.com/apache/sling-org-apache-sling-starter [2] https://github.com/redhataccess/pantheon/pull/221 [3] https://github.com/redhataccess/pantheon/blob/master/container/Dockerfile On Thu, Jan 30, 2020 at 4:47 AM Robert Munteanu <romb...@apache.org> wrote: > Hi Carlos, > > Yes, this may be a timing issue. > > I could not follow the link you sent me for some reason. I think the > build log is the one from [1]. If that is the case, the error is > visible at shutdown, and probably does not have the same root cause. > > I'd still like to get some steps to reproduce - even if it's a change > of 1 in 3, even if it depends on containers. > > Thanks, > Robert > > > [1]: > https://builds.apache.org/job/Sling/job/sling-org-apache-sling-starter/job/master/104/console > > On Wed, 2020-01-29 at 21:27 -0500, Carlos Munoz wrote: > > Robert, I checked the latest (master) pipeline build logs for the > > starter > > project: > > > > > https://builds.apache.org/blue/organizations/jenkins/Sling%2Fsling-org-apache-sling-starter/detail/master/104/pipeline/24 > > > > and found that there is a very similar error being reported > > (different > > principal and bundle), but same type of exception nonetheless. > > > > Carlos > > > > > > On Wed, Jan 29, 2020 at 7:47 PM Carlos Munoz <camu...@redhat.com> > > wrote: > > > > > Robert, I wonder if this is a timing issue. I’m not sure I > > > understand how > > > Sling is loading bundles and configurations, but is it possible > > > that it > > > could load up a bundle which needs a specific configuration before > > > said > > > configuration has finished loading? > > > > > > I mention this because we are seeing the error now on a > > > containerized > > > environment where resources may be more virtualized than in a local > > > environment, where the application seems to run without any issues. > > > > > > Regards, > > > > > > Carlos > > > > > > On Tue, Jan 28, 2020 at 10:11 PM Carlos Munoz <camu...@redhat.com> > > > wrote: > > > > > > > Hi Robert, I'm picking up this thread again since we briefly > > > > talked about > > > > this problem; allow me to recap: > > > > We are attempting to migrate bundle versions for a Sling > > > > application from > > > > their Sling 11 versions to the latest stable versions. The > > > > application is > > > > running against an already populated mongo database and we are > > > > seeing the > > > > following exception when deploying. > > > > > > > > 29.01.2020 02:58:59.571 *ERROR* [Apache Sling Repository Startup > > > > Thread > > > > #4] ERROR: Bundle '160' EventDispatcher: Error during dispatch. > > > > (org.apache.sling.api.SlingException: Can't create the JCR event > > > > listener.) > > > > org.apache.sling.api.SlingException: Can't create the JCR event > > > > listener. > > > > at > > > > org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProv > > > > ider.registerListeners(JcrResourceProvider.java:227) > > > > > > > > at > > > > org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProv > > > > ider.start(JcrResourceProvider.java:182) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Handler.activate(ResourceProviderHandler.java:74) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker.activate(ResourceProviderTracker.java:360) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker.register(ResourceProviderTracker.java:192) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker.access$200(ResourceProviderTracker.java:59) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker$1.addingService(ResourceProviderTracker.java:130) > > > > > > > > at > > > > org.apache.sling.resourceresolver.impl.providers.ResourceProvider > > > > Tracker$1.addingService(ResourceProviderTracker.java:106) > > > > > > > > at > > > > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(Ser > > > > viceTracker.java:943) > > > > > > > > at > > > > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(Ser > > > > viceTracker.java:871) > > > > > > > > at > > > > org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked > > > > .java:256) > > > > at > > > > org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java: > > > > 229) > > > > at > > > > org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(Servi > > > > ceTracker.java:903) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.invokeServiceListenerC > > > > allback(EventDispatcher.java:990) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireEventImmediately(E > > > > ventDispatcher.java:838) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireServiceEvent(Event > > > > Dispatcher.java:545) > > > > > > > > at > > > > org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4833 > > > > ) > > > > at > > > > org.apache.felix.framework.Felix.registerService(Felix.java:3804) > > > > at > > > > org.apache.felix.framework.BundleContextImpl.registerService(Bund > > > > leContextImpl.java:328) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager$3.regi > > > > ster(AbstractComponentManager.java:906) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager$3.regi > > > > ster(AbstractComponentManager.java:892) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.RegistrationManager.changeRegis > > > > tration(RegistrationManager.java:128) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager.regist > > > > erService(AbstractComponentManager.java:959) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.AbstractComponentManager.activa > > > > teInternal(AbstractComponentManager.java:732) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticC > > > > ustomizer.addedService(DependencyManager.java:1045) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticC > > > > ustomizer.addedService(DependencyManager.java:999) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customiz > > > > erAdded(ServiceTracker.java:1216) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customiz > > > > erAdded(ServiceTracker.java:1137) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked. > > > > trackAdding(ServiceTracker.java:944) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked. > > > > track(ServiceTracker.java:880) > > > > > > > > at > > > > org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceC > > > > hanged(ServiceTracker.java:1168) > > > > > > > > at > > > > org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.s > > > > erviceChanged(BundleComponentActivator.java:125) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.invokeServiceListenerC > > > > allback(EventDispatcher.java:990) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireEventImmediately(E > > > > ventDispatcher.java:838) > > > > > > > > at > > > > org.apache.felix.framework.EventDispatcher.fireServiceEvent(Event > > > > Dispatcher.java:545) > > > > > > > > at > > > > org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4833 > > > > ) > > > > at > > > > org.apache.felix.framework.Felix.registerService(Felix.java:3804) > > > > at > > > > org.apache.felix.framework.BundleContextImpl.registerService(Bund > > > > leContextImpl.java:328) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager.register > > > > Service(AbstractSlingRepositoryManager.java:218) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager.initiali > > > > zeAndRegisterRepositoryService(AbstractSlingRepositoryManager.jav > > > > a:541) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager.access$3 > > > > 00(AbstractSlingRepositoryManager.java:92) > > > > > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepositoryManager$4.run(Ab > > > > stractSlingRepositoryManager.java:496) > > > > > > > > Caused by: javax.jcr.LoginException: Can neither derive user name > > > > nor > > > > principal names for bundle org.apache.sling.jcr.resource [154] > > > > and sub > > > > service observation > > > > at > > > > org.apache.sling.jcr.base.AbstractSlingRepository2.loginService(A > > > > bstractSlingRepository2.java:387) > > > > > > > > at > > > > org.apache.sling.jcr.resource.internal.JcrListenerBaseConfig.<ini > > > > t>(JcrListenerBaseConfig.java:62) > > > > > > > > at > > > > org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProv > > > > ider.registerListeners(JcrResourceProvider.java:218) > > > > > > > > ... 41 more > > > > > > > > > > > > The application deploys fine when not running against mongo, or > > > > when > > > > running against a clean mongo instance. > > > > > > > > The changes are located here for reference: > > > > > > > > > https://github.com/redhataccess/pantheon/pull/219/files#diff-e93a9e4b7b62ab20d546f78f9ac775c8L33 > > > > > > > > Any ideas on what could be going wrong? > > > > > > > > Regards, > > > > > > > > Carlos > > > > > > > > > > > > > > > > On Mon, Jan 27, 2020 at 4:57 AM Robert Munteanu < > > > > romb...@apache.org> > > > > wrote: > > > > > > > > > Happy to hear that you got it sorted out! Feel free to come > > > > > back with > > > > > more questions if you have any. > > > > > > > > > > Thanks, > > > > > Robert > > > > > > > > > > On Fri, 2020-01-24 at 10:58 -0500, Carlos Munoz wrote: > > > > > > Thanks Robert. I think we actually found out what was going > > > > > > on: it > > > > > > seems we > > > > > > have a poorly defined index which was being deployed as part > > > > > > of our > > > > > > bundle > > > > > > and which was interfering with some of the other indexes. As > > > > > > soon as > > > > > > we > > > > > > removed it everything started working once again. We are > > > > > > working on a > > > > > > better index for the query right now. > > > > > > > > > > > > Really appreciate your willingness to help here... ++ > > > > > > > > > > > > On Fri, Jan 24, 2020 at 5:03 AM Robert Munteanu < > > > > > > romb...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > I tried building the app from source code but did not > > > > > > > reproduce the > > > > > > > problem. I guess this matches your experience - this > > > > > > > happens only > > > > > > > during an 'upgrade'. > > > > > > > > > > > > > > Can you please give me a set of steps to reproduce? Ideally > > > > > > > without > > > > > > > MongoDB, but if that's required leave it in :-) > > > > > > > > > > > > > > Thanks, > > > > > > > Robert > > > > > > > > > > > > > > On Wed, 2020-01-22 at 22:08 -0500, Carlos Munoz wrote: > > > > > > > > I double checked and we do have the mapping. We copied > > > > > > > > all the > > > > > > > > provisioning > > > > > > > > files from the commit you recommended earlier [1] and > > > > > > > > deployed > > > > > > > > like > > > > > > > > that. > > > > > > > > > > > > > > > > In fact, you can see our provisioning files here: [2] We > > > > > > > > are only > > > > > > > > adding a > > > > > > > > single file with our own bundle and configurations. > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > https://github.com/apache/sling-org-apache-sling-starter/commit/c4f6e3b > > > > > > > > [2] > > > > > > > > > > > > > > https://github.com/redhataccess/pantheon/tree/upgrade-sling-bundles/pantheon-slingstart/src/main/provisioning > > > > > > > > On Wed, Jan 22, 2020 at 4:54 PM Robert Munteanu < > > > > > > > > romb...@apache.org> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > On Wed, 2020-01-22 at 16:16 -0500, Carlos Munoz wrote: > > > > > > > > > > Thanks for the tip Daniel! > > > > > > > > > > > > > > > > > > > > Robert - we were able to successfully package the > > > > > > > > > > sling > > > > > > > > > > starter > > > > > > > > > > with > > > > > > > > > > the > > > > > > > > > > latest definitions as you pointed, but when deploying > > > > > > > > > > on top > > > > > > > > > > of > > > > > > > > > > an > > > > > > > > > > existing > > > > > > > > > > database we started getting a JCR error: > > > > > > > > > > > > > > > > > > > > javax.jcr.LoginException: Can neither derive user > > > > > > > > > > name nor > > > > > > > > > > principal > > > > > > > > > > names > > > > > > > > > > for bundle org.apache.sling.jcr.resource [152] and > > > > > > > > > > sub > > > > > > > > > > service > > > > > > > > > > observation > > > > > > > > > > > > > > > > > > > > We don't get the same error when deploying on a fresh > > > > > > > > > > database. > > > > > > > > > > > > > > > > > > It seems that you have some missing service user > > > > > > > > > mappings. > > > > > > > > > Those > > > > > > > > > might > > > > > > > > > be required by newer versions of the bundles that you > > > > > > > > > just > > > > > > > > > consumed. In > > > > > > > > > the Sling Starter the current mapping is defined at > > > > > > > > > [1]. > > > > > > > > > > > > > > > > > > Does adding that as a configuration to your application > > > > > > > > > help? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Robert > > > > > > > > > > > > > > > > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > https://github.com/apache/sling-org-apache-sling-starter/blob/7eac121fc3f00c95ef5b8ac38133f6796a4a6c08/src/main/provisioning/sling.txt#L199-L202 > > > > > > > > > > -- > > > > > > Carlos A. Muñoz > > > > > > Manager, Software Engineering - Customer Platform > > > > > > Red Hat <https://www.redhat.com> > > > <https://red.ht/sig> > > > > >
