This provisioning snippet triggered it for me, but in a more customized instance, so I'm not sure if it is enough by itself to cause issues:
[configurations] org.apache.sling.auth.form.FormAuthenticationHandler form.login.form="/login" On 2/10/20 4:48 AM, Robert Munteanu wrote: > I tried to reproduce this but I could not, unfortunately. What I did > was > > 1. Build the latest Sling Starter locally > 2. Started up MongoDB > > $ docker run --name mongo-sling -p 27017:27017 -d mongo:3.6 > > 3. Started up Sling > $ java -jar target/org.apache.sling.starter-12-SNAPSHOT.jar > -Dsling.run.modes=oak_mongo > > 4. Waited for Sling to start, clicked around a bit and then shut it > down with Ctrl-C. > > 5. I started up Sling again, with the same command as the one from Step > 3. > > Sling started up successfully, no errors in the log. > > Do you have some simplified steps to reproduce I can try? Sharing a k8s > resource file is fine as well, as long as it is self-contained and I > can `kubectl apply -f` it and then follow some directions to get Sling > broken. > > Thanks, > Robert > > > On Fri, 2020-02-07 at 16:58 -0500, Carlos Munoz wrote: >> Thanks Robert. We tried ensuring only a single Sling pod was hitting >> the >> database at one time with some strange results: >> >> The first time it runs (against an empty database) everything goes >> well: >> the database is populated and the pod comes up with no issues. >> >> We then bring this pod down, and then try to bring the same exact one >> up >> again with the original exception popping up again: >> >> 29.01.2020 02:58:59.571 *ERROR* [Apache Sling Repository Startup >> Thread #4] >> ERROR: Bundle '160' EventDispatcher: Error during dispatch. >> (org.apache.sling.api.SlingException: Can't create the JCR event >> listener.) >> org.apache.sling.api.SlingException: Can't create the JCR event >> listener. >> ... >> ... >> Caused by: javax.jcr.LoginException: Can neither derive user name nor >> principal names for bundle org.apache.sling.jcr.resource [154] and >> sub >> service observation >> at >> org.apache.sling.jcr.base.AbstractSlingRepository2.loginService(Abstr >> actSlingRepository2.java:387) >> >> I wonder if the sling pod is leaving the database in an unusable >> state when >> being brought down. >> >> Regards, >> >> Carlos >> >> On Thu, Feb 6, 2020 at 4:11 AM Robert Munteanu <romb...@apache.org> >> wrote: >> >>> On Wed, 2020-02-05 at 21:17 -0500, Carlos Munoz wrote: >>>> Hi all, >>>> >>>> I think I have a theory for our issues here, and it may have to >>>> do >>>> with the >>>> fact that we are running on a heavily containerized environment >>>> (kubernetes). I wanted to consult with the community experts to >>>> see >>>> what >>>> you thought. >>>> >>>> The way our container platform works on an update is that it will >>>> try >>>> to >>>> bring up a new container with sling (and our application) against >>>> the >>>> same >>>> mongo database that an original (and still running) container is >>>> running >>>> against. Now this works fine when the only thing being updated is >>>> our >>>> application bundle, but it starts encountering problems when >>>> several >>>> other >>>> bundles and configurations are being updated (some removed, some >>>> added, >>>> some updated). I *think* the core of the problem here is that the >>>> bundles >>>> and configurations are all stored in the database itself, and two >>>> containers with potentially different bundle versions and >>>> configurations >>>> are attempting to use it simultaneously. >>> >>> That is a pretty good guess I'd say :-) >>> >>> I did see some similar problems when using Sling for development >>> purposes on k8s. I never went to production with it, but for my own >>> purposes it was enough to ensure that only one Sling pod was >>> starting >>> up at a time. Maybe you can try that as well? >>> >>> A more involved solution would be to use the CompositeNodeStore >>> [1], >>> which is designed to separate the storage of /libs and /apps from >>> the >>> rest of the repository. So for instance you'd have /libs and /apps >>> stored on a local segment store for each pod, and the rest of the >>> content in Mongo. >>> >>> Unfortunately there is very little documentation and no tooling >>> around >>> it available, so that makes it a difficult proposition. >>> >>> Thanks, >>> Robert >>> >>> >>> [1]: >>> https://jackrabbit.apache.org/oak/docs/nodestore/compositens.html >>> >>>> If I am right, then our core problem to figure out is how to >>>> upgrade >>>> a >>>> database from one sling version to the next. >>>> >>>> Let me know what you all think. >>>> >>>> Regards, >>>> >>>> Carlos >>>> >>>> On Tue, Feb 4, 2020 at 7:06 AM Carlos Munoz <camu...@redhat.com> >>>> wrote: >>>> >>>>> Thanks Bertrand! I will continue my fact finding mission here >>>>> :) >>>>> >>>>> Regards, >>>>> >>>>> Carlos >>>>> >>>>> On Tue, Feb 4, 2020 at 4:31 AM Bertrand Delacretaz < >>>>> bdelacre...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> On Sun, Feb 2, 2020 at 4:50 AM Carlos Munoz < >>>>>> camu...@redhat.com> >>>>>> wrote: >>>>>>> ...do configurations from the >>>>>>> repoinit files get installed in a specific order with >>>>>>> relation >>>>>>> to the >>>>>>> artifacts?... >>>>>> >>>>>> The repoinit configs are applied by a single >>>>>> SlingRepositoryInitializer [1] service which is implemented >>>>>> by >>>>>> org.apache.sling.jcr.repoinit.impl.RepositoryInitializer [2]. >>>>>> >>>>>> The execution order of the SlingRepositoryInitializer >>>>>> services is >>>>>> based on their service rankings [4] and the >>>>>> RepositoryInitializer >>>>>> processes its configurations in the order in which they are >>>>>> provided >>>>>> by the OSGi framework, sequentially. >>>>>> >>>>>> All this happens before the SlingRepository service is made >>>>>> available [3] >>>>>> >>>>>> The logs should help understand what's going on but IIRC it >>>>>> all >>>>>> happens in a single thread. >>>>>> >>>>>> -Bertrand >>>>>> >>>>>> [1] >>>>>> >>> https://sling.apache.org/documentation/bundles/repository-initialization.html >>>>>> [2] >>>>>> >>> https://github.com/apache/sling-org-apache-sling-jcr-repoinit/blob/master/src/main/java/org/apache/sling/jcr/repoinit/impl/RepositoryInitializer.java >>>>>> [3] >>>>>> >>> https://github.com/apache/sling-org-apache-sling-jcr-base/blob/e8fe5e004b5af1802bb2a76dbbb583a437f848ee/src/main/java/org/apache/sling/jcr/base/AbstractSlingRepositoryManager.java#L511 >>>>>> [4] >>>>>> >>> https://github.com/apache/sling-org-apache-sling-jcr-base/blob/e8fe5e004b5af1802bb2a76dbbb583a437f848ee/src/main/java/org/apache/sling/jcr/base/AbstractSlingRepositoryManager.java#L581 >>>>>> > -- Sergiu Dumitriu http://purl.org/net/sergiu