Good morning Carsten, I have the log files with debugging enabled on "org.apache.sling.installer".
I enabled logging on the primary instance just after launch and before installing the test package. After I copied the data to the standby, I wiped the logs of the standby before starting it up. They are quite big (> 300 MB), so I zipped them and placed them on my OneDrive: https://rto365-my.sharepoint.com/:f:/g/personal/wim_symons_vrt_be/EhVANoi3NgRIuBeuby9nbHYBWPY2RfYGwBSR5BhQs-cnDw?e=XhiaHc I really don't know what I should be looking for. Maybe you can make sense of it? But I saw something which caught my attention. In all the OSGi config files you have to create before starting up AEM: - install/install.primary/org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config - install/install.primary/org.apache.jackrabbit.oak.segment.standby.store.StandbyStoreService.config - install/install.standby/org.apache.jackrabbit.oak.segment.standby.store.StandbyStoreService.config - install/install.standby/org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config there is this line: org.apache.sling.installer.configuration.persist=B"false" If I understand this correctly by also looking at the source code of https://github.com/apache/sling-org-apache-sling-installer-factory-configuration/tree/org.apache.sling.installer.factory.configuration-1.2.2, this means OSGi configuration should NOT be persisted to disk. Is it possible this configuration is not picked up by the OSGi installer? In https://github.com/apache/sling-org-apache-sling-installer-factory-configuration/blob/6f14c8d17610d4d4c995f4927ebe01db053f0513/src/main/java/org/apache/sling/installer/factories/configuration/impl/ConfigTaskCreator.java#L142 this gets passed on to a changeListener via attrs: https://github.com/apache/sling-org-apache-sling-installer-factory-configuration/blob/6f14c8d17610d4d4c995f4927ebe01db053f0513/src/main/java/org/apache/sling/installer/factories/configuration/impl/ConfigTaskCreator.java#L154. This eventually lands into https://github.com/apache/sling-org-apache-sling-installer-factory-configuration/blob/org.apache.sling.installer.factory.configuration-1.2.2/src/main/java/org/apache/sling/installer/factories/configuration/impl/ConfigInstallTask.java. When I look at it's Git history, could this https://issues.apache.org/jira/browse/SLING-1971 be related? Kind regards Wim Op 30/07/2020 17:23 heeft Carsten Ziegeler <[email protected]> geschreven: Hi Wim that should be enough Thanks Carsten Am 30.07.2020 um 16:35 schrieb Wim Symons: > Hi Carsten, > > I'll enable DEBUG logging on org.apache.sling.installer.provider.jcr.impl and org.apache.sling.installer.core.impl in primary mode to see what it spits out when restarting it standby mode. > > Should that suffice, or should I also enable logging elsewhere? > > Kind regards > > Wim > > Op 30/07/2020 16:25 heeft Carsten Ziegeler <[email protected]> geschreven: > > Hi Wim, > > hmm, ok, now the JCR installer thinks that the "primary" run mode is > still active. So my first guess would be that this is the case. If thats > really not the case, then this needs more debugging of the JCR installer > and why it thinks that "primary" is still active. > > Regards > Carsten > > Am 30.07.2020 um 15:17 schrieb Wim Symons: > > Hi Carsten, > > > > Still haven't received your reply through the mail, I poked our helpdesk if it's stuck somewhere, but I could see it on the Internet at https://lists.apache.org/thread.html/ra950fefaf92aaede7ca2b61d5790d615c5dedf897920172fc372849d%40%3Cusers.sling.apache.org%3E. > > > > Yes, the configuration is in a config.author.primary folder. > > > > And yes, the configuration is listed in the OSGi installer tab of the console. > > > > See the screenshots at: > > - https://rto365-my.sharepoint.com/:i:/g/personal/wim_symons_vrt_be/ET_pBk0vtPRKpO04MubWSJgBhY1BBwmWuHkQfly5IvwrBw?e=PTEKKd > > - https://rto365-my.sharepoint.com/:i:/g/personal/wim_symons_vrt_be/EazREkn5gbRDq6gSBPUnWD4BlxYheRMd1KzTaa2NvXbaAg?e=mOg1Ls > > > > Kind regards > > > > Wim > > > > Op 30/07/2020 14:32 heeft Wim Symons <[email protected]> geschreven: > > > > Hi Carsten, > > > > I just tried your suggestion on a blank instance with a very simple test project. > > But, unfortunately, the result is the same. > > > > The component is active on the standby and has the config from the primary instance. > > > > What I did: > > > > - created an author-primary directory > > - created an instance with -r primary,crx3,crx3tar,nosamplecontent > > - verified the active runmodes: crx3, s7connect, nosamplecontent, author, crx3tar, primary > > - verified the Standby MBean: mode=primary, running=true > > - installed my test project > > - verified the component config: exists > > - verified the component is active and running > > - shutdown the instance > > > > - copied the entire author-primary to an author-standby directory > > - added the org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config and org.apache.jackrabbit.oak.segment.standby.store.StandbyStoreService.config to the crx-quickstart/install/install.standby folder as described at https://docs.adobe.com/content/help/en/experience-manager-64/deploying/deploying/tarmk-cold-standby.html > > - started the instance with -r standby,crx3,crx3tar,nosamplecontent -p 4602 > > - verified the active runmodes: standby,s7connect,crx3,nosamplecontent,author,crx3tar > > - verified the Standby MBean: mode=client, running=true, in sync > > - verified the component config: exists (!) > > - verified the component: active (!) and running > > - shut down the instance > > > > So, we're back to square one. > > > > Is this is an Apache Sling bug or an unsupported feature? > > > > Kind regards > > > > Wim > > > > Op 30/07/2020 09:59 heeft Wim Symons <[email protected]> geschreven: > > > > We will give it a try. > > > > Thanks!! > > > > Wim > > > > Op 30/07/2020 09:57 heeft Carsten Ziegeler <[email protected]> geschreven: > > > > Hi, > > > > The referenced documentation describes the procedure - which I believe > > works. It is not mentioning to delete configurations in the launchpad > > folder. > > > > So I guess what you want to achieve is to not activate components in > > standby mode - this can be done by using the run mode combination of > > author and primary for these configurations, so putting them into a > > config.author.primary folder. > > > > The primary is started with the run mode "primary" and therefore the > > configurations apply. The standby is not started with "primary" but > > "standby" - the OSGi/JCR installer will take care of it and handle the > > configurations properly. > > > > Regards > > Carsten > > > > > > Am 30.07.2020 um 09:44 schrieb Wim Symons: > > > Hi Carsten, > > > > > > Thanks for your swift reply. > > > > > > But that means, the procedure Adobe describes to correctly set up a standby instance on https://helpx.adobe.com/experience-manager/kb/how-to-setup-cold-standby-instance-in-AEM.html is not clear on this subject. > > > > > > Is there *any* way you know to make this work? > > > Can I reset the state of the JCR installer so it picks up all config again? > > > > > > Other thought is to change the components that really should not get active in standby mode to check the active runmode through the SlingSettings interface upon component start. > > > But that's quite a few, I'm thinking about components that start immediately, or components that are scheduled with a "scheduler.expression" (https://sling.apache.org/documentation/bundles/scheduler-service-commons-scheduler.html#scheduling-with-a-cron-expression) property. That way the component stays active for OSGi, but doesn't act because I programmed it to do so. > > > > > > Too bad it renders this statement (https://sling.apache.org/documentation/bundles/sling-settings-org-apache-sling-settings.html#getting-the-run-modes-of-the-sling-instance-1) unusable: > > > > > >> Getting run modes in this way is usually not needed, it's better to define bundles or configurations that are only valid in specific run modes, rather than making decisions in code based on run modes. > > > > > > Kind regards > > > > > > Wim > > > > > > Op 30/07/2020 09:13 heeft Carsten Ziegeler <[email protected]> geschreven: > > > > > > Hi, > > > > > > the run mode "author" is sticky - once you start a AEM instance with > > > that run mode it will continue to use it, regardless of what you specify > > > on the next startup. So in your example, the run mode author is active > > > for all three startups. > > > > > > Messing with the launchpad/config folder is not recommended as you can > > > run in exactly the problems you describe. The OSGi installer (the main > > > component used by the JCR installer) is not reverting changes done by > > > other tools or humans. So what you describe is actually expected: > > > > > > When you manually remove configurations from launchpad/config and > > > restart the instance (standby restart), the installer still has the > > > state that it has installed the configurations previously. So it is not > > > acting - this is the intended behaviour as the installer is not messing > > > with changes done through other means (like this manual remove). And as > > > "author" is a sticky run mode, the installer sees no change in the set > > > of known configurations read from JCR. > > > > > > Similar on the next restart (without standby), the installer does not do > > > anything. Again, author run mode still active, no change. > > > > > > Regards > > > Carsten > > > > > > Am 30.07.2020 um 08:38 schrieb Wim Symons: > > > > Hi all, > > > > > > > > After 2 weeks of waiting on Daycare to respond, I’m reaching out here, > > > > as I think this is more Apache Sling related than AEM specific. > > > > > > > > I’m trying to successfully make an AEM primary author from a cold > > > > standby author, but it fails. > > > > > > > > It fails because all our OSGi components with config bound to the > > > > ‘author’ runmode doesn’t get installed after switching runmode. > > > > > > > > I looked at the source code, but it seems there is no way to trigger the > > > > JcrInstaller to re-initialize all OSGi config stored in the repository. > > > > > > > > Is there any way to do that? > > > > > > > > A more detailed explanation of our process: > > > > > > > > You start by creating a standby author instance from a primary instance > > > > (runmode “author” is active). > > > > > > > > The OSGi configuration for our components is stored in sling:OsgiConfig > > > > nodes in the JCR below “config.author” folders. > > > > > > > > It is properly installed by the JCR Installer and the config files can > > > > be found in the launchpad/config folder on disk. > > > > > > > > We shut down the instance, *and remove all our OSGi config from the > > > > launchpad/config folder on disk*. > > > > > > > > We do that to avoid our components getting active in the standby runmode > > > > as this is not wanted. > > > > > > > > The OSGi config stored in the launchpad/config folder on disk is used, > > > > not the OSGi config from the JCR. > > > > > > > > I think this is **issue (1)**. > > > > > > > > We start the instance with runmode “standby”. > > > > > > > > All is fine, AEM is acting as a standby instance, and our components are > > > > not active. > > > > > > > > We restart the instance, but now give it the runmode “author” again. > > > > > > > > We would expect all OSGi configuration from the sling:OsgiConfig nodes > > > > below “config.author” folders would become installed by the OSGi > > > > framework, but in fact, it does not. > > > > > > > > This is **issue (2)**. > > > > > > > > AEM acts as an author instance again, but all our components are not > > > > active because they don’t have any OSGi config (launchpad/config is empty). > > > > > > > > How can I fix this? > > > > > > > > Or is this a situation Apache Sling can’t handle? > > > > > > > > Kind regards, > > > > > > > > *Wim Symons > > > > AEM Lead Architect > > > > *Digital Production Center > > > > > > > > > > > > [email protected] <mailto:[email protected]> > > > > Auguste Reyerslaan 52 > > > > B-1043 Brussel > > > > > > > > > > > > VRT logo <http://www.vrt.be/> > > > > > > > > > > > > -- Disclaimer -- > > > > Vlaamse Radio- en Televisieomroeporganisatie > > > > Auguste Reyerslaan 52 > > > > 1043 Brussel > > > > > > > > nv van publiek recht > > > > BTW BE 0244.142.664 > > > > RPR Brussel > > > > VRT Gebruikersvoorwaarden <http://www.vrt.be/gebruiksvoorwaarden > > > > > > > > > > > -- > > > -- > > > Carsten Ziegeler > > > Adobe Research Switzerland > > > [email protected] > > > > > > > > > -- Disclaimer -- > > > Vlaamse Radio- en Televisieomroeporganisatie > > > Auguste Reyerslaan 52 > > > 1043 Brussel > > > > > > nv van publiek recht > > > BTW BE 0244.142.664 > > > RPR Brussel > > > VRT Gebruikersvoorwaarden <http://www.vrt.be/gebruiksvoorwaarden> > > > > > > > -- > > -- > > Carsten Ziegeler > > Adobe Research Switzerland > > [email protected] > > > > > > -- Disclaimer -- > > Vlaamse Radio- en Televisieomroeporganisatie > > Auguste Reyerslaan 52 > > 1043 Brussel > > > > nv van publiek recht > > BTW BE 0244.142.664 > > RPR Brussel > > VRT Gebruikersvoorwaarden <http://www.vrt.be/gebruiksvoorwaarden> > > > > > > > > -- Disclaimer -- > > Vlaamse Radio- en Televisieomroeporganisatie > > Auguste Reyerslaan 52 > > 1043 Brussel > > > > nv van publiek recht > > BTW BE 0244.142.664 > > RPR Brussel > > VRT Gebruikersvoorwaarden <http://www.vrt.be/gebruiksvoorwaarden> > > > > > > > > > > -- Disclaimer -- > > Vlaamse Radio- en Televisieomroeporganisatie > > Auguste Reyerslaan 52 > > 1043 Brussel > > > > nv van publiek recht > > BTW BE 0244.142.664 > > RPR Brussel > > VRT Gebruikersvoorwaarden <http://www.vrt.be/gebruiksvoorwaarden> > > > > -- > -- > Carsten Ziegeler > Adobe Research Switzerland > [email protected] > > > -- Disclaimer -- > Vlaamse Radio- en Televisieomroeporganisatie > Auguste Reyerslaan 52 > 1043 Brussel > > nv van publiek recht > BTW BE 0244.142.664 > RPR Brussel > VRT Gebruikersvoorwaarden <http://www.vrt.be/gebruiksvoorwaarden> > -- -- Carsten Ziegeler Adobe Research Switzerland [email protected] -- Disclaimer -- Vlaamse Radio- en Televisieomroeporganisatie Auguste Reyerslaan 52 1043 Brussel nv van publiek recht BTW BE 0244.142.664 RPR Brussel VRT Gebruikersvoorwaarden <http://www.vrt.be/gebruiksvoorwaarden>
