Thanks for the details Jim. Does it happen in a docker container or directly on a machine ? How performant is the machine ? (in order for me to reproduce).
Thanks ! Regards JB On Thu, Mar 28, 2024 at 1:44 PM James Ziesig <james.zie...@broadcom.com> wrote: > > Hi JB, > > Thanks for looking into this. Yes, I know that some of the files are immune > to the problem. I could see that only some of the files in the etc directory > were modified between startup and the timestamp of the first log with ms > precision: > > "INFO | s4j.pax.logging) | 2024-03-27T20:25:52,051 | > EventAdminConfigurationNotifier | .EventAdminConfigurationNotifier 48 | > gging.pax-logging-log4j2 | | Sending Event Admin notification > (configuration successful) to org/ops4j/pax/logging/Configuration" > > I couldn't figure out how to enable fileinstall debug logging to see if it > was the one modifying the files, though. > > I've seen the problem happen with jmx, command, logging, feature service, and > our own configuration files. > > Hopefully you can identify the problem. > > Thanks again! > > Jim Ziesig > > On Thu, Mar 28, 2024 at 1:23 AM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: >> >> Hi Jim >> >> We did some improvements of potential race conditions between features >> and config admin services. Remember some files in etc are not managed >> by config admin (for instance startup.properties is used before the >> services are actually started and not managed by config admin). >> >> Let me try to reproduce your test case and I will let you know. >> >> Thanks, >> Regards >> JB >> >> On Thu, Mar 28, 2024 at 4:48 AM James Ziesig <james.zie...@broadcom.com> >> wrote: >> > >> > Hello, >> > >> > I have been tracking a problem in recent versions of Karaf where random >> > configuration files in the etc directory are corrupted (or re-initialized) >> > during karaf initialization. Most of the times that the problem occurs >> > all of the property settings are removed from the file, but sometimes the >> > file is re-initialized with properties in it (just not the values that >> > were present before startup). This only seems to happen when the >> > apache-karaf-4.x.x/data directory is cleared out prior to startup >> > (something we do on install/upgrade or after an ungraceful shutdown). The >> > issue seems to occur 3-5% of the time that Karaf is re-initialized. >> > >> > I have seen https://issues.apache.org/jira/browse/KARAF-6866 which covers >> > an example of the problem, but doesn't cover all cases. It led me to try >> > a "fix" where I alter startup.properties to start fileinstall at level 9 >> > so it starts before configadmin. I have had 100% success with well over >> > 300 restarts with this configuration, but I don't know if that >> > rearrangement may lead to some other issue. Is this a valid solution? >> > >> > This problem has occurred on various versions of CentOS/RHEL/Rocky Linux >> > (primarily 8.x). My testing was done on Rocky Linux release 8.9 (Green >> > Obsidian), using OpenJDK 64-Bit Server VM Temurin-17.0.10+7. >> > >> > I have attached a simple bash script for reproduction. >> > >> > Reproduction steps: >> > Extract apache-karaf-4.4.3.tar.gz. >> > cd apache-karaf-4.4.3 >> > cp -r etc etc_orig >> > rm -Rf data/* >> > bin/start >> > diff -r -w etc etc_orig/ >> > check if anything was already overwritten and reset etc with etc_orig if >> > it was and then repeat until etc looks good. >> > cp -r etc etc_backup (for use by script) >> > copy loop.sh to apache-karaf-4.4.3 >> > execute loop.sh >> > >> > Please let me know if you need any additional info. >> > >> > Thanks in advance, >> > >> > Jim Ziesig >> > >> > This electronic communication and the information and any files >> > transmitted with it, or attached to it, are confidential and are intended >> > solely for the use of the individual or entity to whom it is addressed and >> > may contain information that is confidential, legally privileged, >> > protected by privacy laws, or otherwise restricted from disclosure to >> > anyone else. If you are not the intended recipient or the person >> > responsible for delivering the e-mail to the intended recipient, you are >> > hereby notified that any use, copying, distributing, dissemination, >> > forwarding, printing, or copying of this e-mail is strictly prohibited. If >> > you received this e-mail in error, please return the e-mail to the sender, >> > delete it from your computer, and destroy any printed copy of it. > > > This electronic communication and the information and any files transmitted > with it, or attached to it, are confidential and are intended solely for the > use of the individual or entity to whom it is addressed and may contain > information that is confidential, legally privileged, protected by privacy > laws, or otherwise restricted from disclosure to anyone else. If you are not > the intended recipient or the person responsible for delivering the e-mail to > the intended recipient, you are hereby notified that any use, copying, > distributing, dissemination, forwarding, printing, or copying of this e-mail > is strictly prohibited. If you received this e-mail in error, please return > the e-mail to the sender, delete it from your computer, and destroy any > printed copy of it.
