Hi all, I'm looking at clustering stability. One of the jobs I've been looking at is controller clustering. This is a good CSIT, in that it stops and starts ODL several times during the run.
In one of failed test runs (sandbox, logs wiped from last week, but I do have this particular karaf log archived locally), ODL is started, and rest calls fail during the test. Looking at the logs, I can see why. Karaf failed to start, or better yet, took a really long time to start. From the snipped below, you can see about 7 mins between when Karaf launched, and did something?, maybe restarted again. But the main thing is that karaf failed to start in a timely manner, taking over 7 minutes to begin to start up blueprints, etc. I ran a job that had karaf debug logging enabled with this setting: log4j.rootLogger=DEBUG This did not go very well. This generates way too much debug info, and was causing timeouts and other various errors in the CSIT run. So, my questions are: 1. Has anyone see this issue where karaf seems to hang on startup (after a kill -9 on karaf pid)? If so, is this a known issue? 2. What debug would be needed to figure out why karaf was hanging? Note the above generated a log file of ~768 MB in a very short timespan. Thanks, Vic Jun 29, 2018 3:43:47 PM org.apache.karaf.main.Main launch INFO: Installing and starting initial bundles Jun 29, 2018 3:43:47 PM org.apache.karaf.main.Main launch INFO: All initial bundles installed and set to start Jun 29, 2018 3:43:47 PM org.apache.karaf.main.lock.SimpleFileLock lock INFO: Trying to lock /tmp/karaf-0.8.3-SNAPSHOT/lock Jun 29, 2018 3:43:47 PM org.apache.karaf.main.lock.SimpleFileLock lock INFO: Lock acquiredJun 29, 2018 3:43:47 PM org.apache.karaf.main.Main$KarafLockCallback lockAquired INFO: Lock acquired. Setting startlevel to 100 Jun 29, 2018 3:50:48 PM org.apache.karaf.main.Main launch INFO: Installing and starting initial bundles Jun 29, 2018 3:50:49 PM org.apache.karaf.main.Main launch INFO: All initial bundles installed and set to start Jun 29, 2018 3:50:49 PM org.apache.karaf.main.lock.SimpleFileLock lock INFO: Trying to lock /tmp/karaf-0.8.3-SNAPSHOT/lock Jun 29, 2018 3:50:49 PM org.apache.karaf.main.lock.SimpleFileLock lock INFO: Lock acquired Jun 29, 2018 3:50:49 PM org.apache.karaf.main.Main$KarafLockCallback lockAquired INFO: Lock acquired. Setting startlevel to 100
_______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev
