Hi all, last week we upgraded our productive instance of Archiva from 1.3 to 2.0.1 and I'm struggling a bit with trying to tweak the performance. My findings / issues so far:
1. Our main pain point is degrading performance for uploads (we have quite a lot Jenkins jobs which upload to Archiva). I'm still not 100% sure what's the matter here, but it looks like it is closely related to the repository-scanning queue. As soon as that one gets above a few hundred Jenkins deployment times rise from 2 minutes to sometimes over 30 minutes. I ran the VisualVM CPU Sampler and a lot of time was spent in "DefaultISMLocking.acquireReadLock()" and "aquireReadWriteLock()". During that time io-waits were at 1-2% and only one core was at 100% - I saw that the scanning and indexing are processed single threaded so this is expected. 2. Sometimes the system comes to a grinding halt (today we had a period of at least 5 minutes are nothing was responding. I couldn't connect via JMX / VisualVM, via remote debugging, Web UI was showing the spinner, Up / Downloads via Maven were blocked. I have no idea what caused it, maybe a search? Or the calculation of dependencies / used by? CPU was at 100% on all cores. Maybe it was GC constantly running as I wanted to check that, the system started acting "normal" again. Regarding memory, is it possible that the wrapper.conf only alows up to 4 GB of heap? I tried setting 6 GB, but VisualVM shows only 4 (yes, 64-bit OS, Java and VisualVM ;) I double checked that). So Archiva runs with 512MB PermGen and 4GB of heap (initial heap is set to 4GB as well). Hardware / Installation: * 4 cores * 8 GB RAM * RHEL 6.5 * somewhat slow FS (VMWare with para-virtual driver and not the fastest SAN in the world) * Oracle JDK 1.7.0_51 * Running Archiva standalone with it's Jetty Are there any recommendations what to do to improve the situation? Should I try patching the repo-scanning to use multiple cores? Should we get more cores? Any VM recommendations? Should I run it inside a Tomcat or another container recommended? Btw. in order to at least have it somewhat stable, I disabled scanning of repositories at all and we only index pom.xml nothing more. Obviously I would like to enable that again. I also tried playing with the Lock Timeout / Skip Locking setting in the Runtime Configuration, imho that didn't change the situation much. And as a last side note: Is it possible that remember me isnt working? I get logged out quite frequently. Thanks for reading that long :) Any pointers are highly welcome! I would hate to downgrade to 1.3 again! Greetings -Sascha-
