https://defect.opensolaris.org/bz/show_bug.cgi?id=18602
Summary: Indexer (or just the jar analyzer?) can run out of
memory with no error message and stop indexing the
current project and move onto the next
Classification: Development
Product: opengrok
Version: unspecified
Platform: ANY/Generic
OS/Version: All
Status: NEW
Severity: normal
Priority: P3
Component: indexer
AssignedTo: [email protected]
ReportedBy: [email protected]
CC: [email protected]
So, my opengrok 0.10 installation wasn't indexing all my files -- it would
sometimes skip a few files, or perhaps thousands of files. It wasn't always
even consistent -- sometimes different files were skipped during subsequent
runs.
A few things did become clear --
-- if I indexed a lower number of collections, the number of files skipped
would go down (perhaps to zero once I discovered that I also had the issue
documented in Bug 18601.)
-- Using the -T 1 flag (to use only one thread) would reduce the number of
files skipped.
In any event, here's the relevant section of the logs. I've added some of my
own debug statements, as well as enabled verbose:gc in this --
--- cut here ---
Jun 22, 2011 12:29:53 AM org.opensolaris.opengrok.index.IndexDatabase indexDown
INFO: Progress: 15,196 (76.01%)
Jun 22, 2011 12:29:53 AM org.opensolaris.opengrok.index.IndexDatabase indexDown
INFO: dougmc: mode working, about to addFile:
/ContentSuite_7.6-cefix/qa-cs-api/thirdparty/bea/weblogic/lib/wlfullclient.jar
Jun 22, 2011 12:29:53 AM
org.opensolaris.opengrok.index.DefaultIndexChangedListener fileAdd
INFO: Add:
/ContentSuite_7.6-cefix/qa-cs-api/thirdparty/bea/weblogic/lib/wlfullclient.jar
(JarAnalyzer)
[GC 1457827K->1003588K(1965376K), 0.0217730 secs]
[GC 1470596K->1070420K(1953472K), 0.0564000 secs]
[GC 1531988K->1114452K(1969216K), 0.0666970 secs]
[GC 1451022K->1192250K(1945600K), 0.0940630 secs]
[GC 1615034K->1269844K(1959040K), 0.0804590 secs]
[GC 1692628K->1276636K(1937088K), 0.0616730 secs]
[GC 1706140K->1428644K(1951424K), 0.0996590 secs]
[Full GC 1428644K->909263K(1951424K), 0.2933280 secs]
[GC 1633679K->1211407K(1963584K), 0.0249790 secs]
[GC 1670735K->1218735K(1971712K), 0.0365590 secs]
[GC 1678063K->1226735K(1968576K), 0.0482960 secs]
[GC 1700975K->1233807K(1977600K), 0.0570360 secs]
[GC 1684708K->1240879K(1978496K), 0.0750460 secs]
[GC-- 1700726K->1700726K(1978496K), 0.1397120 secs]
[Full GC 1700726K->1243810K(1978496K), 0.5750480 secs]
[GC 1243810K->1243810K(1986560K), 0.0140220 secs]
[Full GC 1243810K->1235948K(1986560K), 0.6938190 secs]
Jun 22, 2011 12:30:02 AM org.opensolaris.opengrok.analysis.Ctags$1 run
WARNING: Error from ctags: ctags: Warning: ignoring null tag in
/u3/x/search/opengrok/stage1/source/ContentSuite_7.6-cefix/appsvcs-ui/src/configconsole/js/config/common.js
ctags: Warning: ignoring null tag in
/u3/x/search/opengrok/stage1/source/ContentSuite_7.6-cefix/appsvcs-ui/src/configconsole/js/dojo/src/widget/PopupContainer.js
ctags: Warning: ignoring null tag in
/u3/x/search/opengrok/stage1/source/ContentSuite_7.6-cefix/appsvcs-ui/src/configconsole/js/dojo/src/widget/SvgWidget.js
ctags: Warning: ignoring null tag in
/u3/x/search/opengrok/stage1/source/ContentSuite_7.6-cefix/appsvcs-ui/src/configconsole/js/dojo/src/widget/svg/Chart.js
ctags: Warning: ignoring null tag in
/u3/x/search/opengrok/stage1/source/ContentSuite_7.6-cefix/appsvcs-ui/src/jsp/common/scripts/common.js
ctags: Warning: ignoring null tag in
/u3/x/search/opengrok/stage1/source/ContentSuite_7.6-cefix/appsvcs-ui/src/jsp/common/scripts/xbDebug.js
Jun 22, 2011 12:30:03 AM org.opensolaris.opengrok.index.IndexDatabase update
INFO: Counting files in /ContentSuite_8.0-cefix ...
Jun 22, 2011 12:30:03 AM org.opensolaris.opengrok.index.IndexDatabase indexDown
--- cut here ---
Side note: this is on Linux, and the java version is
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
and the java command line options are: -Xms1024m -Xmx2048m -verbose:gc
Some things to notice here --
-- these ctag warnings seem completely normal. They seem to happen at the end
of indexing any collection. I originally thought they were part of the
problem, but now I don't think so.
-- it's doing an awful lot of GC right here, but it never seems to run out of
memory.
-- it's chugging along, indexing the ContentSuite_7.6-cefix project, 76% done
-- and then it just stops and goes onto the next product. It doesn't even
optimize the database (which using "luke" confirms) when it normally does.
-- there's no error or anything, it just stops and goes on to the next.
Looking at the code, the indexDown() method invokes addFile() for this file --
but the call never returns (some debug statements showed this.)
-- wlfullclient.jar is a massive jar -- 59 MB, 40,315 files.
-- if I remove all the other products and index only ContentSuite_7.6-cefix, it
indexes everything successfully, going all the way to 100%.
In any event, increasing the -Xmx value to 3000m (along with leaving the "-T 1"
flag enabled) fixed the problem -- but it wasn't obvious that this was needed
at first, as the large jar file seemed to be indexed (but possibly not
completely) -- instead, it was a bunch of other seemingly insignificant files
that weren't indexed -- and there was no error whatsoever.
SO ... at least to my untrained eye, the code seems to be silently swallowing
up an OutOfMemory exception somewhere in the jar processing code, and when it
does so, it ends the thread which stops processing of that collection, and it
just moves on to the next?
If there's a bug here, it wouldn't be that it's running out of memory -- we
probably can't fix that -- it would be that the error is not displayed
properly, and possibly that the processing of that one project stops there
(though probably the entire indexing operation should stop there with a big
error.)
--
Configure bugmail: https://defect.opensolaris.org/bz/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
_______________________________________________
opengrok-dev mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/opengrok-dev
