Hello, everyone, We've been doing trial ingests of sets of objects, and we're running into a problem with Mulgara using extreme amounts of memory. Our total working set was approximately 2.2 million FOXML files, but I segmented them into 20 sets. Shortly after beginning an ingest of one of these sets, the Fedora java process would allocate over 10 GB of virtual memory, and as the hours progressed, RSIZE would exceed the Xmx setting for the JVM by 4x or so, and the machine would fall over into swap. This slowed down ingest considerably. Interestingly, the shared RAM value for the Fedora process would be some large percentage of the RSIZE, for example, like 80%, and I attached Netbeans' heap profiler to Tomcat and didn't notice anything taking up an inordinate amount of space.
I turned off the Resource Index for the remainder of the batches and then attempted a reindex afterward, and even the fedora-rebuild.sh process consumed far more memory than the heap allocation: VSIZE: 9.9g RSIZE: 2.9g SHARED: 2.4g Xmx for fedora-rebuild.sh: 512m (No other JVM args for that script were altered) The rebuild was still progressing, but the machine had fallen into swap again, so I halted it and brought Fedora back up without the Resource Index. Some system info: uname -a: Linux fedora01.nypl.net 2.6.18-92.el5 #1 SMP Tue Apr 29 13:16:15 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux java -version: java version "1.6.0_12" Java(TM) SE Runtime Environment (build 1.6.0_12-b04) Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode) Fedora version: 3.1 Anybody know of any magic buttons I should press? Should I attempt to update to 3.2 to see if my issues go away? Thanks for any advice anyone may be able to provide, Michael Della Bitta NYPL ------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects _______________________________________________ Fedora-commons-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
