Hi Bengt,
we had a similar problem using rebuild.sh. In our case increasing memory in
env-server.sh to
-Xmn128m -Xms256m -Xmx8192m
did solve the problem.
Greetings
Gudrun Siedersleben
Von: Bengt Neiss [mailto:[email protected]]
Gesendet: Mittwoch, 10. Juli 2013 12:52
An: Support and info exchange list for Fedora users.
Betreff: Re: [fcrepo-user] Problem using fedora-rebuild.sh
Hi all,
Some more info:
Adding object #660356: ptfe:941075
Adding object #660357: ptfe:1238122
Adding object #660358: ptfe:1286602
Adding object #660359: ptfe:1083889
Adding object #660360: ptfe:266983
Adding object #660361: ptfe:61167
Adding object #660362: ptfe:536362
Adding object #660363: ptfe:1039951
Adding object #660364: ptfe:189217
Exception in thread "pool-2-thread-1" java.lang.OutOfMemoryError: Java heap
space
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
at java.lang.Class.getDeclaredMethod(Class.java:1935)
at
java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1377)
at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:50)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:436)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:411)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:308)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1114)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
at java.util.HashSet.writeObject(HashSet.java:267)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:940)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
at sun.rmi.server.UnicastRef.marshalValue(UnicastRef.java:274)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:133)
at org.mulgara.server.rmi.RemoteJRDFSessionImpl_Stub.insert(Unknown
Source)
Adding object #660365: ptfe:549471
Adding object #660366: ptfe:98846
Adding object #660367: ptfe:892656
Adding object #660368: ptfe:291389
Adding object #660369: ptfe:969542
Adding object #660370: ptfe:1235248
Adding object #660371: ptfe:223915
Adding object #660372: ptfe:1264717
Adding object #660373: ptfe:69392
Adding object #660374: ptfe:520121
Adding object #660375: ptfe:1066324
Adding object #660376: ptfe:558063
Adding object #660377: ptfe:340028
Adding object #660378: ptfe:458161
Adding object #660379: ptfe:442074
Adding object #660380: ptfe:907369
Adding object #660381: ptfe:955114
Adding object #660382: ptfe:347046
Adding object #660383: ptfe:1169851
Adding object #660384: ptfe:669687
Adding object #660385: ptfe:282799
Adding object #660386: ptfe:77195
Adding object #660387: ptfe:866141
Adding object #660388: ptfe:433844
Adding object #660389: ptfe:902726
Adding object #660390: ptfe:767282
Adding object #660391: ptfe:538596
Exception in thread "RMI RenewClean-[10.50.170.35:53976]"
java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedInputStream.<init>(BufferedInputStream.java:178)
at java.io.BufferedInputStream.<init>(BufferedInputStream.java:158)
at
sun.rmi.transport.tcp.TCPConnection.getInputStream(TCPConnection.java:85)
at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:226)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
at sun.rmi.transport.DGCImpl_Stub.dirty(Unknown Source)
at
sun.rmi.transport.DGCClient$EndpointEntry.makeDirtyCall(DGCClient.java:342)
at
sun.rmi.transport.DGCClient$EndpointEntry.access$1600(DGCClient.java:153)
at
sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:555)
at java.lang.Thread.run(Thread.java:662)
Adding object #660392: ptfe:1091595
Exception in thread "pool-2-thread-10" java.lang.OutOfMemoryError: Java heap
space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.StringValue.from(StringValue.java:24)
at java.lang.String.<init>(String.java:178)
at org.apache.xerces.util.SymbolTable$Entry.<init>(Unknown Source)
at org.apache.xerces.util.SymbolTable.addSymbol(Unknown Source)
at org.apache.xerces.impl.XMLEntityScanner.scanQName(Unknown Source)
at
org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown
Source)
at org.openrdf.rio.rdfxml.RDFXMLParser.parse(RDFXMLParser.java:260)
at org.openrdf.rio.rdfxml.RDFXMLParser.parse(RDFXMLParser.java:209)
at org.trippi.io.RIOTripleIterator.run(RIOTripleIterator.java:163)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
All the best!
//Bengt
---------------------------------------------------------
Bengt Neiss
Kungl. Biblioteket / National Library of Sweden
Phone: +46 (0)10 709 35 41
Från: Bengt Neiss [mailto:[email protected]]
Skickat: den 9 juli 2013 14:29
Till:
[email protected]<mailto:[email protected]>
Ämne: [fcrepo-user] Problem using fedora-rebuild.sh
Hi all,
We are trying to rebuild a resource index on a remote Mulgara instance.
The Mulgara instance is on a server of its own and we try to rebuild
approximately 5 million triples.
What happens is that the rebuild script starts up as it should but after about
500 000 triples built it slows down and seems be in a flush-wait-flush state.
After another 100 000 triples the rebuild crashes with a java.lang.outofmemory
error (heap space in SAX parser if I remember correctly).
We have tried to increase the memory setting in env-server.sh but without any
luck.
Both servers runs on Redhat 6 and the following versions of software are used:
Fedora-Commons 3.6
Mulgara 2.1.12
Does anyoneknow if this a known issue?
Or have any suggestions on how to solve this?
//Bengt
---------------------------------------------------------
Bengt Neiss
Kungl. Biblioteket / National Library of Sweden
Phone: +46 (0)10 709 35 41
------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Fedora-commons-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
