Re: Does Jena need maintainance regarding to disk space?

Thu, 17 Feb 2022 13:08:16 -0800 

Mikael,

You've had this problem before.

To revert your database:
1/ Stop the Fuseki server
2/ Delete Data-0002
3/ You can bnow restart

Now the undelying problem:

Do you have a backup?

Try to dump the database without Fuskei running.
If that works (I doubt it), run compact outside Fuseki.


The suggestion previously was :
  - run on different hardware.
  - run multiple times
  - look at the data and see if anything unusual is in it.

    Andy

On 17/02/2022 13:46, Mikael Pesonen wrote:


I get this exception with compact after couple of hours:

15:38:41 WARN  Compact         :: [10] **** Exception in compact
org.apache.jena.tdb2.TDBException: NodeTableTRDF/Read
        at org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:87) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:102) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:206) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:131) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:65) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:103) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:52) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.atlas.iterator.Iter$IterMap.next(Iter.java:417) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:41) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.dboe.transaction.txn.IteratorTxnTracker.next(IteratorTxnTracker.java:39) ~[fuseki-server.jar:4.3.2] 
         at java.util.Iterator.forEachRemaining(Iterator.java:133) ~[?:?]
        at org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$0(CopyDSG.java:38) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.exec(Txn.java:77) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.executeWrite(Txn.java:125) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$1(CopyDSG.java:36) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.exec(Txn.java:77) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.executeRead(Txn.java:115) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.CopyDSG.copy(CopyDSG.java:35) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:261) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:210) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.DatabaseMgr.compact(DatabaseMgr.java:80) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.fuseki.ctl.ActionCompact$CompactTask.run(ActionCompact.java:109) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.fuseki.async.AsyncPool.lambda$submit$0(AsyncPool.java:66) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.fuseki.async.AsyncTask.call(AsyncTask.java:100) [fuseki-server.jar:4.3.2] 
         at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 
         at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.thrift.protocol.TProtocolException: Unrecognized type 0         at org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:144) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:60) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:433) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.TUnion.read(TUnion.java:138) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:82) ~[fuseki-server.jar:4.3.2] 
         ... 30 more
15:38:41 ERROR Server          :: Exception in task 1 execution
org.apache.jena.tdb2.TDBException: NodeTableTRDF/Read
        at org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:87) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:102) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:206) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:131) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:65) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:103) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:52) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.atlas.iterator.Iter$IterMap.next(Iter.java:417) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:41) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.dboe.transaction.txn.IteratorTxnTracker.next(IteratorTxnTracker.java:39) ~[fuseki-server.jar:4.3.2] 
         at java.util.Iterator.forEachRemaining(Iterator.java:133) ~[?:?]
        at org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$0(CopyDSG.java:38) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.exec(Txn.java:77) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.executeWrite(Txn.java:125) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$1(CopyDSG.java:36) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.exec(Txn.java:77) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.system.Txn.executeRead(Txn.java:115) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.CopyDSG.copy(CopyDSG.java:35) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:261) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:210) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.DatabaseMgr.compact(DatabaseMgr.java:80) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.fuseki.ctl.ActionCompact$CompactTask.run(ActionCompact.java:109) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.fuseki.async.AsyncPool.lambda$submit$0(AsyncPool.java:66) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.fuseki.async.AsyncTask.call(AsyncTask.java:100) [fuseki-server.jar:4.3.2] 
         at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 
         at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.thrift.protocol.TProtocolException: Unrecognized type 0         at org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:144) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:60) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:433) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213) ~[fuseki-server.jar:4.3.2]         at org.apache.thrift.TUnion.read(TUnion.java:138) ~[fuseki-server.jar:4.3.2]         at org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:82) ~[fuseki-server.jar:4.3.2] 
         ... 30 more
15:38:41 INFO  Server          :: [Task 1] finishes : Compact
15:40:43 INFO  Admin           :: [21] Tasks


Data-0002 folder was created with 2,5G of data (Data-0001 has 88G).


On 14/02/2022 21.30, Andy Seaborne wrote:

Yes, it's a good idea.
TDB2 in Fuseki has the "compact" operation to do this without stopping the server. It creates a new "Data-NNNN/" directory and you can delete lower numbered databases.
TDB1 - needs the server stopping for a rebuild. If you can stop updates, stop updates, 9server now read-only) backup, rebuild from backup then stop the server and swap the databases. 

    Andy

On 14/02/2022 11:39, Mikael Pesonen wrote:


Hi,
we have now 13M triples and space usage of Jena data folder is 88G which seems high. This is not including text index. Should we cleanup/compress/rebuild etc the database regularly in order to keep disk usage lower, or is this normal disk usage? 

BR

Reply via email to