On 18/02/2022 11:05, Mikael Pesonen wrote:
Yes I think we ran out of disk space also last time and it might corrupt
something.
How does one dump the database without fuseki?
tdb2.tdbdump
Thanks!
On 17/02/2022 23.07, Andy Seaborne wrote:
Mikael,
You've had this problem before.
To revert your database:
1/ Stop the Fuseki server
2/ Delete Data-0002
3/ You can bnow restart
Now the undelying problem:
Do you have a backup?
Try to dump the database without Fuskei running.
If that works (I doubt it), run compact outside Fuseki.
The suggestion previously was :
- run on different hardware.
- run multiple times
- look at the data and see if anything unusual is in it.
Andy
On 17/02/2022 13:46, Mikael Pesonen wrote:
I get this exception with compact after couple of hours:
15:38:41 WARN Compact :: [10] **** Exception in compact
org.apache.jena.tdb2.TDBException: NodeTableTRDF/Read
at
org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:87)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:102)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:206)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:131)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:65)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:103)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:52)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.atlas.iterator.Iter$IterMap.next(Iter.java:417)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:41)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.dboe.transaction.txn.IteratorTxnTracker.next(IteratorTxnTracker.java:39)
~[fuseki-server.jar:4.3.2]
at java.util.Iterator.forEachRemaining(Iterator.java:133)
~[?:?]
at
org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$0(CopyDSG.java:38)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.exec(Txn.java:77)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.executeWrite(Txn.java:125)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$1(CopyDSG.java:36)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.exec(Txn.java:77)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.executeRead(Txn.java:115)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.tdb2.sys.CopyDSG.copy(CopyDSG.java:35)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:261)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:210)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.DatabaseMgr.compact(DatabaseMgr.java:80)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.fuseki.ctl.ActionCompact$CompactTask.run(ActionCompact.java:109)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.fuseki.async.AsyncPool.lambda$submit$0(AsyncPool.java:66)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.fuseki.async.AsyncTask.call(AsyncTask.java:100)
[fuseki-server.jar:4.3.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.thrift.protocol.TProtocolException:
Unrecognized type 0
at
org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:144)
~[fuseki-server.jar:4.3.2]
at
org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:60)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:433)
~[fuseki-server.jar:4.3.2]
at
org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
~[fuseki-server.jar:4.3.2]
at
org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
~[fuseki-server.jar:4.3.2]
at org.apache.thrift.TUnion.read(TUnion.java:138)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:82)
~[fuseki-server.jar:4.3.2]
... 30 more
15:38:41 ERROR Server :: Exception in task 1 execution
org.apache.jena.tdb2.TDBException: NodeTableTRDF/Read
at
org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:87)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:102)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:206)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:131)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:65)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:103)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:52)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.atlas.iterator.Iter$IterMap.next(Iter.java:417)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:41)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.dboe.transaction.txn.IteratorTxnTracker.next(IteratorTxnTracker.java:39)
~[fuseki-server.jar:4.3.2]
at java.util.Iterator.forEachRemaining(Iterator.java:133)
~[?:?]
at
org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$0(CopyDSG.java:38)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.exec(Txn.java:77)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.executeWrite(Txn.java:125)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$1(CopyDSG.java:36)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.exec(Txn.java:77)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.system.Txn.executeRead(Txn.java:115)
~[fuseki-server.jar:4.3.2]
at org.apache.jena.tdb2.sys.CopyDSG.copy(CopyDSG.java:35)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:261)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:210)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.DatabaseMgr.compact(DatabaseMgr.java:80)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.fuseki.ctl.ActionCompact$CompactTask.run(ActionCompact.java:109)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.fuseki.async.AsyncPool.lambda$submit$0(AsyncPool.java:66)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.fuseki.async.AsyncTask.call(AsyncTask.java:100)
[fuseki-server.jar:4.3.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.thrift.protocol.TProtocolException:
Unrecognized type 0
at
org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:144)
~[fuseki-server.jar:4.3.2]
at
org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.java:60)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:433)
~[fuseki-server.jar:4.3.2]
at
org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
~[fuseki-server.jar:4.3.2]
at
org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
~[fuseki-server.jar:4.3.2]
at org.apache.thrift.TUnion.read(TUnion.java:138)
~[fuseki-server.jar:4.3.2]
at
org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:82)
~[fuseki-server.jar:4.3.2]
... 30 more
15:38:41 INFO Server :: [Task 1] finishes : Compact
15:40:43 INFO Admin :: [21] Tasks
Data-0002 folder was created with 2,5G of data (Data-0001 has 88G).
On 14/02/2022 21.30, Andy Seaborne wrote:
Yes, it's a good idea.
TDB2 in Fuseki has the "compact" operation to do this without
stopping the server. It creates a new "Data-NNNN/" directory and you
can delete lower numbered databases.
TDB1 - needs the server stopping for a rebuild. If you can stop
updates, stop updates, 9server now read-only) backup, rebuild from
backup then stop the server and swap the databases.
Andy
On 14/02/2022 11:39, Mikael Pesonen wrote:
Hi,
we have now 13M triples and space usage of Jena data folder is 88G
which seems high. This is not including text index.
Should we cleanup/compress/rebuild etc the database regularly in
order to keep disk usage lower, or is this normal disk usage?
BR
