Hi Andy, Lorenz.. I tried the TDBLoader and it generated the indexes, etc.
[image: image.png] boot up geosparql with the tdb file and I run out of memory too. java -jar jena-fuseki-geosparql-3.17.0.jar --convert_geo --tdb spatialdb2 -i *Exception in thread "main" java.lang.OutOfMemoryError: Java heap space* Now, I'm sure the solution is to add a param akin to *-Xss8m* to book the JVM with more memory. java -jar *-Xss8m* jena-fuseki-geosparql-3.17.0.jar --convert_geo --tdb spatialdb2 Does this sound right to you? On Tue, 13 Jul 2021 at 22:09, Andy Seaborne <[email protected]> wrote: > > > On 13/07/2021 11:31, Matt Whitby wrote: > > Morning all. > > > > > > It's just on my laptop (though with 64gb of memory so more than enough I > > would assume). > > Unless you have changed something, it is using 16G. > > However, it is not clear that it is a memory space issue. > > > > > The file is about 850mb, so not that big in the scheme of things. > > > > I don't see any log files. > > They have been printed to stdout as shown. They can go elsewhere (it's > log4j2). > > > > > The full stack trace is... > > There was an Java Error (the code doesn't print it - an oversight). Out > of memory is an error. > > How long after the "DatasetOperations :: Reading RDF - Started - File:" > output does it fail? > > It is worth checking the file parses correctly. e.g. some encoding > errors become Java "errors" in 3.17.0. > > > 11:26:01 INFO DatasetOperations :: In-Memory Dataset > > That means there are multiple copies in-memory during loading. > This does not explain using 16G. > > But the database can be loaded using the TDB bulkloader separately from > the server starting and then pass in the persistent database so the file > does not have to be read each start-up. > > Andy > > > > C:\Data\apache-jena-fuseki-3.17.0>java -jar > > jena-fuseki-geosparql-3.17.0.jar --convert_geo -rf "nhle_spatial3.ttl" -i > > > > 11:26:01 INFO Main :: Arguments Received: [--convert_geo, > -rf, > > nhle_spatial3.ttl, -i] > > 11:26:01 INFO DatasetOperations :: Server Configuration: port=3030, > > datsetName=ds, loopbackOnly=true, updateAllowed=false, inference=true, > > applyDefaultGeometry=false, validateGeometryLiteral=false, > > convertGeoPredicates=true, removeGeoPredicates=false, queryRewrite=true, > > tdbFile=null, > fileGraphFormats=[FileGraphFormat{rdfFile=nhle_spatial3.ttl, > > graphName=, rdfFormat=Turtle/pretty}], fileGraphDelimiters=[], > > indexEnabled=true, indexSizes=[-1, -1, -1], indexExpiries=[5000, 5000, > > 5000], spatialIndexFile=null, tdb2=false, help=false > > 11:26:01 INFO DatasetOperations :: In-Memory Dataset > > 11:26:02 INFO DatasetOperations :: Reading RDF - Started - File: > > nhle_spatial3.ttl, Graph Name: , RDF Format: Turtle/pretty > > 11:26:02 WARN system :: The ôSIS_DATAö environment variable is > > not set. > > Exception in thread "main" > org.apache.jena.sparql.JenaTransactionException: > > Write transaction - no commit or abort before end() > > at > > > org.apache.jena.sparql.core.TransactionalLock.error(TransactionalLock.java:179) > > at > > > org.apache.jena.sparql.core.TransactionalLock.end(TransactionalLock.java:162) > > at > > org.apache.jena.sparql.core.DatasetGraphMap.end(DatasetGraphMap.java:80) > > at > org.apache.jena.sparql.core.DatasetImpl.end(DatasetImpl.java:164) > > at > > > org.apache.jena.fuseki.geosparql.DatasetOperations.loadData(DatasetOperations.java:170) > > at > > > org.apache.jena.fuseki.geosparql.DatasetOperations.setup(DatasetOperations.java:68) > > at org.apache.jena.fuseki.geosparql.Main.main(Main.java:64) > > > > > > > > > > On Mon, 12 Jul 2021 at 20:32, Andy Seaborne <[email protected]> wrote: > > > >> There would have been more logging and more stack trace > >> > >> but it looks like you are loading all the data into memory > >> Did it log "In-Memory Dataset"? > >> > >> How big is nhle_spatial5.ttl? How big is the machine it is running on? > >> > >> Andy > >> > >> On 12/07/2021 14:44, Matt Whitby wrote: > >>> Good afternoon all. > >>> > >>> I've been trying to import a TTL file into GeoSparql. It's worked fine > >> for > >>> smaller datasets, but now they're getting bigger (the production one > >> would > >>> be about 900mb) I'm getting an error. > >>> > >>> java -jar jena-fuseki-geosparql-3.17.0.jar --convert_geo -rf > >>> "nhle_spatial5.ttl" -i > >>> > >>> Error: > >>> > >>> Reading RDF - Started - File: nhle_spatial5.ttl, Graph Name: , RDF > >> Format: > >>> Turtle/pretty > >>> Exception in thread "main" > >> org.apache.jena.sparql.JenaTransactionException: > >>> Write transaction - no commit or abort before end() > >>> at > >>> > >> > org.apache.jena.sparql.core.TransactionalLock.error(TransactionalLock.java:179) > >>> > >>> Might I be correct in thinking it's a size issue? > >>> > >>> > >>> Kind Regards, > >>> M > >>> > >> > > > > > -- Matt Southend. Essex, England Guff follows.... Me: http://www.about.me/matt.whitby Photography: http://www.whitbyphoto.com Travels: http://www.whitbyadventures.com Music: http://www.last.fm/user/MattWhitby <http://www.last.fm/user/MattWhitby/%3C/a%3E> Reading: https://www.goodreads.com/user_challenges/19398505 Development: https://www.hackerrank.com/matt_whitby
