1.) do you need convert_geo param?
2.) Xss does increase the stack size, if you run out of memory you
should increase Xmx to e.g. 4G or something depending on your machine.
If the geospatial index has been generated I'm wondering what is done
in-memory, so if you don't need the convert_geo option, you should omit
it and try again
On 15.07.21 13:28, Matt Whitby wrote:
Hi Andy, Lorenz..
I tried the TDBLoader and it generated the indexes, etc.
image.png
boot up geosparql with the tdb file and I run out of memory too.
java -jar jena-fuseki-geosparql-3.17.0.jar --convert_geo --tdb
spatialdb2 -i
/Exception in thread "main" java.lang.OutOfMemoryError: Java heap space/
Now, I'm sure the solution is to add a param akin to *-Xss8m* to book
the JVM with more memory.
java -jar *-Xss8m* jena-fuseki-geosparql-3.17.0.jar --convert_geo
--tdb spatialdb2
Does this sound right to you?
On Tue, 13 Jul 2021 at 22:09, Andy Seaborne <[email protected]
<mailto:[email protected]>> wrote:
On 13/07/2021 11:31, Matt Whitby wrote:
> Morning all.
>
>
> It's just on my laptop (though with 64gb of memory so more than
enough I
> would assume).
Unless you have changed something, it is using 16G.
However, it is not clear that it is a memory space issue.
>
> The file is about 850mb, so not that big in the scheme of things.
>
> I don't see any log files.
They have been printed to stdout as shown. They can go elsewhere
(it's
log4j2).
>
> The full stack trace is...
There was an Java Error (the code doesn't print it - an
oversight). Out
of memory is an error.
How long after the "DatasetOperations :: Reading RDF - Started -
File:"
output does it fail?
It is worth checking the file parses correctly. e.g. some encoding
errors become Java "errors" in 3.17.0.
> 11:26:01 INFO DatasetOperations :: In-Memory Dataset
That means there are multiple copies in-memory during loading.
This does not explain using 16G.
But the database can be loaded using the TDB bulkloader separately
from
the server starting and then pass in the persistent database so
the file
does not have to be read each start-up.
Andy
>
> C:\Data\apache-jena-fuseki-3.17.0>java -jar
> jena-fuseki-geosparql-3.17.0.jar --convert_geo -rf
"nhle_spatial3.ttl" -i
>
> 11:26:01 INFO Main :: Arguments Received:
[--convert_geo, -rf,
> nhle_spatial3.ttl, -i]
> 11:26:01 INFO DatasetOperations :: Server Configuration:
port=3030,
> datsetName=ds, loopbackOnly=true, updateAllowed=false,
inference=true,
> applyDefaultGeometry=false, validateGeometryLiteral=false,
> convertGeoPredicates=true, removeGeoPredicates=false,
queryRewrite=true,
> tdbFile=null,
fileGraphFormats=[FileGraphFormat{rdfFile=nhle_spatial3.ttl,
> graphName=, rdfFormat=Turtle/pretty}], fileGraphDelimiters=[],
> indexEnabled=true, indexSizes=[-1, -1, -1], indexExpiries=[5000,
5000,
> 5000], spatialIndexFile=null, tdb2=false, help=false
> 11:26:01 INFO DatasetOperations :: In-Memory Dataset
> 11:26:02 INFO DatasetOperations :: Reading RDF - Started - File:
> nhle_spatial3.ttl, Graph Name: , RDF Format: Turtle/pretty
> 11:26:02 WARN system :: The ôSIS_DATAö environment
variable is
> not set.
> Exception in thread "main"
org.apache.jena.sparql.JenaTransactionException:
> Write transaction - no commit or abort before end()
> at
>
org.apache.jena.sparql.core.TransactionalLock.error(TransactionalLock.java:179)
> at
>
org.apache.jena.sparql.core.TransactionalLock.end(TransactionalLock.java:162)
> at
>
org.apache.jena.sparql.core.DatasetGraphMap.end(DatasetGraphMap.java:80)
> at
org.apache.jena.sparql.core.DatasetImpl.end(DatasetImpl.java:164)
> at
>
org.apache.jena.fuseki.geosparql.DatasetOperations.loadData(DatasetOperations.java:170)
> at
>
org.apache.jena.fuseki.geosparql.DatasetOperations.setup(DatasetOperations.java:68)
> at
org.apache.jena.fuseki.geosparql.Main.main(Main.java:64)
>
>
>
>
> On Mon, 12 Jul 2021 at 20:32, Andy Seaborne <[email protected]
<mailto:[email protected]>> wrote:
>
>> There would have been more logging and more stack trace
>>
>> but it looks like you are loading all the data into memory
>> Did it log "In-Memory Dataset"?
>>
>> How big is nhle_spatial5.ttl? How big is the machine it is
running on?
>>
>> Andy
>>
>> On 12/07/2021 14:44, Matt Whitby wrote:
>>> Good afternoon all.
>>>
>>> I've been trying to import a TTL file into GeoSparql. It's
worked fine
>> for
>>> smaller datasets, but now they're getting bigger (the
production one
>> would
>>> be about 900mb) I'm getting an error.
>>>
>>> java -jar jena-fuseki-geosparql-3.17.0.jar --convert_geo -rf
>>> "nhle_spatial5.ttl" -i
>>>
>>> Error:
>>>
>>> Reading RDF - Started - File: nhle_spatial5.ttl, Graph Name: ,
RDF
>> Format:
>>> Turtle/pretty
>>> Exception in thread "main"
>> org.apache.jena.sparql.JenaTransactionException:
>>> Write transaction - no commit or abort before end()
>>> at
>>>
>>
org.apache.jena.sparql.core.TransactionalLock.error(TransactionalLock.java:179)
>>>
>>> Might I be correct in thinking it's a size issue?
>>>
>>>
>>> Kind Regards,
>>> M
>>>
>>
>
>
--
Matt
Southend. Essex, England
Guff follows....
Me: http://www.about.me/matt.whitby <http://www.about.me/matt.whitby>
Photography: http://www.whitbyphoto.com <http://www.whitbyphoto.com/>
Travels: http://www.whitbyadventures.com
<http://www.whitbyadventures.com/>
Music: http://www.last.fm/user/MattWhitby
<http://www.last.fm/user/MattWhitby/%3C/a%3E>
Reading: https://www.goodreads.com/user_challenges/19398505
<https://www.goodreads.com/user_challenges/19398505>
Development: https://www.hackerrank.com/matt_whitby
<https://www.hackerrank.com/matt_whitby>
