Hi David,
On 30/05/2022 07:27, Lorenz Buehmann wrote:
Hi David,
On 29.05.22 15:34, David Habgood wrote:
Hi,
I've been running Apache Jena Fuseki 4.5.0 in a docker container. I've
loaded data to it two ways: though the graph store protocol, and using
tdb2.tdbloader before starting Jena Fuseki. No issues with either,
however
I'm interested in what differences the two methods have.
With the graph store protocol, I can put larger RDF files 'close' to
where
the docker container is running and handle any network issues, so the
loads
have been fine. Loading data this way is convenient and allows updates
while Jena Fuseki is running. Are indexes continually updated as more
data
is loaded through the graph store protocol?
Yes. The storage database is updated as data is loaded.
Are there any other
disadvantages to this method or reasons it (may) not be advised for large
datasets? Conversely, I'm aware tdb2.tdbloader can load large
datasets, is
there any reason/s it should be used over graph store protocol?
The difference is how fast the data is loaded. The graph store protocol
doesn't do anything special for large data - it transactionally loads
the incoming stream.
The various varieties of loaders have one task - load large data. They
manipulate the internal datastructures of TDB directly, need exclusive
access and only apply when loading an initially empty database. If data
is already present, the loader command does a simple load like GSP.
("Large" being 100 million+ - hardware dependent and to some extent data
shape dependent as to the cut-over).
So less convenient but faster at scale.
Are there any other methods I should be considering (other than SPARQL
INSERT)?
Those are all the data loading methods.
I'll also be running GeoSPARQL Jena for some instances, and needing to
spatially index data. I think this will necessitate using tdb2.tdbloader
and generating the spatial index 'offline' before starting Jena/Fuseki
- or
are there other ways?
At least when you have the GeoSPARQL layer enabled in your Fuseki
assembler config, the index should be computed on the first start of
Fuseki just once and serialized at the configured destination. Only the
text index has to be generated offline before
Thanks
David Habgood
