Benchmark data is here: https://doi.org/10.17632/jfrdpnb945.1
On 26.08.20 10:07, Andy Seaborne wrote: > > > > On 26/08/2020 08:36, Andy Seaborne wrote: >> Thanks for the report. >> I don't have access to the paper. > > https://t.co/ivcVMnPGEr > > via > > https://twitter.com/alphaverda/status/1276836317100421121 > > Unfortunately, the evaluation is a custom app for each database. > Without seeing the app, it's opaque. > > I was hoping to see which features of SHACL they used in their shapes. > While Jena covers a all of core and SPARQL, the more obscure ones are > maybe slow. But if they are comparing across databases, maybe they > went for an implemented subset. > > Where are the shapes they used? > > Andy > > "18 GB of heap memory" ... on a 16G machine. Hmm. > > >> >> I don't see a reference to TopQuadrant SHACL in the references. >> >> On 26/08/2020 07:23, Lorenz B. wrote: >>> Hi all, >>> >>> >>> as usual when I see something regarding Jena in recent publications: >>> "Benchmark for Performance Evaluation of SHACL Implementations in Graph >>> Databases" [1] >>> >>> As the title indicates, it's a benchmark about SHACL validation >>> performance. The benchmark comprise 58 SHACL shapes tested on I guess 1 >>> million triples. Jena got second place close to Stardog - which I think >>> is a success. >> >> Especially for something that isn't in the slightest optimized other >> than the fact it compiles the shapes to an execution tree. >> Incremental validation for transactions is "work in progress". >> >>> Some other metrics like memory consumption might be something to >>> investigate - not sure if those numbers make sense, but according to >>> the >>> paper Jena needs 14GB of RAM? RDF4J even 16GB, but Stardog only 1.2GB. >> >> Sounds suspect to me for one million triples. A TDB database on disk >> is likely less than 1G bytes. >> >> Of course, Jena uses the Java heap and that just grows until a GC >> happens but it's not all in use. > > >> >> (Stardog does a lot outside the heap) >> >>> >>> A weak point of Jena was "lack of documentation". >> >> That's fixable - what were they looking for? A hands on-guide to SHACL? >> >> Again, thanks >> Andy >> >>> >>> Anyways, good job Andy (and contributors). >>> >>> Happy to see comments and thoughts from your side. >>> >>> >>> Cheers, >>> >>> Lorenz >>> >>> >>> [1] https://link.springer.com/chapter/10.1007/978-3-030-57977-7_6 >>> >>>
