Re: java.lang.Error: Maximum lock count exceeded
It is Jena 4.8.0 . Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le dim. 16 juil. 2023 à 21:53, Andy Seaborne a écrit : > https://github.com/apache/jena/issues/1499 > > Are you are using 4.6.0? > > Andy > > On 16/07/2023 14:54, Jean-Marc Vanel wrote: > > Every few weeks, I get this stack, and the database is unusable > afterwards. > > Is there some "purge" to call now and then ? > > > > java.lang.Error: Maximum lock count exceeded > > at > > > java.base/java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:535) > > at > > > java.base/java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:494) > > at > > > java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1323) > > at > > > java.base/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738) > > at > > > org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.beginMultiMode(DatasetGraphTxnCtl.java:334) > > at > > > org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.tryNonExclusiveMode(DatasetGraphTxnCtl.java:252) > > at > > > org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.enterTransaction(DatasetGraphTxnCtl.java:110) > > at > > > org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.begin(DatasetGraphTxnCtl.java:74) > > at > > > org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.begin(DatasetGraphTxnCtl.java:99) > > at > > org.apache.jena.sparql.core.DatasetImpl.begin(DatasetImpl.java:120) > > > > Jean-Marc Vanel > > < > http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > +33 (0)6 89 16 29 52 > > Twitter: @jmvanel , @jmvanel_fr > > Chroniques jardin > > < > http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle > > > > >
java.lang.Error: Maximum lock count exceeded
Every few weeks, I get this stack, and the database is unusable afterwards. Is there some "purge" to call now and then ? java.lang.Error: Maximum lock count exceeded at java.base/java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:535) at java.base/java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:494) at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1323) at java.base/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738) at org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.beginMultiMode(DatasetGraphTxnCtl.java:334) at org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.tryNonExclusiveMode(DatasetGraphTxnCtl.java:252) at org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.enterTransaction(DatasetGraphTxnCtl.java:110) at org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.begin(DatasetGraphTxnCtl.java:74) at org.apache.jena.dboe.storage.system.DatasetGraphTxnCtl.begin(DatasetGraphTxnCtl.java:99) at org.apache.jena.sparql.core.DatasetImpl.begin(DatasetImpl.java:120) Jean-Marc Vanel <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: JSON-LD : ERROR with riot, but not with com.apicatalog.jsonld API
Le mer. 11 mai 2022 à 11:51, Andy Seaborne a écrit : > > On 11/05/2022 09:12, Jean-Marc Vanel wrote: > > > As a side question, is there a way to configure riot or Jena API to have > > (more) verbose JSON-LD error messages, possibly with line numbers. > > Does titanium provide this information? > line numbers, not currently ; but JsonPath could be produced by Titanium ; see (stale) issue https://github.com/filip26/titanium-json-ld/issues/71 See also my issue where discussion about JSON-LD error messages started: https://github.com/filip26/titanium-json-ld/issues/59 Maybe Jena could catch com.apicatalog.jsonld.JsonLdError ... Anyway the question remains of why does Titanium succeed where Jena / riot fails on the same data ? How can I help investigate ? NOTE, Titanium has a CLI : JSON-LD-CLI <https://github.com/filip26/json-ld-cli> is a native command line utility for Ubuntu, Mac, Windows
Re: JSON-LD : ERROR with riot, but not with com.apicatalog.jsonld API
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le mer. 11 mai 2022 à 11:48, Andy Seaborne a écrit :
> What version of java are you running? Is it 11.0.14?
>
NO
java --version
openjdk *11.0.15* 2022-04-19
OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1)
OpenJDK 64-Bit Server VM (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1, mixed
mode, sharing)
>
> Does 78272_pp-absolute.jsonld use a https URL in @context?
>
No , for the purpose of easy testing , I put file: URL's everywhere .
>
> You can use --syntax jsonld10 to get jsonld-java.
>
OK. Not my use currently .
> Andy
>
> On 11/05/2022 09:12, Jean-Marc Vanel wrote:
> > The context is that I'm debugging a JSON API with JSON-LD keys and an
> > @context that I recommended .
> >
> > riot from Jena 4.5.0 says :
> > $JENA/bin/riot --syntax=json-ld --verbose --debug
> 78272_pp-absolute.jsonld
> > 09:42:21 ERROR riot:: There was a problem encountered
> loading a
> > remote context [code=LOADING_REMOTE_CONTEXT_FAILED].
> >
> > but using directly JsonLd.toRdf() succeeds :
> >
> > import com.apicatalog.jsonld._
> > import scala.collection.JavaConverters._
> > val res2 = JsonLd.toRdf("file:/home/jmv/data/
> > grottocenter.org/tests_2022-05-10/78272_pp-absolute.jsonld")
> >.get() .toList.asScala
> > for ( e <- res2 )
> > println ( "<" + e.getSubject + "> <" + e.getPredicate + "> " +
> > e.getObject )
> >
> > <_:b0> <http://purl.org/dc/terms/created> 2022-04-14T10:44:18.228Z^^
> > http://www.w3.org/2001/XMLSchema#string
> > <_:b0> <http://purl.org/dc/terms/creator> 3188^^
> > http://www.w3.org/2001/XMLSchema#integer
> > <_:b0> <http://purl.org/dc/terms/spatial> 78272^^
> > http://www.w3.org/2001/XMLSchema#integer
> > <_:b0> <http://www.w3.org/2000/01/rdf-schema#label> Trou de Beget^^
> > http://www.w3.org/2001/XMLSchema#string
> > etc ...
> >
> > I'm afraid that it would not work with Jena API either.
> > As a side question, is there a way to configure riot or Jena API to have
> > (more) verbose JSON-LD error messages, possibly with line numbers.
> > Another side question, it seems that riot no longer accepts HTTP URL's ,
> > only files. Of course , it's easy to use wget or other tools, but that
> was
> > convenient ...
> >
> > The test data are here : http://jmvanel.free.fr/test/?M=D
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33 (0)6 89 16 29 52
> > Twitter: @jmvanel , @jmvanel_fr
> > Chroniques jardin
> > <
> http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle
> >
> >
>
JSON-LD : ERROR with riot, but not with com.apicatalog.jsonld API
The context is that I'm debugging a JSON API with JSON-LD keys and an
@context that I recommended .
riot from Jena 4.5.0 says :
$JENA/bin/riot --syntax=json-ld --verbose --debug 78272_pp-absolute.jsonld
09:42:21 ERROR riot:: There was a problem encountered loading a
remote context [code=LOADING_REMOTE_CONTEXT_FAILED].
but using directly JsonLd.toRdf() succeeds :
import com.apicatalog.jsonld._
import scala.collection.JavaConverters._
val res2 = JsonLd.toRdf("file:/home/jmv/data/
grottocenter.org/tests_2022-05-10/78272_pp-absolute.jsonld")
.get() .toList.asScala
for ( e <- res2 )
println ( "<" + e.getSubject + "> <" + e.getPredicate + "> " +
e.getObject )
<_:b0> <http://purl.org/dc/terms/created> 2022-04-14T10:44:18.228Z^^
http://www.w3.org/2001/XMLSchema#string
<_:b0> <http://purl.org/dc/terms/creator> 3188^^
http://www.w3.org/2001/XMLSchema#integer
<_:b0> <http://purl.org/dc/terms/spatial> 78272^^
http://www.w3.org/2001/XMLSchema#integer
<_:b0> <http://www.w3.org/2000/01/rdf-schema#label> Trou de Beget^^
http://www.w3.org/2001/XMLSchema#string
etc ...
I'm afraid that it would not work with Jena API either.
As a side question, is there a way to configure riot or Jena API to have
(more) verbose JSON-LD error messages, possibly with line numbers.
Another side question, it seems that riot no longer accepts HTTP URL's ,
only files. Of course , it's easy to use wget or other tools, but that was
convenient ...
The test data are here : http://jmvanel.free.fr/test/?M=D
Jean-Marc Vanel
<http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
jena sparql extension for querying nominatim API or other API
Is there a Jena sparql extension for querying Nominatim API or other web API returning location name (or even URI from Wikidata) from coordinates? Something like ( 4.0 , 45.0 ) ext:locationName ?NAME . I found about the existing Jena function library, and the java: URI extension mechanism: https://jena.apache.org/documentation/query/java-uri.html Jean-Marc Vanel <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Datatype format exception : float - detect the incorrect triple
The situation with my data is not so bad !
Doing SPARQL query on rdf:type of my 2 main classes works fine (I have only
a handful of user entered RDF classes, the rest are copies of Internet RDF
data ).
And strangely, I get correct float data for this particular triple in the
original post:
<http://semantic-forms.cc:1952/ldp/1640450652251-3388247986206631>
<http://www.w3.org/2003/01/geo/wgs84_pos#long>
*"4.77962"*^^<http://www.w3.org/2001/XMLSchema#float> .
So my binary TDB database is corrupted, but once again, I'm able to recover
the important data.
And that 's an opportunity to test TDB2 ...
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le sam. 1 janv. 2022 à 22:36, Andy Seaborne a écrit :
>
>
> On 01/01/2022 20:50, Andy Seaborne wrote:
> >
> >
> > On 01/01/2022 19:45, Jean-Marc Vanel wrote:
> >> During TDB initialization, I get this warning :
> >> WARN o.apache.jena.sparql.expr.NodeValue - Datatype format exception:
> >> "4,77962"^^xsd:float
> >>
> >> But I am unable to detect the incorrect triple , with this query :
> >> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
> >> SELECT * WHERE {
> >>?S ?P "4,77962"^^xsd:float .
> >> }
> >>
> >> I also tried this:
> >> SELECT * WHERE {
> >>?S ?P ?O.
> >>FILTER ( STR(?O) = "4,77962" )
> >
> > Try
> >
> > FILTER ( CONTAINS(STR(?O), "779") )
> >
> >> }
> >>
> >> This uses the latest jena Version 4.3.2 , with TDB 1 .
> >> I suspect this bad data prevents geosparl queries from getting any
> >> result.
>
> "4,77962"^^xsd:float has no value.
>
> So if you are using geo, you have a lat without a long or viceversa.
> It's not legal data and will not be indexes or in the results of a
> geo-spatial query.
>
> >
> > Did it work in a previous version?
> > Did you load the TDB database with 4.3.2?
> > Does it work with TDB2?
> > Does the original data have a comma in a float?
> >
> > Andy
> >
> >>
> >> Jean-Marc Vanel
> >> <
> http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
>
> >>
> >> +33 (0)6 89 16 29 52
> >> Twitter: @jmvanel , @jmvanel_fr
> >>
>
Re: Datatype format exception : float - detect the incorrect triple
Le sam. 1 janv. 2022 à 21:50, Andy Seaborne a écrit :
>
> On 01/01/2022 19:45, Jean-Marc Vanel wrote:
> > During TDB initialization, I get this warning :
> > WARN o.apache.jena.sparql.expr.NodeValue - Datatype format exception:
> > "4,77962"^^xsd:float
> >
> > But I am unable to detect the incorrect triple , with this query :
> > PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
> > SELECT * WHERE {
> >?S ?P "4,77962"^^xsd:float .
> > }
> >
> > I also tried this:
> > SELECT * WHERE {
> >?S ?P ?O.
> >FILTER ( STR(?O) = "4,77962" )
>
> Try
>
> FILTER ( CONTAINS(STR(?O), "779") )
>
See result below.
> }
> >
> > This uses the latest jena Version 4.3.2 , with TDB 1 .
> > I suspect this bad data prevents geosparl queries from getting any
> result.
>
> Did it work in a previous version?
>
Yes, that's an old database that I kept using for maybe 3 years .
> Did you load the TDB database with 4.3.2?
>
No.
> Does it work with TDB2?
>
Didn't try .
> Does the original data have a comma in a float?
>
yes it had that; I tried to remove it but it seems that there is still some
bad data present.
It seems that my database is corrupted, the query does not terminate. And
alas I have non recent backup.
If I try to make a backup now it stops with the same error and stack as
below: *AtlasException: Unknown escape: \< *. Happily it produces a partial
dump .
cat test-andy.rq
SELECT * WHERE {
graph ?g {
?S ?P ?O.
FILTER ( CONTAINS(STR(?O), "779") )
}}
$JENA/bin/tdbquery --loc=TDB --query test-andy.rq
21:30:31 WARN NodeValue :: Datatype format exception:
"-27-01-01T00:00:00Z"^^xsd:dateTime
21:30:32 WARN NodeValue :: Datatype format exception:
"1778-1-10"^^xsd:date
21:30:32 WARN NodeValue :: Datatype format exception:
"1707-5-23"^^xsd:date
21:30:32 WARN NodeValue :: Datatype format exception:
"2013-6-23"^^xsd:date
21:30:32 WARN NodeValue :: Datatype format exception:
"-287"^^xsd:gYear
21:30:32 WARN NodeValue :: Datatype format exception:
"-371"^^xsd:gYear
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(5.71,None,None)"^^xsd:double
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(1427,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(6925,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(2012,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(1600,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(45,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(1212.8,None,None)"^^xsd:double
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(167,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(8.48,None,None)"^^xsd:double
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(69278,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(4029,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(1990,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(69730,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(45,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(475,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"1885-9-25"^^xsd:date
21:30:33 WARN NodeValue :: Datatype format exception:
"1810-5-10"^^xsd:date
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(2009,None,None)"^^xsd:integer
21:30:33 WARN NodeValue :: Datatype format exception:
"ParseResult(978,None,None)"^^xsd:integer
21:30:34 WARN NodeValue :: Datatype format exception:
"ParseResult(35,None,None)"^^xsd:integer
21:30:34 WARN NodeValue :: Datatype format exception:
"ParseResult(40,None,None)"^^xsd:integer
21:30:34 WARN NodeValue :: Datatype format exception:
"ParseResult(1933,None,None)"^^xsd:integer
21:30:34 WARN NodeValue :: Datatype format exception:
"ParseResult(2003,None,None)"^^xsd:integer
21:30:34 WARN NodeValue :: Datatype format exception:
"ParseResult(83,None,None)"^^xsd:integ
Datatype format exception : float - detect the incorrect triple
During TDB initialization, I get this warning :
WARN o.apache.jena.sparql.expr.NodeValue - Datatype format exception:
"4,77962"^^xsd:float
But I am unable to detect the incorrect triple , with this query :
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT * WHERE {
?S ?P "4,77962"^^xsd:float .
}
I also tried this:
SELECT * WHERE {
?S ?P ?O.
FILTER ( STR(?O) = "4,77962" )
}
This uses the latest jena Version 4.3.2 , with TDB 1 .
I suspect this bad data prevents geosparl queries from getting any result.
Jean-Marc Vanel
<http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr
Re: Error initializing geosparql
I fixed the WKT not having the right datatype, as said before; here are the
SPARQL used to check and fix:
COUNT-spatial-wkt-as-string.rq
<https://github.com/jmvanel/semantic_forms/blob/master/sparql/COUNT-spatial-wkt-as-string.rq>
FIX-spatial-wkt-as-string.upd.rq
<https://github.com/jmvanel/semantic_forms/blob/master/sparql/FIX-spatial-wkt-as-string.upd.rq>
Now this is not the end of the road . Another imperfect data causing
geosparql initialization to fail :
*Exception: Build WKT Geometry Exception - Type: point, Coordinates:
(2.353821,48.83399,0). Index 1 out of bounds for length 1*
2021-12-05T15:48:54.166Z [application-akka.actor.default-dispatcher-5]
ERROR jena - Exception class:class
org.apache.jena.datatypes.DatatypeFormatException
2021-12-05T15:48:54.167Z [application-akka.actor.default-dispatcher-5]
ERROR jena - Exception
org.apache.jena.datatypes.DatatypeFormatException: Build WKT Geometry
Exception - Type: point, Coordinates: (2.353821,48.83399,0). Index 1 out of
bounds for length 1
org.apache.jena.geosparql.implementation.parsers.wkt.WKTReader.buildGeometry(WKTReader.java:141)
org.apache.jena.geosparql.implementation.parsers.wkt.WKTReader.(WKTReader.java:50)
org.apache.jena.geosparql.implementation.parsers.wkt.WKTReader.extract(WKTReader.java:292)
org.apache.jena.geosparql.implementation.datatype.WKTDatatype.read(WKTDatatype.java:89)
org.apache.jena.geosparql.implementation.index.GeometryLiteralIndex.retrieveMemoryIndex(GeometryLiteralIndex.java:69)
org.apache.jena.geosparql.implementation.index.GeometryLiteralIndex.retrieve(GeometryLiteralIndex.java:51)
org.apache.jena.geosparql.implementation.datatype.GeometryDatatype.parse(GeometryDatatype.java:57)
org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1176)
org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1137)
org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1147)
org.apache.jena.geosparql.configuration.ModeSRS.search(ModeSRS.java:61)
org.apache.jena.geosparql.configuration.GeoSPARQLOperations.findModeSRS(GeoSPARQLOperations.java:520)
Is it because the WKT separator should be a space instead of a comma, or
because 3D is not allowed ?
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le dim. 5 déc. 2021 à 13:03, Jean-Marc Vanel a
écrit :
> After looking at this code, failing line in bold:
>
> jena-geosparql/src/main/java/org/apache/jena/geosparql/configuration/ModeSRS.java
>
> https://github.com/apache/jena/blob/main/jena-geosparql/src/main/java/org/apache/jena/geosparql/configuration/ModeSRS.java
>
> ExtendedIterator nodeIter =
> model.listObjectsOfProperty(Geo.HAS_SERIALIZATION_PROP);
> boolean isGeometryLiteralsFound = nodeIter.hasNext();
> if (!isGeometryLiteralsFound) {
> NodeIterator wktNodeIter =
> model.listObjectsOfProperty(Geo.AS_WKT_PROP);
> NodeIterator gmlNodeIter =
> model.listObjectsOfProperty(Geo.AS_GML_PROP);
> nodeIter = wktNodeIter.andThen(gmlNodeIter);
> }
>
> while (nodeIter.hasNext()) {
> RDFNode node = nodeIter.next();
> if (node.isLiteral()) {
> *GeometryWrapper geometryWrapper =
> GeometryWrapper.extract(node.asLiteral());*
>
> I did SELECT queries to try to understand what is wrong .
> It appears that these triples are not present:
> ?S <http://www.opengis.net/ont/geosparql#hasSerialization> ?O .
> ?S <http://www.opengis.net/ont/geosparql#asGML> ?O .
>
> BUT, there is a bunch of these triples:
> ?S <http://www.opengis.net/ont/geosparql#asWKT> ?O .
>
> Here is one example of the object values:
> "POINT(-4.189911,54.880557,0)"
> I probably imported them by hacking a JSON API as JSON-LD , I have to
> check my journals ...
>
> Looking at the OGC GeoSPARQL standard, I saw that the WKT strings should
> have this datatype :
> http://www.opengis.net/ont/geosparql#wktLiteral
>
> So I can make a SPARQL update to FIX my data .
> But maybe Jena GeoSPARQL could be forgiving about the string datatype for
> WKT data .
> And the error message should be more explicit ...
>
> Thanks Andy for the quick answer.
>
> Jean-Marc Vanel
> <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
> +33
> (0)6 89 16 29 52
>
>
> Le dim. 5 déc. 2021 à 11:52, Jean-Marc Vanel a
> écrit :
>
>> After having fixed bad data in the TDB database (latitude is present but
>> not longitude, coordinates as strings , see issue
>> https://issues.apache.org/jira/browse/JENA-2202 ),
>> there is an exception, probably related to the database content.
>> Here is the
Re: Error initializing geosparql
After looking at this code, failing line in bold:
jena-geosparql/src/main/java/org/apache/jena/geosparql/configuration/ModeSRS.java
https://github.com/apache/jena/blob/main/jena-geosparql/src/main/java/org/apache/jena/geosparql/configuration/ModeSRS.java
ExtendedIterator nodeIter =
model.listObjectsOfProperty(Geo.HAS_SERIALIZATION_PROP);
boolean isGeometryLiteralsFound = nodeIter.hasNext();
if (!isGeometryLiteralsFound) {
NodeIterator wktNodeIter =
model.listObjectsOfProperty(Geo.AS_WKT_PROP);
NodeIterator gmlNodeIter =
model.listObjectsOfProperty(Geo.AS_GML_PROP);
nodeIter = wktNodeIter.andThen(gmlNodeIter);
}
while (nodeIter.hasNext()) {
RDFNode node = nodeIter.next();
if (node.isLiteral()) {
*GeometryWrapper geometryWrapper =
GeometryWrapper.extract(node.asLiteral());*
I did SELECT queries to try to understand what is wrong .
It appears that these triples are not present:
?S <http://www.opengis.net/ont/geosparql#hasSerialization> ?O .
?S <http://www.opengis.net/ont/geosparql#asGML> ?O .
BUT, there is a bunch of these triples:
?S <http://www.opengis.net/ont/geosparql#asWKT> ?O .
Here is one example of the object values:
"POINT(-4.189911,54.880557,0)"
I probably imported them by hacking a JSON API as JSON-LD , I have to check
my journals ...
Looking at the OGC GeoSPARQL standard, I saw that the WKT strings should
have this datatype :
http://www.opengis.net/ont/geosparql#wktLiteral
So I can make a SPARQL update to FIX my data .
But maybe Jena GeoSPARQL could be forgiving about the string datatype for
WKT data .
And the error message should be more explicit ...
Thanks Andy for the quick answer.
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le dim. 5 déc. 2021 à 11:52, Jean-Marc Vanel a
écrit :
> After having fixed bad data in the TDB database (latitude is present but
> not longitude, coordinates as strings , see issue
> https://issues.apache.org/jira/browse/JENA-2202 ),
> there is an exception, probably related to the database content.
> Here is the log:
> Dec 05, 2021 9:57:33 AM org.apache.sis.referencing.factory.sql.EPSGFactory
>
> WARNING: The “SIS_DATA” environment variable is not set.
> 2021-12-05T09:57:33.940Z [application-akka.actor.default-dispatcher-9]
> INFO jena - SpatialIndex: isFunctionRegistered true
> 2021-12-05T09:57:33.941Z [application-akka.actor.default-dispatcher-9]
> INFO jena - Before setupSpatialIndex
> 2021-12-05T09:57:33.948Z [application-akka.actor.default-dispatcher-9]
> INFO o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Started
>
> And here is the exception:
> *Exception: Unrecognised Geometry Datatype:
> http://www.w3.org/2001/XMLSchema#string
> <http://www.w3.org/2001/XMLSchema#string> Ensure that Datatype is extending
> GeometryDatatype.*
>
> org.apache.jena.geosparql.implementation.datatype.GeometryDatatype.get(GeometryDatatype.java:78)
>
> org.apache.jena.geosparql.implementation.datatype.GeometryDatatype.get(GeometryDatatype.java:86)
>
> org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1175)
>
> org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1137)
>
> org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1147)
> org.apache.jena.geosparql.configuration.ModeSRS.search(ModeSRS.java:61)
>
> org.apache.jena.geosparql.configuration.GeoSPARQLOperations.findModeSRS(GeoSPARQLOperations.java:520)
>
> org.apache.jena.geosparql.spatial.SpatialIndex.buildSpatialIndex(SpatialIndex.java:336)
>
> org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupSpatialIndex(GeoSPARQLConfig.java:263)
>
> deductions.runtime.jena.RDFStoreLocalJenaProviderObject$.createDatabase(RDFStoreLocalJenaProvider.scala:175)
>
> I use the latest Jena release 4.2.0 . Note that there is no trouble on my
> development machine, only on the production site , although the source is
> the same .
>
> Jean-Marc Vanel
>
>
Error initializing geosparql
After having fixed bad data in the TDB database (latitude is present but not longitude, coordinates as strings , see issue https://issues.apache.org/jira/browse/JENA-2202 ), there is an exception, probably related to the database content. Here is the log: Dec 05, 2021 9:57:33 AM org.apache.sis.referencing.factory.sql.EPSGFactory WARNING: The “SIS_DATA” environment variable is not set. 2021-12-05T09:57:33.940Z [application-akka.actor.default-dispatcher-9] INFO jena - SpatialIndex: isFunctionRegistered true 2021-12-05T09:57:33.941Z [application-akka.actor.default-dispatcher-9] INFO jena - Before setupSpatialIndex 2021-12-05T09:57:33.948Z [application-akka.actor.default-dispatcher-9] INFO o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Started And here is the exception: *Exception: Unrecognised Geometry Datatype: http://www.w3.org/2001/XMLSchema#string <http://www.w3.org/2001/XMLSchema#string> Ensure that Datatype is extending GeometryDatatype.* org.apache.jena.geosparql.implementation.datatype.GeometryDatatype.get(GeometryDatatype.java:78) org.apache.jena.geosparql.implementation.datatype.GeometryDatatype.get(GeometryDatatype.java:86) org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1175) org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1137) org.apache.jena.geosparql.implementation.GeometryWrapper.extract(GeometryWrapper.java:1147) org.apache.jena.geosparql.configuration.ModeSRS.search(ModeSRS.java:61) org.apache.jena.geosparql.configuration.GeoSPARQLOperations.findModeSRS(GeoSPARQLOperations.java:520) org.apache.jena.geosparql.spatial.SpatialIndex.buildSpatialIndex(SpatialIndex.java:336) org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupSpatialIndex(GeoSPARQLConfig.java:263) deductions.runtime.jena.RDFStoreLocalJenaProviderObject$.createDatabase(RDFStoreLocalJenaProvider.scala:175) I use the latest Jena release 4.2.0 . Note that there is no trouble on my development machine, only on the production site , although the source is the same . Jean-Marc Vanel
Spatial Index creation can fail , FIX proposed
In the special case when latitude is present but not longitude, a null
pointer exception is raised, which kills the Spatial initialization.
In
jena-geosparql/src/main/java/org/apache/jena/geosparql/spatial/SpatialIndex.java
in method getGeoPredicateIndexItems() line 453 (latest git code),
it's enough to add a continue instruction, to allow the loop on RDF
statements to finish:
GeometryWrapper geometryWrapper = null;
try {
Literal lon =
feature.getRequiredProperty(SpatialExtension.GEO_LON_PROP).getLiteral();
Literal latLonPoint =
ConvertLatLon.toLiteral(lat.getFloat(), lon.getFloat());
geometryWrapper = GeometryWrapper.extract(latLonPoint);
} catch (org.apache.jena.shared.PropertyNotFoundException ex) {
LOGGER.warn("Geo predicates: Lat found but not Lon. " +
feature);
*continue;*
}
Jean-Marc Vanel
<http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ;
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Followup GeoSparql + TDB 1 : freeze
For GeoSparql + TDB 1, I applied this initialization code on my site:
val res = configureLuceneIndex(dts, useTextQuery, useSpatialIndex)
if (useSpatialIndex) {
import org.apache.jena.geosparql.configuration._
GeoSPARQLConfig.setupMemoryIndex // actually registers special SPARQL
predicates!
GeoSPARQLConfig.setupSpatialIndex(res)
}
Full code:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/RDFStoreLocalJenaProvider.scala#L164
and I could indeed make "rectangular" queries on my geo: (
http://www.w3.org/2003/01/geo/wgs84_pos# ) data,
but even when not running any GeoSparql query,
the application web site got frozen, several times, so I was obliged to
deactivate GeoSparql initialization.
here is an example of a stack of a number of threads that run for a long
time and retain memory:
#132 daemon prio=5 os_prio=0 cpu=44.41ms elapsed=215.79s
tid=0x7f69e8012000 nid=0x2370 waiting on condition [0x7f68826d4000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.10/Native Method)
- parking to wait for <0xbcd9d198> (a
java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.10
/LockSupport.java:194)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.10
/AbstractQueuedSynchronizer.java:885)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@11.0.10
/AbstractQueuedSynchronizer.java:917)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11.0.10
/AbstractQueuedSynchronizer.java:1240)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(java.base@11.0.10
/ReentrantReadWriteLock.java:959)
at
org.apache.jena.tdb.transaction.TransactionManager.startExclusiveMode(TransactionManager.java:706)
at
org.apache.jena.tdb.transaction.TransactionManager.exclusiveFlushQueue(TransactionManager.java:661)
at
org.apache.jena.tdb.transaction.TransactionManager.notifyCommit(TransactionManager.java:567)
at
org.apache.jena.tdb.transaction.Transaction.commit(Transaction.java:143)
at
org.apache.jena.tdb.transaction.DatasetGraphTxn.commit(DatasetGraphTxn.java:61)
at
org.apache.jena.tdb.transaction.DatasetGraphTransaction.commit(DatasetGraphTransaction.java:200)
at
org.apache.jena.sparql.core.DatasetImpl.commit(DatasetImpl.java:152)
at
org.w3.banana.jena.JenaDatasetStore.$anonfun$rw$1(JenaDatasetStore.scala:26)
at
org.w3.banana.jena.JenaDatasetStore$$Lambda$1102/0x000840a8b840.apply(Unknown
Source)
at scala.util.Try$.apply(Try.scala:213)
at org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:22)
at org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:10)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction(SPARQLHelpers.scala:237)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction$(SPARQLHelpers.scala:235)
at
controllers.WebPagesApp$$anon$1.wrapInTransaction(WebPages.scala:51)
at
deductions.runtime.abstract_syntax.InstanceLabelsInferenceMemory.$anonfun$makeInstanceLabelFuture$1(InstanceLabelsInferenceMemory.scala:49)
This transaction just adds one literal triple to TDB.
There are several such stacks present, seen when executing kill -3 when the
web application is frozen:
http://jmvanel.free.fr/tmp/thread_dump.txt
Jean-Marc Vanel
<http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Spatial index automatically updated on TDB change ?
I don't think this is possible in current API, similar to what happens by default in TextDataset . My use case is biodiversity field observations with GPS positions . I guess an issue would be appropriate. Jean-Marc Vanel <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
tested GeoSPARQL + Lucene API's together: it works!
I tested GeoSPARQL + Lucene API's together, extending previous code for GeoSPARQL alone. Correct results are obtained for both types of queries: GeoSparql & text. Sketch of TDB initialization: textualDataset = TextDatasetFactory.create(dataset, textIndex) load2Cities(textualDataset) GeoSPARQLConfig.setupSpatialIndex(textualDataset) GeoSPARQLConfig.setupMemoryIndex() Source code is here: https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GeoSPARQLluceneTest.scala#L20 https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GeoSPARQLtest.scala#L13 That's with TDB 1 and Jena 3.17 . I suppose it would work also for TDB 2 and Jena 4.0.0-SNAPSHOT ... Jean-Marc Vanel <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: New attempt with GeoSparql API
About performance, all I can say is that indexing 2 dbPedia cities takes 608 ms elapsed time from scratch, and re-indexing after loading one more city takes 4 ms . This is acceptable, and hopefully the re-indexing time is mostly dependent on the increment, not on the overall size of already indexed spatial data. I'll try GeoSPARQL + Lucene, to see for myself; I see no fundamental reason preventing having two different indices on a database (actually altogether 16 , the 12 TDB/*.idn plus Lucene plus GeoSparql ). Time permitting, I also want to try 4.0.0-SNAPSHOT. Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le sam. 27 févr. 2021 à 10:14, Marco Neumann a écrit : > On Sat, Feb 27, 2021 at 8:48 AM Jean-Marc Vanel > wrote: > > > The result is now correct. The missing call is > > GeoSPARQLConfig.setupMemoryIndex() > > source code updated: > > > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GeoSPARLtest.scala#L11 > > > > NOTES > > > >- but need to re-index after RDF addition; > >- setupMemoryIndex() actually registers special SPARQL > predicates, > >which is not apparent in method name; > >- QUESTIONS: > > - how expensive in terms of CPU, elapsed time and storage is > > re-indexing? > > > you will have to test that yourself > did you check out the > > https://github.com/galbiston/geosparql-benchmarking > > and > > https://github.com/OpenLinkSoftware/GeoSPARQLBenchmark > > would be nice to compare them > > > - how to make re-index automatic? > > > > they should be, of course it depends on your conformance requirements with > OGC Geosparql as well. query rewriting requires inferencing. try the > standalone implementations for your tests first, > > > > - is GeoSPARQL indexing compatible with Lucene indexing? > > > > no, the geospatial modul uses a different approach to indexing. The lucene > index is not directly resusably in the Apache Jena geosparql module. But > Andy mentioned a resurrection of our lucene spatial integration with Jena > 4. > > we may approach this integration with a compliance register in mind in the > future from a OGC GeoSPARQL 1,0 conformance level point of view. > > > > > > Jean-Marc Vanel > > < > > > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > > +33 > > (0)6 89 16 29 52 > > > > > > Le mer. 24 févr. 2021 à 09:17, Jean-Marc Vanel > > > a > > écrit : > > > > > The Scala code is here; > > > > > > > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GeoSPARLtest.scala#L11 > > > starting with empty TDB 1; just load 2 dbPedia cities with geo: > > > coordinates, initialize with > > > GeoSPARQLConfig.setupSpatialIndex(dataset) > > > and query with spatial:withinBox . > > > Alas, the result is empty (see bold line). > > > > > > Log output : > > > 2021-02-24T08:04:14.609Z [run-main-6] INFO > o.a.j.g.c.GeoSPARQLOperations > > > - Find Mode SRS - Started > > > 2021-02-24T08:04:14.633Z [run-main-6] INFO > o.a.j.g.c.GeoSPARQLOperations > > > - Find Mode SRS - Completed > > > 2021-02-24T08:04:14.634Z [run-main-6] INFO > > > o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - > Started > > > 2021-02-24T08:04:14.634Z [run-main-6] INFO > > > o.a.j.geosparql.spatial.SpatialIndex - Geo predicate statements found. > > > févr. 24, 2021 8:04:14 AM > > > org.apache.sis.referencing.factory.sql.EPSGFactory > > > > > > *AVERTISSEMENT: La variable environnementale « SIS_DATA » n’est pas > > > définie.*2021-02-24T08:04:14.973Z [run-main-6] INFO > > > o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - > > Completed > > > > > > *?feature*[success] Total time: 5 s, completed 24 févr. 2021 à 08:04:15 > > > > > > If someone wants Java code to try, send me a private mail and I'll > write > > > it :) . > > > > > > Jean-Marc Vanel > > > < > > > http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > > > +33 (0)6 89 16 29 52 > > > Twitter: @jmvanel , @jmvanel_fr ; chat: irc:// > irc.freenode.net#eulergui > > > Chroniques jardin > > > < > > > http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle > > > > > > > > > > > > > > -- > > > --- > Marco Neumann > KONA >
Re: New attempt with GeoSparql API
The result is now correct. The missing call is GeoSPARQLConfig.setupMemoryIndex() source code updated: https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GeoSPARLtest.scala#L11 NOTES - but need to re-index after RDF addition; - setupMemoryIndex() actually registers special SPARQL predicates, which is not apparent in method name; - QUESTIONS: - how expensive in terms of CPU, elapsed time and storage is re-indexing? - how to make re-index automatic? - is GeoSPARQL indexing compatible with Lucene indexing? Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le mer. 24 févr. 2021 à 09:17, Jean-Marc Vanel a écrit : > The Scala code is here; > > https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GeoSPARLtest.scala#L11 > starting with empty TDB 1; just load 2 dbPedia cities with geo: > coordinates, initialize with > GeoSPARQLConfig.setupSpatialIndex(dataset) > and query with spatial:withinBox . > Alas, the result is empty (see bold line). > > Log output : > 2021-02-24T08:04:14.609Z [run-main-6] INFO o.a.j.g.c.GeoSPARQLOperations > - Find Mode SRS - Started > 2021-02-24T08:04:14.633Z [run-main-6] INFO o.a.j.g.c.GeoSPARQLOperations > - Find Mode SRS - Completed > 2021-02-24T08:04:14.634Z [run-main-6] INFO > o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - Started > 2021-02-24T08:04:14.634Z [run-main-6] INFO > o.a.j.geosparql.spatial.SpatialIndex - Geo predicate statements found. > févr. 24, 2021 8:04:14 AM > org.apache.sis.referencing.factory.sql.EPSGFactory > > *AVERTISSEMENT: La variable environnementale « SIS_DATA » n’est pas > définie.*2021-02-24T08:04:14.973Z [run-main-6] INFO > o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - Completed > > *?feature*[success] Total time: 5 s, completed 24 févr. 2021 à 08:04:15 > > If someone wants Java code to try, send me a private mail and I'll write > it :) . > > Jean-Marc Vanel > <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> > +33 (0)6 89 16 29 52 > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > Chroniques jardin > <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle> > >
New attempt with GeoSparql API
The Scala code is here; https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GeoSPARLtest.scala#L11 starting with empty TDB 1; just load 2 dbPedia cities with geo: coordinates, initialize with GeoSPARQLConfig.setupSpatialIndex(dataset) and query with spatial:withinBox . Alas, the result is empty (see bold line). Log output : 2021-02-24T08:04:14.609Z [run-main-6] INFO o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Started 2021-02-24T08:04:14.633Z [run-main-6] INFO o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Completed 2021-02-24T08:04:14.634Z [run-main-6] INFO o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - Started 2021-02-24T08:04:14.634Z [run-main-6] INFO o.a.j.geosparql.spatial.SpatialIndex - Geo predicate statements found. févr. 24, 2021 8:04:14 AM org.apache.sis.referencing.factory.sql.EPSGFactory *AVERTISSEMENT: La variable environnementale « SIS_DATA » n’est pas définie.*2021-02-24T08:04:14.973Z [run-main-6] INFO o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - Completed *?feature*[success] Total time: 5 s, completed 24 févr. 2021 à 08:04:15 If someone wants Java code to try, send me a private mail and I'll write it :) . Jean-Marc Vanel <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Is it possible to use UTF8 IRIs in Turtle?
As a practical complement to what Andy wrote,
UTF-8 is just like another character encoding.
If you use a well behaved text editor like gvim
<https://www.vim.org/download.php> , you can set file encoding to utf-8 by
typing
:set fileencoding=utf-8
and the Turtle or SPARQL you type will be OK.
As an exemple, on one Jena based site I run, searching "Corrençon"
generates under the hood an UTF-8 SPARQL query with text search :
http://semantic-forms.cc:1952/search?q=Corren%C3%A7on
As another exemple, this UTF-8 query on http://dbpedia.org/sparql/ runs
fine:
select distinct * where {
?S ?P <http://dbpedia.org/resource/Corren*ç*on-en-Vercors>
}
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le jeu. 31 déc. 2020 à 08:50, Laura Morales a écrit :
> Is there a way to write UTF8 IRIs with Turtle without all the %-encoded
> characters? I mean like this or or ex:"alice
> smith"? The only way that I know to write those characters is like this
> , ie. by writing the encoded URI myself. Is there any syntax
> that I can use to write UTF8 characters instead, and have those characters
> automatically be parsed as IRIs? Like when I type a string in my browser, I
> type UTF8 but it's automatically url-encoded to a URL?
>
Re: Update Jena from 3.16 to 3.17 : problem with Jackson, diverging version in another software
With a little help from stackoverflow , I fixed that. I had forgotten the
indirect dependency in Banana-RDF <https://github.com/banana-rdf/banana-rdf>,
on which I applied the same exclusion as Jena itself:
exclude("com.fasterxml.jackson.core", "jackson-databind")
cf
https://github.com/jmvanel/semantic_forms/blob/master/scala/project/Common.scala#L17
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le mar. 15 déc. 2020 à 09:36, Jean-Marc Vanel a
écrit :
> I feel that I'm cornered with this problem; so I threw a bottle in the sea:
>
> https://stackoverflow.com/questions/65302371/playframework-cannot-override-sbt-dependency
> but my forecast is that I will stay at Jena 3.16 until Play Framework
> updates its Jackson dependencies .
>
> Or someday I will quit Play Framework in favor of Akka, which is anyway
> the underlying server of Play Framework.
> I use very little of Play Framework; my XHTML templates are in plain Scala
> XML.
>
> Jean-Marc Vanel
> <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
> +33
> (0)6 89 16 29 52
>
>
> Le lun. 14 déc. 2020 à 19:23, Jean-Marc Vanel
> a écrit :
>
>> The exclude declarations looked in fact OK,
>> because the jar for jackson-databind is (like jackson-core) in folder
>> com.fasterxml.jackson.core , which is being excluded, e.g. in :
>>
>> val jenaTextDependency = "org.apache.jena" % "jena-text" % jenaVersion
>> excludeAll(
>> ExclusionRule(organization = "*com.fasterxml.jackson.core*" ) ,
>> ExclusionRule(organization = "com.fasterxml.jackson.datatype" ) ,
>> ExclusionRule(organization = "com.fasterxml.jackson.module" )
>>
>> After that I tried a more radical SBT exclusion rule, excluding jackson
>> transitively through all application modules:
>>
>> *excludeDependencies* ++= Seq(
>> ExclusionRule("com.fasterxml.jackson.core" ),
>> ExclusionRule(organization = "com.fasterxml.jackson.datatype" ) ,
>> ExclusionRule(organization = "com.fasterxml.jackson.module" )
>> )
>>
>> which excludes everything under "com.fasterxml.jackson.*" for all
>> dependencies :
>>
>> https://github.com/jmvanel/semantic_forms/blob/master/scala/project/Common.scala#L20
>>
>> NOTE: the exception is thrown by Akka : akka % akka-serialization-jackson
>>
>> [error] com.fasterxml.jackson.databind.JsonMappingException: Scala
>> module 2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0
>> [error] at
>> com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:61)
>> [error] at
>> com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:46)
>> [error] at
>> com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:17)
>> [error] at
>> com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:819)
>> [error] at
>> akka.serialization.jackson.JacksonObjectMapperProvider$.$anonfun$configureObjectMapperModules$4(JacksonObjectMapperProvider.scala:223)
>> [error] at
>> akka.serialization.jackson.JacksonObjectMapperProvider$.$anonfun$configureObjectMapperModules$4$adapted(JacksonObjectMapperProvider.scala:222)
>> [error] at
>> scala.collection.immutable.List.foreach(List.scala:431)
>> [error] at akka.
>> *serialization.jackson.JacksonObjectMapperProvider*
>> $.configureObjectMapperModules(JacksonObjectMapperProvider.scala:222)
>> ...
>> [error] at
>> akka.actor.ActorSystem$.apply(ActorSystem.scala:290)
>> [error] at play.core.server.*DevServerStart*
>> $.$anonfun$mainDev$1(DevServerStart.scala:248)
>> [error] at
>> play.utils.Threads$.withContextClassLoader(Threads.scala:22)
>> [error] at
>> play.core.server.DevServerStart$.mainDev(DevServerStart.scala:76)
>>
>>
>> I guess I will ask to Play community ...
>>
>> Jean-Marc Vanel
>> <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
>> +33
>> (0)6 89 16 29 52
>>
>>
>> Le lun. 14 déc. 2020 à 15:36, Andy Seaborne a écrit :
>>
>>>
>>>
>>> On 14/12/2020 13:12, Jean-Marc Vanel wrote:
>>> > A found a cleaner way with SBT build engine to exclude all Jackson
>>> stuff
>>> > coming from Jena
Re: Indexing datatypes
Andy can confirm, but AFAIK everything is indexed in Jena. And a number is stored as such . Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le mer. 16 déc. 2020 à 17:23, Laura Morales a écrit : > I'm able to configure Fuseki for fulltext search with Lucene, but I don't > see any other type of indexes in the documentation. If there is a property > that is (ab)used to link multiple datatypes, say for example > > Turtle > -- > :alice :age 21; :age "twenty-one" . > > How is this stored by Jena (everything as a string?)? Can I create an > index for a specific datatype, say integer or datetime, and how does this > work if the property is abused like in the example above? >
Re: Update Jena from 3.16 to 3.17 : problem with Jackson, diverging version in another software
I feel that I'm cornered with this problem; so I threw a bottle in the sea:
https://stackoverflow.com/questions/65302371/playframework-cannot-override-sbt-dependency
but my forecast is that I will stay at Jena 3.16 until Play Framework
updates its Jackson dependencies .
Or someday I will quit Play Framework in favor of Akka, which is anyway the
underlying server of Play Framework.
I use very little of Play Framework; my XHTML templates are in plain Scala
XML.
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le lun. 14 déc. 2020 à 19:23, Jean-Marc Vanel a
écrit :
> The exclude declarations looked in fact OK,
> because the jar for jackson-databind is (like jackson-core) in folder
> com.fasterxml.jackson.core , which is being excluded, e.g. in :
>
> val jenaTextDependency = "org.apache.jena" % "jena-text" % jenaVersion
> excludeAll(
> ExclusionRule(organization = "*com.fasterxml.jackson.core*" ) ,
> ExclusionRule(organization = "com.fasterxml.jackson.datatype" ) ,
> ExclusionRule(organization = "com.fasterxml.jackson.module" )
>
> After that I tried a more radical SBT exclusion rule, excluding jackson
> transitively through all application modules:
>
> *excludeDependencies* ++= Seq(
> ExclusionRule("com.fasterxml.jackson.core" ),
> ExclusionRule(organization = "com.fasterxml.jackson.datatype" ) ,
> ExclusionRule(organization = "com.fasterxml.jackson.module" )
> )
>
> which excludes everything under "com.fasterxml.jackson.*" for all
> dependencies :
>
> https://github.com/jmvanel/semantic_forms/blob/master/scala/project/Common.scala#L20
>
> NOTE: the exception is thrown by Akka : akka % akka-serialization-jackson
>
> [error] com.fasterxml.jackson.databind.JsonMappingException: Scala
> module 2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0
> [error] at
> com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:61)
> [error] at
> com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:46)
> [error] at
> com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:17)
> [error] at
> com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:819)
> [error] at
> akka.serialization.jackson.JacksonObjectMapperProvider$.$anonfun$configureObjectMapperModules$4(JacksonObjectMapperProvider.scala:223)
> [error] at
> akka.serialization.jackson.JacksonObjectMapperProvider$.$anonfun$configureObjectMapperModules$4$adapted(JacksonObjectMapperProvider.scala:222)
> [error] at
> scala.collection.immutable.List.foreach(List.scala:431)
> [error] at akka.
> *serialization.jackson.JacksonObjectMapperProvider*
> $.configureObjectMapperModules(JacksonObjectMapperProvider.scala:222)
> ...
> [error] at akka.actor.ActorSystem$.apply(ActorSystem.scala:290)
> [error] at play.core.server.*DevServerStart*
> $.$anonfun$mainDev$1(DevServerStart.scala:248)
> [error] at
> play.utils.Threads$.withContextClassLoader(Threads.scala:22)
> [error] at
> play.core.server.DevServerStart$.mainDev(DevServerStart.scala:76)
>
>
> I guess I will ask to Play community ...
>
> Jean-Marc Vanel
> <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
> +33
> (0)6 89 16 29 52
>
>
> Le lun. 14 déc. 2020 à 15:36, Andy Seaborne a écrit :
>
>>
>>
>> On 14/12/2020 13:12, Jean-Marc Vanel wrote:
>> > A found a cleaner way with SBT build engine to exclude all Jackson stuff
>> > coming from Jena :
>> >
>> https://github.com/jmvanel/semantic_forms/blob/2b5bfb31a47376f86cfbb7f2820be1dbeada1a8b/scala/project/Common.scala#L21
>> >
>> > But this does not fix the problem "JsonMappingException: Scala module
>> > 2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0 " .
>>
>> databind != datatype
>>
>> Jena includes:
>> com.fasterxml.jackson.core:jackson-core:jar:2.12.0:compile
>> com.fasterxml.jackson.core:jackson-databind:jar:2.12.0:compile
>>
>> >
>> > I failed to find other dependencies that would bring Jackson 2.11.3 .
>> > So either the SBT exclusion attempt does not work, or there is some
>> other
>> > dependency bringing Jackson 2.11.3 ( Akka ? ) .
>> >
>> > Jean-Marc Vanel
>> > <
>> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.
Re: Does Jena support RDF*?
The source code , with store and SPARQL to retrieve, and a few explanations, is here: https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/semlogs/TimeSeries.scala#L27 The trick used is to create a new URI for each modification of the main database ( find http://localhost:9000/ldp/1486136803523-89228787433384#1486136808956 in the source code ) , that is used both as a subject for metadata, and as a graph for the current modified triples. It deserves more explanations I think. Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le lun. 14 déc. 2020 à 13:57, Laura Morales a écrit : > > This is an alternative to RDF* , AFAIK . > > If someone is interested, I can document the structure of the secondary > TDB > > database. > > I don't want to abuse your time but yes, this would be helpful. Even just > a sketch of it, just to get the idea. >
Re: Update Jena from 3.16 to 3.17 : problem with Jackson, diverging version in another software
A found a cleaner way with SBT build engine to exclude all Jackson stuff
coming from Jena :
https://github.com/jmvanel/semantic_forms/blob/2b5bfb31a47376f86cfbb7f2820be1dbeada1a8b/scala/project/Common.scala#L21
But this does not fix the problem "JsonMappingException: Scala module
2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0 " .
I failed to find other dependencies that would bring Jackson 2.11.3 .
So either the SBT exclusion attempt does not work, or there is some other
dependency bringing Jackson 2.11.3 ( Akka ? ) .
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le sam. 12 déc. 2020 à 15:16, Andy Seaborne a écrit :
>
>
> On 11/12/2020 16:26, Jean-Marc Vanel wrote:
> > I use latest Play Framework 2.8.5 with Scala (my project is
> > https://github.com/jmvanel/semantic_forms). I try to upgrade my Jena
> > dependency from 3.16.0 to 3.17.0 .
> > I get this message when running:
> > [error] com.fasterxml.jackson.databind.JsonMappingException: *Scala
> module
> > 2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0.*
> > Indeed Jena requires jackson-databind 2.11.3 , while Play is at 2.10.4 .
> I
> > tried to add exclude("com.fasterxml.jackson.core", "jackson-databind" )
> in
> > build.sbt to all Jena dependencies, but no result :( .
>
> Try excluding all com.fasterxml.jackson from the Jena dependency
> declaration recursively.
>
> Jena would also get jackson from jsonld-java but excludes it.
>
> jsonld-java/0.13.2 currently at Jackson 2.11.x
>
> Jackson had a number of CVEs so taking control of the dependency in Jena
> meant Jena could release specifying fixed versions.
>
> Jena should work with 2.10.* For the jsonld-java usage, there wasn't a
> breaking change.
>
> 2.11.* has an architecture change than makes the 2.10.* CVE issues go
> away by design.
>
> 2.12.x isn't a breaking change either for the usage made of it
> (according to the test suite).
>
> Of course, jackson is used by many libraries so it could be picked up by
> several paths.
>
> Andy
>
> > I added on Jena all the exclusions for each jackson dependency , even
> > transitive , but problem is still here :
> >
> > exclude("com.fasterxml.jackson.core", "jackson-core" )
> > exclude("com.fasterxml.jackson.core", "jackson-databind" )
> > exclude("com.fasterxml.jackson.core", "jackson-annotations")
> > exclude("com.fasterxml.jackson.datatype", "jackson-datatype" )
> > exclude("com.fasterxml.jackson.datatype", "jackson-datatypejsr310")
> > exclude("com.fasterxml.jackson.datatype", "jackson-datatype-jdk8")
> > exclude("com.fasterxml.jackson.module", "jackson-module-parameter-names")
> > exclude("com.fasterxml.jackson.module", "jackson-module-scala")
> > exclude("com.fasterxml.jackson.module", "jackson-module-paranamer")
> >
> > I have verified which artifacts are resolved by the build tool, by
> > dependencyTree .
> > The Jena requirements of the latest jenaVersion = "3.17.0" is
> > :jackson-databind:2.11.3, which brings jackson-annotations and
> jackson-core
> > .
> >
> > Running show playDependencyClasspath , I get :
> > [info] * Attributed(/home/jmv/.cache/coursier/v1/https/
> >
> repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.11.3/jackson-core-2.11.3.jar
> > )
> > [info] * Attributed(/home/jmv/.cache/coursier/v1/https/
> >
> repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.11.3/jackson-annotations-2.11.3.jar
> > )
> > [info] * Attributed(/home/jmv/.cache/coursier/v1/https/
> >
> repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.10.4/jackson-datatype-jdk8-2.10.4.jar
> > )
> > [info] * Attributed(/home/jmv/.cache/coursier/v1/https/
> >
> repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.10.4/jackson-datatype-jsr310-2.10.4.jar
> > )
> > [info] * Attributed(/home/jmv/.cache/coursier/v1/https/
> >
> repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.11.3/jackson-databind-2.11.3.jar
> > )
> >
> > I hoped to drop the Jackson Jena dependencies, and depend only on
> > Jackson 2.10.4 like Play, but that 's not the case...
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33 (0)6 89 16 29 52
> > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
> > Chroniques jardin
> > <
> http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle
> >
> >
>
Re: Does Jena support RDF*?
I use on the SF (1) site (powered by (2) ) a secondary TDB database, not exposed via SPARQL, which contains annotations on the primary data. This enables to have roughly the equivalent of git; its records a history of user edits. Via web pages, only a bit of this data is exposed, allowing to show this historic table view: http://semantic-forms.cc:1952/history?limit=50 This is an alternative to RDF* , AFAIK . If someone is interested, I can document the structure of the secondary TDB database. (1) SF site http://semantic-forms.cc:1952/ (2) SF software http://semantic-forms.cc:9112/ldp/semantic_forms Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le dim. 13 déc. 2020 à 20:35, Laura Morales a écrit : > > What's your interest in RDF*? > > There seems to have been this endless debate about triplestores vs > property graphs since as far as I can remember. This new standard > apparently promises to be the best of both world by supporting RDF plus > what they call "richer types" (aka nodes, vertexes). "Richer" compared to > the extremely atomic level of triples. So my interest is mostly to try it > and see how it compares. Also from a storage point of view since everything > that I've read claims that property graphs are faster to traverse because > their storage is not "index-based" like triples. I've personally tried to > use a couple of property graphs databases but I keep going back to > triplestores for the only reason that they use more standardized > technology. Every property graph instead seems to have its own way of doing > things; I couldn't even find a standardized format for exporting/importing > graphs or a standardized query language (although there are some efforts > toward one called GQL). So if RDF* can combine the best parts of both > worlds, I want to try it :) > Please note that I'm not personally interested in the semantic web or the > RDF artificial intelligence koolaid. I'm interested in the graph model with > a great appreciation for free standards and simplicity. If RDF* can make > the design of graphs simpler (ie. richer structures, fewer hacks and > workarounds) then it's definitely something that I will use. > Another issue for me with property graphs, but I would like to hear your > feedback on this, is that properties are indexed globally and it's my > understanding that they only accept one data type (eg. Integer). So I'm not > sure how indexing work over there from a storage point of view. I think > they would require me to define 2 properties instead of one or some kind of > namespace, let's say "ns1_age" and "ns2_age" where one property takes > Integer and the other one String for example. Which, at the end of the day, > is the same thing as using RDF prefixes. >
Update Jena from 3.16 to 3.17 : problem with Jackson, diverging version in another software
I use latest Play Framework 2.8.5 with Scala (my project is
https://github.com/jmvanel/semantic_forms). I try to upgrade my Jena
dependency from 3.16.0 to 3.17.0 .
I get this message when running:
[error] com.fasterxml.jackson.databind.JsonMappingException: *Scala module
2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0.*
Indeed Jena requires jackson-databind 2.11.3 , while Play is at 2.10.4 . I
tried to add exclude("com.fasterxml.jackson.core", "jackson-databind" ) in
build.sbt to all Jena dependencies, but no result :( .
I added on Jena all the exclusions for each jackson dependency , even
transitive , but problem is still here :
exclude("com.fasterxml.jackson.core", "jackson-core" )
exclude("com.fasterxml.jackson.core", "jackson-databind" )
exclude("com.fasterxml.jackson.core", "jackson-annotations")
exclude("com.fasterxml.jackson.datatype", "jackson-datatype" )
exclude("com.fasterxml.jackson.datatype", "jackson-datatypejsr310")
exclude("com.fasterxml.jackson.datatype", "jackson-datatype-jdk8")
exclude("com.fasterxml.jackson.module", "jackson-module-parameter-names")
exclude("com.fasterxml.jackson.module", "jackson-module-scala")
exclude("com.fasterxml.jackson.module", "jackson-module-paranamer")
I have verified which artifacts are resolved by the build tool, by
dependencyTree .
The Jena requirements of the latest jenaVersion = "3.17.0" is
:jackson-databind:2.11.3, which brings jackson-annotations and jackson-core
.
Running show playDependencyClasspath , I get :
[info] * Attributed(/home/jmv/.cache/coursier/v1/https/
repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.11.3/jackson-core-2.11.3.jar
)
[info] * Attributed(/home/jmv/.cache/coursier/v1/https/
repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.11.3/jackson-annotations-2.11.3.jar
)
[info] * Attributed(/home/jmv/.cache/coursier/v1/https/
repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.10.4/jackson-datatype-jdk8-2.10.4.jar
)
[info] * Attributed(/home/jmv/.cache/coursier/v1/https/
repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.10.4/jackson-datatype-jsr310-2.10.4.jar
)
[info] * Attributed(/home/jmv/.cache/coursier/v1/https/
repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.11.3/jackson-databind-2.11.3.jar
)
I hoped to drop the Jackson Jena dependencies, and depend only on
Jackson 2.10.4 like Play, but that 's not the case...
Jean-Marc Vanel
<http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: TDB(1) : get prefixes map ?
Thanks for the clarifications. I approve the global per dataset prefixes management. By the way, I found this nice list of 2677 prefix Turtle declarations , ever up to date : $JENA/bin/riot --syntax=turtle --nocheck *http://prefix.cc/popular/all.file.sparql <http://prefix.cc/popular/all.file.sparql>* (one has a wrong URI) Not knowing there was an adapter developed in both directions PrefixMap <-> PrefixMapping , I wrote a class PrefixMapImpl (partial implementation of PrefixMap in Scala) with a constructor taking a plain Map, enough for being used by a WriterGraphRIOT : https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/GraphWriterPrefixMap.scala#L42 Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le lun. 7 déc. 2020 à 17:22, Andy Seaborne a écrit : > Hi there - thanks for investigating. > ...
Re: A seemingly simple question for beginners
As Lorenz said your pasting of Jena results is not very readable.
But I think I understand your problem, indeed a beginner's one .
This query (1) brings no results:
PREFIX : <http://dsc.nlp-bigdatalab.org:8086/>
SELECT * WHERE {
:湿疹样 rdf:type ?o.
?s ?p ?o.
}
If we take each line of the above query, we have 2 queries that bring some
results:
PREFIX : <http://dsc.nlp-bigdatalab.org:8086/>
SELECT * WHERE {
:湿疹样 rdf:type ?o.
} # query 2
==
PREFIX : <http://dsc.nlp-bigdatalab.org:8086/>
SELECT * WHERE {
?s ?p ?o.
} LIMIT 5} # query 3
Then, why does query 1 bring no result ?
A query as like an equation with variables.
Query 1 is overconstrained, your dataset has no triples satisfying the 2
lines.
Obviously ?o in
:湿疹样 rdf:type ?o.
is the resource of a class, so if you load your vocabulary (ontology) into
the database or query,
it will succeed.
So, you need in your database one triple like:
:西医症状 rdf:type <http://www.w3.org/2002/07/owl#Class> .
and then ?s in query 1 will match :西医症状 , ?p will match rdf:type , and ?o
will match <http://www.w3.org/2002/07/owl#Class> .
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le lun. 7 déc. 2020 à 02:20, 周蓉 a écrit :
> Hello, I have a problem when I query in Fuseki in this way:
>
>
> PREFIX : <http://dsc.nlp-bigdatalab.org:8086/>
>
> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
>
> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
>
> SELECT * WHERE {
>
> ?s ?p ?o.
>
> }
>
>
>
> It can return the results like this:
>
> | 82 |
> :皮肤血管蜘蛛痣
> |
> rdf:type
> |
> :西医症状
> |
> | 83 |
> :血红蛋白E病
> |
> rdfs:label
> |
> "血红蛋白E病"@ZH
> |
> | 84 |
> :复方枇杷氯化铵糖浆
> |
> rdf:type
> |
> :药品
> |
> |
> | | |
> |
> | 86 |
> :快节奏综合症
> |
> rdf:type
> |
> :疾病
> |
> | 87 |
> :作腐提脓膏
> |
> rdfs:label
> |
> "作腐提脓膏"@ZH
> |
> but when I just try to add a constraint to expect the result like line85
> above，：
> | 85 |
> :湿疹样
> |
> rdf:type
> |
> :西医症状
> |
> I modify the query ：
>
> PREFIX : <http://dsc.nlp-bigdatalab.org:8086/>
> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
> SELECT * WHERE {
> :湿疹样 rdf:type ?o.
> ?s ?p ?o.
> }
>
> and then it does not work!
> the result shows "No data available in the table"
> I do write query statements based on existing data
> so it has confused me for several days
> I need your help，thanks!
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Re: TDB(1) : get prefixes map ?
I made careful tests ( https://github.com/jmvanel/jena-examples) , and with jenaVersion = "3.17.0" and TDB 1, I observed the following, which is globally *satisfying*: - each named graph keeps its separate prefix map, coming from the TTL loaded - the default graph also keeps its separate prefix map - Model.getNsPrefixMap() gets the prefix map - the union graph does not provide a prefix map - WriterDatasetRIOT.write() outputs prefixes from given prefix map - PrefixMappingUtils.calcInUsePrefixMapping() analyses the triples and keeps the prefixes actually used among the given PrefixMapping - there are 3 data structures: PrefixMapping (subinterface of Model), PrefixMap (used for writing), and Map ; which is not very convenient - for testing calcInUsePrefixMapping() I took the PrefixMapping of the Default Model; if I would create one from scratch I could use org.apache.jena.shared.impl.PrefixMappingImpl I didn't understand from Andy what is coming after Jena 3.17 about prefixes . Probably the doc. should mention WriterDatasetRIOT in https://jena.apache.org/documentation/io/rdf-output.html#api . Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le mer. 2 déc. 2020 à 22:31, Andy Seaborne a écrit : > > > On 02/12/2020 17:33, Jean-Marc Vanel wrote: > > I thought I could retrieve the prefix map from TDB 1 with > > dataset.getDefaultModel().getNsPrefixMap() > > > PrefixMapping prefixMapping = dataset.getDefaultModel(); > > PrefixMapping is a subinterface of Model. > > See > > https://jena.apache.org/documentation/javadoc/jena/org/apache/jena/rdf/model/Model.html > > > > > but that was a mistake; really I don't know how to get the prefix map. > > > > I started a project called jena-examples, that may grow with help from > the > > community. > > Here is the Java code that I tried to retrieve the prefix map and use it > to > > print a named graph: > > > https://github.com/jmvanel/jena-examples/blob/main/src/main/java/WriteWithPrefixMap1.java#L22 > > > > ( I use my local test database, 13M triples; I can add to code to > populate > > an empty TDB ) > > ( I just provided SBT files, tell me if you want a pom.xml ) > > > > Jean-Marc Vanel > > < > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > +33 > > (0)6 89 16 29 52 > > > > > > Le mar. 1 déc. 2020 à 22:36, Andy Seaborne a écrit : > > > >> > >> > >> On 01/12/2020 21:15, Jean-Marc Vanel wrote: > >> > This is fine , I can get all the TDB prefix mappings with: > >> > dataset.getDefaultModel().getNsPrefixMap() , > >> > but I don't see how I can add them to a graph resulting from a > SPARQL > >> query, > >> > >> You mean CONSTRUCT or DESCRIBE? > >> > >> Add after return or use the execConstruct(model) form to pass in a model > >> with prefixes already set. > >> > >> > in order to write it. I see nothing for this in RDFDataMgr . > >> > >> RDFDataMgr is a convenience library. > >> > >> The lower level interface WriterGraphRIOT takes a PrefixMap. > >> > >> You can get a writer(factory) from the RDFWriterRegistry. > >> > >> > And then, if I can add them while printing a graph, will the big > list > >> > of prefixes be added, > >> > or only the prefixes actually used will be printed ? > >> > >> Declared unless you use PrefixMappingUtils.calcInUsePrefixMapping which > >> is not so cheap as to automatically run it each time. > >> > >> Andy > >> > >> > > >> > Jean-Marc Vanel > >> > > >> < > >> > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > >>> > >> > +33 > >> > (0)6 89 16 29 52 > >> > > >> > > >> > Le mar. 1 déc. 2020 à 21:34, Andy Seaborne a > écrit : > >> > > >> >> > >> >> > >> >> On 01/12/2020 19:02, Jean-Marc Vanel wrote: > >> >>> Empty maps are obtained by calling getPrefixMapping() , > >> >>> either with a graph from a SPARQL query, or from the Union Graph. > >> >> > >> >> Perfect timing! > >> >> > >> >> JENA-2006 (in-progress as of 2020-12-01 - PR went in today. > >> >> adds Dataset.getPrefixMappin
Re: 3.17
The announcement mail is here : https://lists.apache.org/thread.html/r911197ad367bde08373614fb6a74a72cef29d886289fc2ff9f6b0251%40%3Cusers.jena.apache.org%3E Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le mer. 2 déc. 2020 à 20:21, Erich Bremer a écrit : > I don't see it here: > https://mvnrepository.com/artifact/org.apache.jena/apache-jena-libs > > Let me look around more... - E > > > On Wed, Dec 2, 2020 at 1:43 PM Jean-Marc Vanel > wrote: > > > 3.17.0 has been announced on this list; > > it works for me with SBT; should work also with Maven :) > > > > Jean-Marc Vanel > > < > > > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > > +33 > > (0)6 89 16 29 52 > > > > > > Le mer. 2 déc. 2020 à 19:38, Erich Bremer a écrit : > > > > > How long before 3.17 is available via maven artifacts? I saw the 3.17 > > > release on github. - Erich > > > > > >
Re: 3.17
3.17.0 has been announced on this list; it works for me with SBT; should work also with Maven :) Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le mer. 2 déc. 2020 à 19:38, Erich Bremer a écrit : > How long before 3.17 is available via maven artifacts? I saw the 3.17 > release on github. - Erich >
Re: TDB(1) : get prefixes map ?
I thought I could retrieve the prefix map from TDB 1 with dataset.getDefaultModel().getNsPrefixMap() but that was a mistake; really I don't know how to get the prefix map. I started a project called jena-examples, that may grow with help from the community. Here is the Java code that I tried to retrieve the prefix map and use it to print a named graph: https://github.com/jmvanel/jena-examples/blob/main/src/main/java/WriteWithPrefixMap1.java#L22 ( I use my local test database, 13M triples; I can add to code to populate an empty TDB ) ( I just provided SBT files, tell me if you want a pom.xml ) Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le mar. 1 déc. 2020 à 22:36, Andy Seaborne a écrit : > > > On 01/12/2020 21:15, Jean-Marc Vanel wrote: > > This is fine , I can get all the TDB prefix mappings with: > > dataset.getDefaultModel().getNsPrefixMap() , > > but I don't see how I can add them to a graph resulting from a SPARQL > query, > > You mean CONSTRUCT or DESCRIBE? > > Add after return or use the execConstruct(model) form to pass in a model > with prefixes already set. > > > in order to write it. I see nothing for this in RDFDataMgr . > > RDFDataMgr is a convenience library. > > The lower level interface WriterGraphRIOT takes a PrefixMap. > > You can get a writer(factory) from the RDFWriterRegistry. > > > And then, if I can add them while printing a graph, will the big list > > of prefixes be added, > > or only the prefixes actually used will be printed ? > > Declared unless you use PrefixMappingUtils.calcInUsePrefixMapping which > is not so cheap as to automatically run it each time. > > Andy > > > > > Jean-Marc Vanel > > > < > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > +33 > > (0)6 89 16 29 52 > > > > > > Le mar. 1 déc. 2020 à 21:34, Andy Seaborne a écrit : > > > >> > >> > >> On 01/12/2020 19:02, Jean-Marc Vanel wrote: > >>> Empty maps are obtained by calling getPrefixMapping() , > >>> either with a graph from a SPARQL query, or from the Union Graph. > >> > >> Perfect timing! > >> > >> JENA-2006 (in-progress as of 2020-12-01 - PR went in today. > >> adds Dataset.getPrefixMapping() and DatsetGraph.prefixes() > >> > >> For a TDB database, all graphs will have the same prefix map, including > >> the union graph. > >> > >> Currently - 3.17.0 - the dataset prefixes (i.e. as used for output) are > >> held with the default graph. While the code supports additonal prefix > >> sets, they are accssible without going into the implementation. > >> > >> JENA-2006 changes all that. > >> > >>> From a SPARQL query, only possible prefix mappings are from the > query, > >> not > >>> the database. > >> > >> They are on the default graph, if there are any from TriG. > >> > >>> Same thing with > >>> PrefixMappingUtils.calcInUsePrefixMapping(graph) > >>> There is a class called DatasetPrefixesTDB , but it seems a low level > >>> implementation class. > >>> Some code comments imply that a prefixes map is not maintained > >>> permanently ... > >>> > >>> Not yet tested with 3.17.0 . > >> > >> No change. > >> > >> Andy > >> > >>> > >>> Jean-Marc Vanel > >>> < > >> > > http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > >>> > >>> +33 (0)6 89 16 29 52 > >>> Twitter: @jmvanel , @jmvanel_fr ; chat: irc:// > irc.freenode.net#eulergui > >>>Chroniques jardin > >>> < > >> > > http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle > >>> > >>> > >> >
Re: Jena geosparql , simple export use case ; practical doc. needed
Not sure if it is relevant, but almost all of my data is in named graphs,
including the ontologies.
The above queries were made with union graph set through ARQ API .
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le lun. 2 nov. 2020 à 08:42, Jean-Marc Vanel a
écrit :
> Responses interleaved ...
>
>
> Le dim. 1 nov. 2020 à 20:49, Greg a écrit :
>
>> Hi Jean-Marc,
>>
>> You refer to a remote endpoint for your example data and don't indicate
>> any data loading into the TDB dataset.
>
> They were loaded since a long time.
> As mentioned, I just checked by this query that they are here on my local
> server with this query:
> PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
> SELECT * WHERE {
> ?S geo:lat ?LAT .
> ?S geo:long ?LON .
> } LIMIT 100
>
> ... The GeoSPARQL functionality needs to be setup in the application or
>> Fuseki
>
> server where the dataset is located."
>
>
> In fact I developed over the years an equivalent of Fuseki with Scala, the
> Jena API and PlayFramework, although that was not planned at the beginning.
>
> The line
>> `TDBFactory.createDataset` only creates an empty in-memory dataset.
>> Either of these may be why you are receiving no results.
>>
>> I'm not sure how you've got the impression that no configuration is
>> required when there is a section titled `SPARQL Query Configuration`
>> which states "Using the library for SPARQL querying requires one line of
>> code." followed by a series of configuration examples.
>>
>
> My setup now as :
>
> org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupMemoryIndex()
>
> org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupSpatialIndex(dataset)
>
> The log now says:
>
> 07:34:09.817 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
> o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Started
> 07:34:40.170 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
> o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Completed
> 07:34:40.170 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
> o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - Started
> 07:34:40.219 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
> o.a.j.geosparql.spatial.SpatialIndex - Geo predicate statements found.
> 07:34:40.238 [play-dev-mode-akka.actor.default-dispatcher-44] ERROR jena -
> ! createDatabase: configureLuceneIndex: Exception:
> http://www.w3.org/2003/01/geo/wgs84_pos#long
> 07:34:40.238 [play-dev-mode-akka.actor.default-dispatcher-44] ERROR jena -
> Exception class org.apache.jena.shared.PropertyNotFoundException
> 07:34:40.238 [play-dev-mode-akka.actor.default-dispatcher-44] ERROR jena -
> Exception org.apache.jena.shared.PropertyNotFoundException:
> http://www.w3.org/2003/01/geo/wgs84_pos#long
>
> NOTE: I have disconnected the Lucene indexing as I fear a detrimental
> interference .
>
> The Spatial Index section in the documentation may assist with the error
>> you are now encountering. I've added an additional note to the
>> `jena-spatial` section in documentation to make the solution clearer in
>> the future. It should only require invoking
>> `GeoSPARQLConfig.setupSpatialIndex(dataset);` or similar on the target
>> dataset.||
>>
>> All the best,
>>
>> Greg
>>
>> On 29/10/2020 09:23, Jean-Marc Vanel wrote:
>> > Hi Greg
>> >
>> > The data are available at this SPARQL endpoint :
>> > http://semantic-forms.cc:1952/sparql2
>> > the queries are given in this thread.
>> >
>> > BUT, I reread the documentation, and added
>> >
>> org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupMemoryIndex()
>> > before calling TDBFactory.createDataset()
>> >
>> > And now , with the same very simple query with spatial:withinBox() , it
>> > says :
>> > org.apache.jena.sparql.expr.ExprEvalException: Dataset Context does not
>> > contain SpatialIndex.
>> >
>> > From the documentation, it is not clear what is the minimal, or
>> optimal,
>> > configuration. A casual reading even gives the impression that no
>> > configuration is necessary.
>> >
>> > Jean-Marc Vanel
>> > <
>> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
>> >
>> > +33
>> > (0)6 89 16 29 52
>> >
>> >
>> > Le mer. 28 oct. 2020 à 23:29, Greg a écrit :
>> >
>> >> Hi Jean-Marc,
>> >>
>> >> If yo
Re: Jena geosparql , simple export use case ; practical doc. needed
Responses interleaved ...
Le dim. 1 nov. 2020 à 20:49, Greg a écrit :
> Hi Jean-Marc,
>
> You refer to a remote endpoint for your example data and don't indicate
> any data loading into the TDB dataset.
They were loaded since a long time.
As mentioned, I just checked by this query that they are here on my local
server with this query:
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
SELECT * WHERE {
?S geo:lat ?LAT .
?S geo:long ?LON .
} LIMIT 100
... The GeoSPARQL functionality needs to be setup in the application or
> Fuseki
server where the dataset is located."
In fact I developed over the years an equivalent of Fuseki with Scala, the
Jena API and PlayFramework, although that was not planned at the beginning.
The line
> `TDBFactory.createDataset` only creates an empty in-memory dataset.
> Either of these may be why you are receiving no results.
>
> I'm not sure how you've got the impression that no configuration is
> required when there is a section titled `SPARQL Query Configuration`
> which states "Using the library for SPARQL querying requires one line of
> code." followed by a series of configuration examples.
>
My setup now as :
org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupMemoryIndex()
org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupSpatialIndex(dataset)
The log now says:
07:34:09.817 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Started
07:34:40.170 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
o.a.j.g.c.GeoSPARQLOperations - Find Mode SRS - Completed
07:34:40.170 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
o.a.j.geosparql.spatial.SpatialIndex - Building Spatial Index - Started
07:34:40.219 [play-dev-mode-akka.actor.default-dispatcher-44] INFO
o.a.j.geosparql.spatial.SpatialIndex - Geo predicate statements found.
07:34:40.238 [play-dev-mode-akka.actor.default-dispatcher-44] ERROR jena -
! createDatabase: configureLuceneIndex: Exception:
http://www.w3.org/2003/01/geo/wgs84_pos#long
07:34:40.238 [play-dev-mode-akka.actor.default-dispatcher-44] ERROR jena -
Exception class org.apache.jena.shared.PropertyNotFoundException
07:34:40.238 [play-dev-mode-akka.actor.default-dispatcher-44] ERROR jena -
Exception org.apache.jena.shared.PropertyNotFoundException:
http://www.w3.org/2003/01/geo/wgs84_pos#long
NOTE: I have disconnected the Lucene indexing as I fear a detrimental
interference .
The Spatial Index section in the documentation may assist with the error
> you are now encountering. I've added an additional note to the
> `jena-spatial` section in documentation to make the solution clearer in
> the future. It should only require invoking
> `GeoSPARQLConfig.setupSpatialIndex(dataset);` or similar on the target
> dataset.||
>
> All the best,
>
> Greg
>
> On 29/10/2020 09:23, Jean-Marc Vanel wrote:
> > Hi Greg
> >
> > The data are available at this SPARQL endpoint :
> > http://semantic-forms.cc:1952/sparql2
> > the queries are given in this thread.
> >
> > BUT, I reread the documentation, and added
> >
> org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupMemoryIndex()
> > before calling TDBFactory.createDataset()
> >
> > And now , with the same very simple query with spatial:withinBox() , it
> > says :
> > org.apache.jena.sparql.expr.ExprEvalException: Dataset Context does not
> > contain SpatialIndex.
> >
> > From the documentation, it is not clear what is the minimal, or optimal,
> > configuration. A casual reading even gives the impression that no
> > configuration is necessary.
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33
> > (0)6 89 16 29 52
> >
> >
> > Le mer. 28 oct. 2020 à 23:29, Greg a écrit :
> >
> >> Hi Jean-Marc,
> >>
> >> If you have an sample of data then I can take a look into this further.
> >>
> >> Thanks,
> >>
> >> Greg
> >>
> >> On 26/10/2020 16:56, Jean-Marc Vanel wrote:
> >>> Indeed this helps, but it's still not clear weather and how I have to
> >>> prepare the database for GeoSparlql queries.
> >>>
> >>> I changed my query according to Greg's advice:
> >>> PREFIX spatial: <http://jena.apache.org/spatial#>
> >>> SELECT * WHERE {
> >>> ?feature spatial:withinBox( 43.0 0.0 48.0 10.0 100 )
> >>> } LIMIT 100
> >>> but no results.
> >>> Of course I checked that I have many triples with geo: coordinates
> within
> >>> that rectangle :
> >>> PREFIX geo: &l
Re: Jena geosparql , simple export use case ; practical doc. needed
Hi Greg
The data are available at this SPARQL endpoint :
http://semantic-forms.cc:1952/sparql2
the queries are given in this thread.
BUT, I reread the documentation, and added
org.apache.jena.geosparql.configuration.GeoSPARQLConfig.setupMemoryIndex()
before calling TDBFactory.createDataset()
And now , with the same very simple query with spatial:withinBox() , it
says :
org.apache.jena.sparql.expr.ExprEvalException: Dataset Context does not
contain SpatialIndex.
>From the documentation, it is not clear what is the minimal, or optimal,
configuration. A casual reading even gives the impression that no
configuration is necessary.
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le mer. 28 oct. 2020 à 23:29, Greg a écrit :
> Hi Jean-Marc,
>
> If you have an sample of data then I can take a look into this further.
>
> Thanks,
>
> Greg
>
> On 26/10/2020 16:56, Jean-Marc Vanel wrote:
> > Indeed this helps, but it's still not clear weather and how I have to
> > prepare the database for GeoSparlql queries.
> >
> > I changed my query according to Greg's advice:
> > PREFIX spatial: <http://jena.apache.org/spatial#>
> > SELECT * WHERE {
> >?feature spatial:withinBox( 43.0 0.0 48.0 10.0 100 )
> > } LIMIT 100
> > but no results.
> > Of course I checked that I have many triples with geo: coordinates within
> > that rectangle :
> > PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
> > SELECT * WHERE {
> > ?S geo:lat ?LAT .
> > ?S geo:long ?LON .
> > } LIMIT 100
> >
> > Regarding simple export from triples with geo: coordinates to GML or
> > GeoJson, I'll write some code that I'll share.
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33
> > (0)6 89 16 29 52
> >
> >
> > Le lun. 26 oct. 2020 à 16:38, Greg Albiston a
> écrit :
> >
> >> Hi Jena-Marc,
> >>
> >> The namespace <http://geovocab.org/spatial#> is for NeoGeo Spatial
> >> ontology, which has similarities to GeoSPARQL specification but is not
> >> fully compliant.
> >>
> >> The Apache Jena spatial property functions use
> >> <http://jena.apache.org/spatial#>. These are additional functions that
> >> extend the GeoSPARQL standard.
> >>
> >> So you are likely getting no result as the "spatial:withinBox" function
> >> in the query is undefined.
> >>
> >> Regarding your other question about GML and GeoJSON documents. The
> >> GeoSPARQL standard doesn't cover serialisation from or to geospatial
> >> document formats into the GeoSPARQL "Feature/Geometry/GeometryLiteral"
> >> structure. This is likely because it is expected that most use cases
> >> would stay in RDF format once setup. From my experience the GML
> >> standards are also unbelievably convoluted to fully support.
> >>
> >> It may be straight forward to convert a list of points to a minimum GML
> >> document that QGIS accepts.
> >>
> >> The "convert" functions are to harmonise a dataset to a single SRS and
> >> datatype to reduce overhead and confusion from switching between
> >> different formats when querying the data. Or to switch a dataset from
> >> one SRS or datatype to another, i.e. consolidating diverse data sources.
> >> They aren't for generating non-RDF spatial documents. These are also for
> >> GeoSPARQL structure datasets so a dataset using the LatLon geo
> >> predicates would need to be converted (for which the
> >> "convertGeoPredicates" might be useful).
> >>
> >> Hope this helps,
> >>
> >> Greg
> >>
> >>
> >> On 26/10/2020 14:17, Jean-Marc Vanel wrote:
> >>> I missed ApacheCon, and I need some concrete hints sooner than next
> year.
> >>>
> >>> Currently my database gives an empty answer to :
> >>> PREFIX spatial: <http://geovocab.org/spatial#>
> >>> # ?feature spatial:withinBox(?latMin ?lonMin ?latMax ?lonMax [ ?limit])
> >>> SELECT * WHERE {
> >>> ?feature spatial:withinBox( 43.0 0.0 46.0 10.0 100 )
> >>> } LIMIT 100
> >>>
> >>> Should I apply once for all GeoSPARQLOperations.convert() ?
> >>> static Dataset
> >>> <
> >>
> https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html?is-external=true
> >>>
Re: Jena geosparql , simple export use case ; practical doc. needed
Indeed this helps, but it's still not clear weather and how I have to
prepare the database for GeoSparlql queries.
I changed my query according to Greg's advice:
PREFIX spatial: <http://jena.apache.org/spatial#>
SELECT * WHERE {
?feature spatial:withinBox( 43.0 0.0 48.0 10.0 100 )
} LIMIT 100
but no results.
Of course I checked that I have many triples with geo: coordinates within
that rectangle :
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
SELECT * WHERE {
?S geo:lat ?LAT .
?S geo:long ?LON .
} LIMIT 100
Regarding simple export from triples with geo: coordinates to GML or
GeoJson, I'll write some code that I'll share.
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le lun. 26 oct. 2020 à 16:38, Greg Albiston a écrit :
> Hi Jena-Marc,
>
> The namespace <http://geovocab.org/spatial#> is for NeoGeo Spatial
> ontology, which has similarities to GeoSPARQL specification but is not
> fully compliant.
>
> The Apache Jena spatial property functions use
> <http://jena.apache.org/spatial#>. These are additional functions that
> extend the GeoSPARQL standard.
>
> So you are likely getting no result as the "spatial:withinBox" function
> in the query is undefined.
>
> Regarding your other question about GML and GeoJSON documents. The
> GeoSPARQL standard doesn't cover serialisation from or to geospatial
> document formats into the GeoSPARQL "Feature/Geometry/GeometryLiteral"
> structure. This is likely because it is expected that most use cases
> would stay in RDF format once setup. From my experience the GML
> standards are also unbelievably convoluted to fully support.
>
> It may be straight forward to convert a list of points to a minimum GML
> document that QGIS accepts.
>
> The "convert" functions are to harmonise a dataset to a single SRS and
> datatype to reduce overhead and confusion from switching between
> different formats when querying the data. Or to switch a dataset from
> one SRS or datatype to another, i.e. consolidating diverse data sources.
> They aren't for generating non-RDF spatial documents. These are also for
> GeoSPARQL structure datasets so a dataset using the LatLon geo
> predicates would need to be converted (for which the
> "convertGeoPredicates" might be useful).
>
> Hope this helps,
>
> Greg
>
>
> On 26/10/2020 14:17, Jean-Marc Vanel wrote:
> > I missed ApacheCon, and I need some concrete hints sooner than next year.
> >
> > Currently my database gives an empty answer to :
> > PREFIX spatial: <http://geovocab.org/spatial#>
> > # ?feature spatial:withinBox(?latMin ?lonMin ?latMax ?lonMax [ ?limit])
> > SELECT * WHERE {
> >?feature spatial:withinBox( 43.0 0.0 46.0 10.0 100 )
> > } LIMIT 100
> >
> > Should I apply once for all GeoSPARQLOperations.convert() ?
> > static Dataset
> > <
> https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html?is-external=true
> >
> > convert
> > <
> https://jena.apache.org/documentation/javadoc/geosparql/org/apache/jena/geosparql/configuration/GeoSPARQLOperations.html#convert-org.apache.jena.query.Dataset-
> >
> > (Dataset
> > <
> https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html?is-external=true
> >
> > dataset)
> > Convert the input dataset to the most frequent coordinate reference
> system
> > and default datatype.
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33
> > (0)6 89 16 29 52
> >
> >
> > Le lun. 26 oct. 2020 à 11:02, Marco Neumann a
> > écrit :
> >
> >> good questions Jean-Marc, I will organize a session early in the new
> year
> >> to address some of them. In your case geo:lat and geo:long are not part
> of
> >> OGC GeoSPARQL but require a transformation. We have a tool in place for
> >> that in Jena. And yes displaying large datasets need dedicated
> strategies
> >> for efficient processing. The points raised by you would make a good
> >> problem statement for our session next year. Not sure if you have
> attended
> >> the ApachCon 2020 GeoSPARQL session last month but it's where I
> >> mentioned further collaboration with third party tool developers as a
> >> possibility. stay tuned.
> >>
> >> Marco
> >>
> >>
> >> On Mon, Oct 26, 2020 at 9:14 AM Jean-Marc Vanel <
> jeanmarc.va...@gmail.com>
> >> wrote:
> >>
> >>>
Re: Jena geosparql , simple export use case ; practical doc. needed
I missed ApacheCon, and I need some concrete hints sooner than next year.
Currently my database gives an empty answer to :
PREFIX spatial: <http://geovocab.org/spatial#>
# ?feature spatial:withinBox(?latMin ?lonMin ?latMax ?lonMax [ ?limit])
SELECT * WHERE {
?feature spatial:withinBox( 43.0 0.0 46.0 10.0 100 )
} LIMIT 100
Should I apply once for all GeoSPARQLOperations.convert() ?
static Dataset
<https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html?is-external=true>
convert
<https://jena.apache.org/documentation/javadoc/geosparql/org/apache/jena/geosparql/configuration/GeoSPARQLOperations.html#convert-org.apache.jena.query.Dataset->
(Dataset
<https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html?is-external=true>
dataset)
Convert the input dataset to the most frequent coordinate reference system
and default datatype.
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le lun. 26 oct. 2020 à 11:02, Marco Neumann a
écrit :
> good questions Jean-Marc, I will organize a session early in the new year
> to address some of them. In your case geo:lat and geo:long are not part of
> OGC GeoSPARQL but require a transformation. We have a tool in place for
> that in Jena. And yes displaying large datasets need dedicated strategies
> for efficient processing. The points raised by you would make a good
> problem statement for our session next year. Not sure if you have attended
> the ApachCon 2020 GeoSPARQL session last month but it's where I
> mentioned further collaboration with third party tool developers as a
> possibility. stay tuned.
>
> Marco
>
>
> On Mon, Oct 26, 2020 at 9:14 AM Jean-Marc Vanel
> wrote:
>
> > After reading the official
> > https://jena.apache.org/documentation/geosparql/
> > I'm puzzled as to concrete how to .
> >
> > My simple export use case
> >
> >- I have a TDB 1 database with geo:lat and long properties; the Jena
> >geosparql dependency is added to my application
> >- I have a LeafLet viewer able to display any RDF document with
> geo:lat
> >and long properties ; example map
> ><
> >
> https://semantic-forms.cc:1953/assets/geo-map/geo-map.html?link-prefix=http://semantic-forms.cc:1953/display?displayuri==fr=https://semantic-forms.cc:1953/sparql?query=%0APREFIX+form%3A+%3Chttp%3A%2F%2Fraw.githubusercontent.com%2Fjmvanel%2Fsemantic_forms%2Fmaster%2Fvocabulary%2Fforms.owl.ttl%23%3E+%0APREFIX+rdfs%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E+%0APREFIX+geo%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23%3E+%0APREFIX+foaf%3A+%3Chttp%3A%2F%2Fxmlns.com%2Ffoaf%2F0.1%2F%3E+%0A%0ACONSTRUCT+%7B%0A++%3Fthing+geo%3Along+%3FLONG+.%0A++%3Fthing+geo%3Alat+%3FLAT+.%0A++%3Fthing+rdfs%3Alabel+%3FLAB+.%0A++%3Fthing+foaf%3Adepiction+%3FIMG+.%0A%7D+WHERE+%7B%0A++graph+%3Fg+%7B%0A%3Fthing+%3Chttp%3A%2F%2Fpurl.org%2FNET%2Fc4dm%2Fevent.owl%23produced_in%3E+%3Chttp%3A%2F%2Fsemantic-forms.cc%3A1952%2Fldp%2FCormoz%3E+.%0A++%7D%0A++graph+%3Fgcoord+%7B%0A%3Fthing+geo%3Along+%3FLONG+.%0A%3Fthing+geo%3Alat+%3FLAT+.%0A++%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fg1+%7B%0A%3Fthing+rdfs%3Alabel+%3FLAB+%7D+%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fg2+%7B%0A%3Fthing+%3Curn%3AdisplayLabel%3E+%3FLAB+%7D+%7D%0A%0A++OPTIONAL+%7B%0A+++graph+%3Fg3+%7B%0A%3Fthing+foaf%3Adepiction+%3FIMG+%7D+%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fg4+%7B%0A%3Fthing+foaf%3Aimg+%3FIMG+%7D+%7D%0A%0AOPTIONAL+%7B%0A+++graph+%3FgrCount+%7B%0A%3Fthing+form%3AlinksCount+%3FCOUNT.%0A++%7D+%7D%0A%7D%0AORDER+BY+DESC%28%3FCOUNT%29%0A
> > >
> >; but I found that JavaScript based displayers are falling on their
> > knees
> >(becoming very slow) for thousands of points
> >- so I plan to use QGIS, so I need an export from RDF to one of the
> >formats QGIS supports: GML, GeoJSON , etc
> >
> > How can I do that ?
> > Do I have to use one of the convert* methods in
> >
> >
> https://jena.apache.org/documentation/javadoc/geosparql/index.html?org/apache/jena/geosparql/configuration/GeoSPARQLOperations.html
> > ?
> >
> > Jean-Marc Vanel
> > <
> >
> http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> > >
> > +33 (0)6 89 16 29 52
> > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
> > Chroniques jardin
> > <
> >
> http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle
> > >
> >
>
>
> --
>
>
> ---
> Marco Neumann
> KONA
>
Jena geosparql , simple export use case ; practical doc. needed
After reading the official https://jena.apache.org/documentation/geosparql/ I'm puzzled as to concrete how to . My simple export use case - I have a TDB 1 database with geo:lat and long properties; the Jena geosparql dependency is added to my application - I have a LeafLet viewer able to display any RDF document with geo:lat and long properties ; example map <https://semantic-forms.cc:1953/assets/geo-map/geo-map.html?link-prefix=http://semantic-forms.cc:1953/display?displayuri==fr=https://semantic-forms.cc:1953/sparql?query=%0APREFIX+form%3A+%3Chttp%3A%2F%2Fraw.githubusercontent.com%2Fjmvanel%2Fsemantic_forms%2Fmaster%2Fvocabulary%2Fforms.owl.ttl%23%3E+%0APREFIX+rdfs%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E+%0APREFIX+geo%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23%3E+%0APREFIX+foaf%3A+%3Chttp%3A%2F%2Fxmlns.com%2Ffoaf%2F0.1%2F%3E+%0A%0ACONSTRUCT+%7B%0A++%3Fthing+geo%3Along+%3FLONG+.%0A++%3Fthing+geo%3Alat+%3FLAT+.%0A++%3Fthing+rdfs%3Alabel+%3FLAB+.%0A++%3Fthing+foaf%3Adepiction+%3FIMG+.%0A%7D+WHERE+%7B%0A++graph+%3Fg+%7B%0A%3Fthing+%3Chttp%3A%2F%2Fpurl.org%2FNET%2Fc4dm%2Fevent.owl%23produced_in%3E+%3Chttp%3A%2F%2Fsemantic-forms.cc%3A1952%2Fldp%2FCormoz%3E+.%0A++%7D%0A++graph+%3Fgcoord+%7B%0A%3Fthing+geo%3Along+%3FLONG+.%0A%3Fthing+geo%3Alat+%3FLAT+.%0A++%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fg1+%7B%0A%3Fthing+rdfs%3Alabel+%3FLAB+%7D+%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fg2+%7B%0A%3Fthing+%3Curn%3AdisplayLabel%3E+%3FLAB+%7D+%7D%0A%0A++OPTIONAL+%7B%0A+++graph+%3Fg3+%7B%0A%3Fthing+foaf%3Adepiction+%3FIMG+%7D+%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fg4+%7B%0A%3Fthing+foaf%3Aimg+%3FIMG+%7D+%7D%0A%0AOPTIONAL+%7B%0A+++graph+%3FgrCount+%7B%0A%3Fthing+form%3AlinksCount+%3FCOUNT.%0A++%7D+%7D%0A%7D%0AORDER+BY+DESC%28%3FCOUNT%29%0A> ; but I found that JavaScript based displayers are falling on their knees (becoming very slow) for thousands of points - so I plan to use QGIS, so I need an export from RDF to one of the formats QGIS supports: GML, GeoJSON , etc How can I do that ? Do I have to use one of the convert* methods in https://jena.apache.org/documentation/javadoc/geosparql/index.html?org/apache/jena/geosparql/configuration/GeoSPARQLOperations.html ? Jean-Marc Vanel <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: simple SPARQL query with UNION => timeout
But putting the most general criterion
?thing a ?CLASS .
at the end fixes the issue !!! :) .
I thought that the Jena optimizer would do it by itself
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le mar. 6 oct. 2020 à 16:24, Jean-Marc Vanel a
écrit :
> I suspect there is a better way to do it.
> Of course both requests without UNION run very fast, one with 56 results ,
> the other 6.
>
> PREFIX dcat: <http://www.w3.org/ns/dcat#>
> PREFIX void: <http://rdfs.org/ns/void#>
> CONSTRUCT {
> ?thing a ?CLASS . }
> WHERE {
> ?thing a ?CLASS .
> { ?thing a dcat:Dataset .} UNION
> { ?thing a void:Dataset .}
> }
>
>
> Jean-Marc Vanel
> <http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
> +33 (0)6 89 16 29 52
> Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
> Chroniques jardin
> <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
>
>
simple SPARQL query with UNION => timeout
I suspect there is a better way to do it.
Of course both requests without UNION run very fast, one with 56 results ,
the other 6.
PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX void: <http://rdfs.org/ns/void#>
CONSTRUCT {
?thing a ?CLASS . }
WHERE {
?thing a ?CLASS .
{ ?thing a dcat:Dataset .} UNION
{ ?thing a void:Dataset .}
}
Jean-Marc Vanel
<http://semantic-forms.cc:1952/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Memory increase in TDB 1 incremental load
So I could FIX my problem of OOM exception during incremental TBD 1 load. It boiled down to 1. QueueBatchSize = 0 // same as Fuseki, 2. raise memory from 900M to 1100M. 3. avoid unnecessary things like sync(), dataset.close() ; use only one of the patterns for transaction The memory kept around there figures: Used Memory:534.842 Free Memory:565.158 Total Memory:1100.0 Max Memory:1100.0 Maybe there is a tendency to raise on the average; I'll plot these data ... Understandably, the delay for a HTML page during the load raised to 10 to 20 seconds, but this is tolerable as long as this does not kill the server. My catching of OOM exceptions is not perfect :( . NOTES: I'm not sure if Jena TDB loading transaction does all the flushing and memory cleaning in the same thread; otherwise a delay between requests is necessary. I couldn't find how to get access to an instance of TransactionManager to call these methods : getCountActiveReaders() ; getCountActiveWriters ; getCountActiveWriters() ; getQueueLength . Le jeu. 3 sept. 2020 à 18:49, Andy Seaborne a écrit : > If necessary, you can reduce the footprint of the node cache. See > StoreParams. But first, determine the root cause. > https://jena.apache.org/documentation/tdb/store-parameters.html > tdb.node2nodeid_cache_size > tdb.nodeid2node_cache_size > (BTW they are larger in TDB2) > > > The node table cache also takes > >> space (which stabilizes after a while - it's LRU). > I keep this under my elbow .. Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 > >>>>> Réseau social http://semantic-forms.cc:1952/ > >
Re: Memory increase in TDB 1 incremental load
Le jeu. 3 sept. 2020 à 14:35, Andy Seaborne a écrit : > > Used Memory:794 > > Free Memory:105 > >loadAction: AFTER System.gc(): Free Memory: 137 Mb > > java.lang.OutOfMemoryError: Java heap space > > Used Memory:892 > > Free Memory:7 > > You'll get more useful figures if you run -Xms and -Xmx set to the same > number. > Indeed that's what man java recommends for a server. Only -Xmx was set. Should I care about -Xss and -Xmn ? > Runtime.getFreeMemory does not reflect the possibility of > growing the heap. > Then what could I use ? > All I can see is that: >Some batch is much larger (in bytes) than others. >The total heap size isn't very big. I raise it to 1100M ; I only have a total of 4Gb at the moment . The node table cache also takes > space (which stabilizes after a while - it's LRU). > Are there some indicators in the API to print for a TDB dashboard ? Is there already a doc. article about tuning TDB and the JVM ? I'll definitely try TDB2 > > >> Jean-Marc Vanel > >>> +33 (0)6 89 16 29 52 > >>> Réseau social http://semantic-forms.cc:1952/ >
Re: Memory increase in TDB 1 incremental load
Le mer. 2 sept. 2020 à 23:39, Andy Seaborne a écrit :
> On 02/09/2020 17:03, Jean-Marc Vanel wrote:
> > To load a big Turtle without stopping my (non Fuseki) server, I did an
> HTTP
> > client that sends the triples, about 3_000_000 (300_000 subjects), by
> chunks of 10_000, talking
> > to a SPARQL 1.1 Graph Store HTTP Protocol
> > The Java heap memory keeps growing until the final crash.
>
> After how many chunks?
>
12
Here is a grep from the logs:
Used Memory:36
Free Memory:56
loadAction: AFTER System.gc(): Free Memory: 64 Mb
Used Memory:59
Free Memory:297
loadAction: AFTER System.gc(): Free Memory: 76 Mb
Used Memory:218
Free Memory:259
loadAction: AFTER System.gc(): Free Memory: 303 Mb
Used Memory:405
Free Memory:153
loadAction: AFTER System.gc(): Free Memory: 240 Mb
Used Memory:555
Free Memory:137
loadAction: AFTER System.gc(): Free Memory: 318 Mb
Used Memory:655
Free Memory:131
loadAction: AFTER System.gc(): Free Memory: 291 Mb
Used Memory:366
Free Memory:533
loadAction: AFTER System.gc(): Free Memory: 76 Mb
Used Memory:227
Free Memory:151
loadAction: AFTER System.gc(): Free Memory: 206 Mb
Used Memory:463
Free Memory:214
loadAction: AFTER System.gc(): Free Memory: 359 Mb
Used Memory:576
Free Memory:101
loadAction: AFTER System.gc(): Free Memory: 319 Mb
Used Memory:702
Free Memory:105
loadAction: AFTER System.gc(): Free Memory: 287 Mb
Used Memory:794
Free Memory:105
loadAction: AFTER System.gc(): Free Memory: 137 Mb
java.lang.OutOfMemoryError: Java heap space
Used Memory:892
Free Memory:7
Used Memory:893
Free Memory:6
Uncaught error from thread [application-scheduler-1]: Java heap space,
shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for
ActorSystem[application]
TDB1 batches commits and only writes back every 10 commits (if it can -
> no transaction bloking the momentary sync that is needed).
> Fuseki switches this off: TransactionManager.QueueBatchSize = 0;
>
I forgot that I already had set *QueueBatchSize = 5* after similar
problems, on your advice :) . I will now go down to 2.
There is also the setting JournalThresholdSize, but the javadoc does
specify the unit: triples, bytes ?
Also, the difference between MaxQueueThreshold and QueueBatchSize is not
clear enough, unless one looks at the issues mentioned.
I also saw these useful indicators :
getCountActiveReaders()
getCountActiveWriters()
getQueueLength()
I should propose enhancements of the documentation after having solved my
problems.
But 10 chunks of 10K triples is small so either the heap size if too
> small or something is stopping every 10th write back.
>
I don't see what it could be, but I hope that the above indicators will
give a hint.
...
> > TDB.sync(dataset)
> Pointless. It is a transaction.
> Don't call close(). Call dataset.end
>
I applied these advices...
or better use Txn:
> Txn.executeWrite(dataset, ()->{
>
I use Banana-RDF <https://github.com/banana-rdf/banana-rdf>, a Scala
library that has a similar feature:
def rw[T](dataset: Dataset, body: => T): Try[T] = Try {
dataset.begin(ReadWrite.WRITE)
try {
val result = body
dataset.commit()
result
} finally dataset.end()
}
A good thing is that a Scala Lambda function can be passed to a Java
library, and vice versa.
> Jean-Marc Vanel
> > +33 (0)6 89 16 29 52
> > Réseau social http://semantic-forms.cc:1952/
>
Memory increase in TDB 1 incremental load
To load a big Turtle without stopping my (non Fuseki) server, I did an HTTP
client that sends the triples, about 300_000, by chunks of 10_000, talking
to a SPARQL 1.1 Graph Store HTTP Protocol
<http://www.w3.org/TR/2013/REC-sparql11-http-rdf-update-20130321> service.
The Java heap memory keeps growing until the final crash.
There were no interactive users at the time.
Following advice in
https://jena.apache.org/documentation/tdb/faqs.html#fuseki-tdb-memory-leak
, I tried sync() and dataset.close() on a newly created Dataset, but no
improvement.
Here is a sketch what my code does:
val dataset = TDBFactory.createDataset("TDBtest")
dataset.begin(ReadWrite.WRITE)
val dsg = dataset.asDatasetGraph
dsg.add(
NodeFactory.createURI("urn:gr" + i),
NodeFactory.createURI("urn:s"),
NodeFactory.createURI("urn:p"),
NodeFactory.createURI("urn:o"))
TDB.sync(dataset)
dataset.commit()
dataset.close()
Note that there is a singleton Dataset that is used for the interactive
HTML usage.
I tried this
if( ! localDataset . isInTransaction)
TDBFactory.release(localDataset)
but it gives:
org.apache.jena.sparql.JenaTransactionException: Already closed
at
org.apache.jena.tdb.transaction.DatasetGraphTransaction.checkNotClosed(DatasetGraphTransaction.java:337)
at
org.apache.jena.tdb.transaction.DatasetGraphTransaction.isInTransaction(DatasetGraphTransaction.java:233)
at
org.apache.jena.sparql.core.DatasetImpl.isInTransaction(DatasetImpl.java:146)
Would it be a use case for moving to TDB2?
I should look at what tdbloader does ...
Jean-Marc Vanel
+33 (0)6 89 16 29 52
Réseau social http://semantic-forms.cc:1952/
Re: Federated update query with SERVICE Wikidata : no change in TDB (1)
Sorry , I forgot that my already filled database on my laptop, although
behaving good so far, has every reason to be roten, as it is used for tests
and sometimes stopped abruptly .
Otherwise, everything is fine with federated query with Wikidata as the
service !
Next, I'll try federated query with dbPedia.org ... Last time I tried that
it failed because of dbPedia.org .
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le lun. 20 juil. 2020 à 13:23, Andy Seaborne a écrit :
> Works for me.
>
> No database "TDB"
>
> tdbupdate --loc TDB --verbose --debug --update=U.ru
>
> tdbdump --loc TDB.
>
> 32903 quads.
>
> On 20/07/2020 11:28, Jean-Marc Vanel wrote:
> > This is what I tried :
> >
> > ~/apps/apache-jena-3.16.0/bin/tdbupdate --loc TDB --verbose --debug
> > --update=taxref-alignments.upd.rq
> >
> > Where file taxref-alignments.upd.rq contains :
>
> .ru
>
> but it works anyway.
>
> >
> > prefix wdtn: <http://www.wikidata.org/prop/direct-normalized/>
> > prefix owl: <http://www.w3.org/2002/07/owl#>
> > # Example of adding alignment triples from external SPARQL endpoint
> > INSERT {
> >GRAPH ?TARGET_GRAPH {
> > ?TAXON_TAXREF owl:sameAs ?TAXON_WKDT .
> >} }
> > WHERE {
> >BIND ( AS ?TARGET_GRAPH )
> >SERVICE <https://query.wikidata.org/sparql> {
> >?TAXON_WKDT wdtn:P3186 ?TAXON_TAXREF .
> > } }
> >
> > Result : the *TDB is not changed*; there are warnings that does not seems
> > important:
> >
> > 11:57:23 WARN ResponseProcessCookies :: Invalid cookie header:
> > "Set-Cookie:
>
> Something from HTTP client.
>
> > WMF-Last-Access=20-Jul-2020;Path=/;HttpOnly;secure;Expires=Fri, 21 Aug
> 2020
> > 00:00:00 GMT". Invalid 'expires' attribute: Fri, 21 Aug 2020 00:00:00 GMT
> > 11:57:23 WARN ResponseProcessCookies :: Invalid cookie header:
> > "Set-Cookie: WMF-Last-Access-Global=20-Jul-2020;Path=/;Domain=.
> wikidata.org;HttpOnly;secure;Expires=Fri,
> > 21 Aug 2020 00:00:00 GMT". Invalid 'expires' attribute: Fri, 21 Aug 2020
> > 00:00:00 GMT
> >
> > I checked by several queries that TDB is not changed:
> >
> > SELECT * WHERE { GRAPH ?G {
> >?S ?P <http://taxref.mnhn.fr/lod/taxon/29388/13.0> .
> > } } LIMIT 100
>
> It is a subject in the database.
>
> <http://taxref.mnhn.fr/lod/taxon/29388/13.0> ?P ?O .
>
> >
> > SELECT * WHERE { GRAPH
> > {?S ?P ?O . } } LIMIT 100
>
> works for me.
>
> > How could I debug this, not knowing whether Jena or Wikidata server is at
> > fault ?
> >
> >
> > NOTE 1 : the corresponding, also federated, query gives results:
> >
> > prefix wdtn: <http://www.wikidata.org/prop/direct-normalized/>
> > SELECT *
> > WHERE {
> >SERVICE <https://query.wikidata.org/sparql> {
> >?TAXON_WKDT wdtn:P3186 ?TAXON_TAXREF .
> >}
> > } LIMIT 44
> >
> > when run by:
> > ~/apps/apache-jena-3.16.0/bin/rsparql --debug --verbose --service=
> > http://localhost:9000/sparql --query taxref-alignments.rq
>
> Not rsparql.
>
> sparql.
>
> You have sent the qhowl queyr to wikidata, not the SERVICE and LIMIT wil
> be done there.
>
> > where localhost is semantic_forms server (Jena based) ;
> > or by:
> > ~/apps/apache-jena-3.16.0/bin/rsparql --debug --verbose --service=
> > https://query.wikidata.org/sparql --query taxref-alignments.rq
> >
> > NOTE 2: I also tried with a more ordinary graph ( fictitious ) URI : <
> > http://alignments.taxref.org>
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33 (0)6 89 16 29 52
> > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
> > Chroniques jardin
> > <
> http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle
> >
> >
>
Federated update query with SERVICE Wikidata : no change in TDB (1)
This is what I tried :
~/apps/apache-jena-3.16.0/bin/tdbupdate --loc TDB --verbose --debug
--update=taxref-alignments.upd.rq
Where file taxref-alignments.upd.rq contains :
prefix wdtn: <http://www.wikidata.org/prop/direct-normalized/>
prefix owl: <http://www.w3.org/2002/07/owl#>
# Example of adding alignment triples from external SPARQL endpoint
INSERT {
GRAPH ?TARGET_GRAPH {
?TAXON_TAXREF owl:sameAs ?TAXON_WKDT .
} }
WHERE {
BIND ( AS ?TARGET_GRAPH )
SERVICE <https://query.wikidata.org/sparql> {
?TAXON_WKDT wdtn:P3186 ?TAXON_TAXREF .
} }
Result : the *TDB is not changed*; there are warnings that does not seems
important:
11:57:23 WARN ResponseProcessCookies :: Invalid cookie header:
"Set-Cookie:
WMF-Last-Access=20-Jul-2020;Path=/;HttpOnly;secure;Expires=Fri, 21 Aug 2020
00:00:00 GMT". Invalid 'expires' attribute: Fri, 21 Aug 2020 00:00:00 GMT
11:57:23 WARN ResponseProcessCookies :: Invalid cookie header:
"Set-Cookie:
WMF-Last-Access-Global=20-Jul-2020;Path=/;Domain=.wikidata.org;HttpOnly;secure;Expires=Fri,
21 Aug 2020 00:00:00 GMT". Invalid 'expires' attribute: Fri, 21 Aug 2020
00:00:00 GMT
I checked by several queries that TDB is not changed:
SELECT * WHERE { GRAPH ?G {
?S ?P <http://taxref.mnhn.fr/lod/taxon/29388/13.0> .
} } LIMIT 100
SELECT * WHERE { GRAPH
{?S ?P ?O . } } LIMIT 100
How could I debug this, not knowing whether Jena or Wikidata server is at
fault ?
NOTE 1 : the corresponding, also federated, query gives results:
prefix wdtn: <http://www.wikidata.org/prop/direct-normalized/>
SELECT *
WHERE {
SERVICE <https://query.wikidata.org/sparql> {
?TAXON_WKDT wdtn:P3186 ?TAXON_TAXREF .
}
} LIMIT 44
when run by:
~/apps/apache-jena-3.16.0/bin/rsparql --debug --verbose --service=
http://localhost:9000/sparql --query taxref-alignments.rq
where localhost is semantic_forms server (Jena based) ;
or by:
~/apps/apache-jena-3.16.0/bin/rsparql --debug --verbose --service=
https://query.wikidata.org/sparql --query taxref-alignments.rq
NOTE 2: I also tried with a more ordinary graph ( fictitious ) URI : <
http://alignments.taxref.org>
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Check a plain literal by API
Indeed I made a confusion between the actual literal content (unrestricted)
and the Turtle representation of it.
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33
(0)6 89 16 29 52
Le jeu. 18 juin 2020 à 13:36, Andy Seaborne a écrit :
>
>
> On 18/06/2020 09:28, Jean-Marc Vanel wrote:
> > Literals can have annoying errors like "Illegal escape sequence value" .
> > These errors create exceptions by Turtle parser.
> > But the API does not check anything:
> > NodeFactory.createLiteral("ggg\\
> "\\" is single \ in scala/java.
>
> That is illegal in Turtle syntax, not an illegal literal.
>
> \< is not a Turtle escape.
>
> [172s] PN_LOCAL_ESC::=
> '\' ('_' | '~' | '.' | '-' | '!' | '$' | '&' | "'" | '(' | ')' | '*' |
> '+' | ',' | ';' | '=' | '/' | '?' | '#' | '@' | '%')
>
> But '\<' is a legal string in RDF - it is not an escape, RDF does not
> have escapes - syntaxes do.
>
> Andy
>
> >
> > Event the org.apache.jena.riot.checker.CheckerLiterals is silent:
> >
> > val handler =
> > org.apache.jena.riot.system.ErrorHandlerFactory.errorHandlerStrict
> > val checker = new org.apache.jena.riot.checker.CheckerLiterals(handler)
> > val obj = "ggg\\ > import org.apache.jena.graph._
> > val lit = NodeFactory.createLiteral(obj)
> > checker.check(lit, 1L, 1L)
> > res28: Boolean = true
> >
> > So, is there a way to check a plain literal by API, other than creating a
> > fake triple and parse it ?
> >
> > Note: the code samples are in Scala command line (REPL), but I think it's
> > easy to understand for Javaists ...
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33 (0)6 89 16 29 52
> > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
> > Chroniques jardin
> > <
> http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle
> >
> >
>
Check a plain literal by API
Literals can have annoying errors like "Illegal escape sequence value" .
These errors create exceptions by Turtle parser.
But the API does not check anything:
NodeFactory.createLiteral("ggg\\http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
Re: slow loading in TDB with Lucene
Sorry for the late answer ; I'm aware of the bad side of autocommit which I never use. I did wrap In Transaction the call to removeGraph I'll make measurements you asked to assert the respective CPU and elapsed times for loading RDF and indexing the text. But for the time being, I had to solve my issue of loading data without stopping my SPARQL + HTML server . So I wrote a client RDF uploader, talking to the SPARQL graph store protocol : https://www.w3.org/TR/sparql11-http-rdf-update/ splitting given RDF file in chunks of 1 triples for sending : https://github.com/jmvanel/semantic_forms/blob/master/scala/clients/src/main/scala/deductions/runtime/clients/RDFuploader.scala#L66 I used for the first time the Riot parser with callback (org.apache.jena.riot.system.StreamRDFBase) , which I'll also test for performance. It is understandable that it can be slow, since the input was a Turtle file , not N-Triple . On server side, I modularized my code, so that now several instances TDB(1) are created on the same directory, which is not a problem for TDB. But apparently this is a problem for Lucene: there is a LockObtainFailedException: "Lock held by this virtual machine: ../LUCENE/write.lock" when creating the second TDB instance connected to Lucene. So 'll ensure that only one TDB database is instantiated. Or maybe I use badly the API (it's configured by API not RDF configuration). NOTES - I'm not sure if LUCENE/write.lock is deleted in all cases when closing the TDB, although it has been specified at text index creation: TextDatasetFactory.create(... closeIndexOnDSGClose = true) - using the GUI Luke in lucene-8.5.2 is useful to inspect Lucene index Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Le sam. 6 juin 2020 à 11:45, Andy Seaborne a écrit : > > > On 04/06/2020 10:25, Jean-Marc Vanel wrote: > > Hi > > > > It took hours loading a TTL document with text indexing (in TDB 3.15.0). > > The TTL document is Taxrefld_taxonomy_classes.ttl (size: 2_676_428 > triples) > > in zip taxref12-core.zip > > < > https://github.com/frmichel/taxref-ld/blob/master/dataset/12.0/taxref12-core.zip > > > > . > > Have you tried with and without the text index to get a information > about where the time is going? > > This is a combination setup so it is harder to say where time is going > without an experiment. > > > > > This method in DatasetGraph is called : > > public void add(Node g, Node s, Node p, Node o) ; > > > > With logging at debug level, it appeared that most of the elapsed time is > > taken by removing the graph, one entity at a time. > > > > In fact I explicitly call *removeGraph()* before, because the data is > > stored in provenance specific graphs in this database. > > The text index has to be updated as well, and I think there is nothing > special about removeGraph for a test index so it undoes all the indexing. > > Also - lucene indexing may be slower that the TDB part. > > > > > Is there a way to accelerate things ? > > I wondered if wrapping removeGraph()operation in a transaction is > mandatory > > or useful. > > useful - If you don't have a transaction, TDB1 is going to be less safe > for your data. > > > At runtime Jena does not protest about that ... > > TDB1 does not ... but it is better to use a transaction and its > mandatory for TDB2. > > Adding an autocommit mode is not as good as it may seem. Like in SQL, > autocommit is nothing more than an automatic transaction around each > step and very easily becomes extremely slow. > > Andy > > > > > A typical block in the data: > > <http://taxref.mnhn.fr/lod/taxon/629656/12.0> > > aowl:Class ; > > rdfs:isDefinedBy < > > http://taxref.mnhn.fr/lod/taxref-ld/12.0> ; > > > > *rdfs:label "Eranthemum pulchellum" ;* > > rdfs:subClassOf < > http://taxref.mnhn.fr/lod/taxon/452421/12.0> ; > > schema:mainEntityOfPage < > > https://inpn.mnhn.fr/espece/cd_nom/629656?lg=en> ; > > taxrefprop:habitat taxrefhab:FreshWater , > > taxrefhab:Terrestrial ; > > taxrefprop:hasRank taxrefrk:Species ; > > taxrefprop:hasReferenceName < > http://taxref.mnhn.fr/lod/name/629656> > > ; > > taxrefprop:hasSynonym< > http://taxref.mnhn.fr/lod/name/633029> > > , <http://taxref.mnhn.fr/lod/name/637984> , < > > http://taxref.mnhn.fr/lod/name/634312> ; > > foaf:homepage< > > https://inpn.mnhn.fr/espece/cd_nom/629656?lg=en> . > > > > Jean-Marc Vanel > > < > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > +33 (0)6 89 16 29 52 > > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > > Chroniques jardin > > < > http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle > > > > >
slow loading in TDB with Lucene
Hi It took hours loading a TTL document with text indexing (in TDB 3.15.0). The TTL document is Taxrefld_taxonomy_classes.ttl (size: 2_676_428 triples) in zip taxref12-core.zip <https://github.com/frmichel/taxref-ld/blob/master/dataset/12.0/taxref12-core.zip> . This method in DatasetGraph is called : public void add(Node g, Node s, Node p, Node o) ; With logging at debug level, it appeared that most of the elapsed time is taken by removing the graph, one entity at a time. In fact I explicitly call *removeGraph()* before, because the data is stored in provenance specific graphs in this database. Is there a way to accelerate things ? I wondered if wrapping removeGraph()operation in a transaction is mandatory or useful. At runtime Jena does not protest about that ... A typical block in the data: <http://taxref.mnhn.fr/lod/taxon/629656/12.0> aowl:Class ; rdfs:isDefinedBy < http://taxref.mnhn.fr/lod/taxref-ld/12.0> ; *rdfs:label "Eranthemum pulchellum" ;* rdfs:subClassOf <http://taxref.mnhn.fr/lod/taxon/452421/12.0> ; schema:mainEntityOfPage < https://inpn.mnhn.fr/espece/cd_nom/629656?lg=en> ; taxrefprop:habitat taxrefhab:FreshWater , taxrefhab:Terrestrial ; taxrefprop:hasRank taxrefrk:Species ; taxrefprop:hasReferenceName <http://taxref.mnhn.fr/lod/name/629656> ; taxrefprop:hasSynonym<http://taxref.mnhn.fr/lod/name/633029> , <http://taxref.mnhn.fr/lod/name/637984> , < http://taxref.mnhn.fr/lod/name/634312> ; foaf:homepage < https://inpn.mnhn.fr/espece/cd_nom/629656?lg=en> . Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: TDB bug: IRI with accent é and _ is altered in TDB
OK, Saw https://issues.apache.org/jira/plugins/servlet/mobile#issue/JENA-1890 Fine and fixed !:) Any idea of a new release date ? Jean-Marc Vanel +33 (0)6 89 16 29 52 Réseau social http://semantic-forms.cc:1952/ Le sam. 2 mai 2020 à 11:31, Andy Seaborne a écrit : > Same: JENA-1890 > > On 02/05/2020 08:21, Jean-Marc Vanel wrote: > > An IRI with an accent é and _ is altered in TDB. > > Example of such an IRI: > > http://Théodore_Monod <http://xn--thodore_monod-chb> < > http://xn--thodore_monod-chb> > > > > The test prints: > > (triple <http://uri> <http://prop> <http://Théodore > <http://xn--thodore-cya> > > <http://xn--thodore-cya>�Monod>) > > (triple <http://uri> <http://prop> <http://Victor_Hugo>) > > > > The test is here: > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/test/TestGraphIRI.java#L18 > > > > With TDB 1 , Jena version = "3.14.0" ; non existing database before > test. > > > > Jean-Marc Vanel > > < > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > +33 (0)6 89 16 29 52 > > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > > Chroniques jardin > > < > http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle > > > > >
TDB bug: IRI with accent é and _ is altered in TDB
An IRI with an accent é and _ is altered in TDB. Example of such an IRI: http://Théodore_Monod <http://xn--thodore_monod-chb> The test prints: (triple <http://uri> <http://prop> <http://Théodore <http://xn--thodore-cya>�Monod>) (triple <http://uri> <http://prop> <http://Victor_Hugo>) The test is here: https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/test/TestGraphIRI.java#L18 With TDB 1 , Jena version = "3.14.0" ; non existing database before test. Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Some malediction in http://dbpedia.org/resource/User_guide => StringIndexOutOfBoundsException in TDB
As was stated by Andy, this is not a parsing issue.
riot is not reporting anything, nor rapper
<http://librdf.org/raptor/rapper.html> .
This is an issue with how TDB renders the URI once it has been stored in
TDB.
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Le sam. 25 avr. 2020 à 09:34, Lorenz Buehmann <
buehm...@informatik.uni-leipzig.de> a écrit :
> Hi,
>
> I tried with cURL + riot CLI tools manually and can't reproduce the
> parsing issue, neither with RDF/XML nor with Turtle.
>
> curl -L -H "Accept: text/turtle" http://dbpedia.org/resource/User_guide
> > /tmp/test.ttl
> curl -L -H "Accept: application/rdf+xml"
> http://dbpedia.org/resource/User_guide > /tmp/test.rdf
>
>
> I know, that a few years ago DBpedia (resp. its Virtuoso backend) had
> some issues with serialization, but this has been fixed long time ago.
>
> Also, I don't understand what you mean by "suspicious"? The parser can
> easily convert the UTF-8 encoded URIs as expected:
>
> riot --check /tmp/test.ttl
>
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://nl.dbpedia.org/resource/Handleiding> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://cs.dbpedia.org/resource/Uživatelská_příručka> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://wikidata.dbpedia.org/resource/Q1057179> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://www.wikidata.org/entity/Q1057179> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://ko.dbpedia.org/resource/사용_설명서> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://es.dbpedia.org/resource/Guía_del_usuario> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://ja.dbpedia.org/resource/マニュアル> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://it.dbpedia.org/resource/Manuale> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://rdf.freebase.com/ns/m.04mqbf> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://fr.dbpedia.org/resource/Mode_d'emploi> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://yago-knowledge.org/resource/User_guide> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://de.dbpedia.org/resource/Gebrauchsanleitung> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://id.dbpedia.org/resource/Manual_pengguna> .
> <http://dbpedia.org/resource/User_guide>
> <http://www.w3.org/2002/07/owl#sameAs>
> <http://dbpedia.org/resource/User_guide> .
>
> On 24.04.20 22:33, Jean-Marc Vanel wrote:
> > Le ven. 24 avr. 2020 à 22:17, Andy Seaborne a écrit :
> >
> >> On 24/04/2020 15:17, Jean-Marc Vanel wrote:
> >>> How to reproduce with 3.14.0
> >>>
> >>> bin/*tdbloader* --loc TDB --graph=
> http://dbpedia.org/resource/User_guide
> >> \
> >>>--verbose http://dbpedia.org/resource/User_guide
> >> Did the log say anything?
> >>
> > NO, nothing special, neither with --debug .
> >
> > As this is a remote URL, did it all arrive and parse without warnings?
> > No warning.
> >
> > Was the database fresh or was there data in it to start with?
> > database fresh, of course.
> >
> >
> >>> echo "
> >>> CONSTRUCT {
> >>> <http://dbpedia.org/resource/User_guide>
> >>>?P ?O . }
> >>> WHERE { GRAPH ?G {
> >>> <http://dbpedia.org/resource/User_guide>
> >>>?P ?O . } }
> >>> LIMIT
> >>> # 30 # OK
> >>> 35 # KO !!!
> >>> " > /tmp/const.ql
> >>>
> >>> bin/*tdbquery* --debug --loc=TDB --query /tmp/const.ql
> >>>
> >>> And here is the *stack*:
>
Re: Some malediction in http://dbpedia.org/resource/User_guide => StringIndexOutOfBoundsException in TDB
Le ven. 24 avr. 2020 à 22:17, Andy Seaborne a écrit :
>
> On 24/04/2020 15:17, Jean-Marc Vanel wrote:
> > How to reproduce with 3.14.0
> >
> > bin/*tdbloader* --loc TDB --graph=http://dbpedia.org/resource/User_guide
> \
> >--verbose http://dbpedia.org/resource/User_guide
>
> Did the log say anything?
>
NO, nothing special, neither with --debug .
As this is a remote URL, did it all arrive and parse without warnings?
>
No warning.
Was the database fresh or was there data in it to start with?
>
database fresh, of course.
> > echo "
> > CONSTRUCT {
> > <http://dbpedia.org/resource/User_guide>
> >?P ?O . }
> > WHERE { GRAPH ?G {
> > <http://dbpedia.org/resource/User_guide>
> >?P ?O . } }
> > LIMIT
> > # 30 # OK
> > 35 # KO !!!
> > " > /tmp/const.ql
> >
> > bin/*tdbquery* --debug --loc=TDB --query /tmp/const.ql
> >
> > And here is the *stack*:
> >
> > 16:14:23 ERROR BindingTDB :: get1(?O)
> > java.lang.StringIndexOutOfBoundsException: String index out of range: 39
> > at java.lang.String.charAt(String.java:658)
> > at org.apache.jena.atlas.lib.StrUtils.decodeHex(StrUtils.java:212)
> > at org.apache.jena.tdb.store.nodetable.NodecSSE.decode(NodecSSE.java:121)
>
> If the load was clean, the database is intact and it is a decoding bug
> in Jena for an URI. The data has a lot of encoded \u terms but its a URI
> in the object position causing a problem. (I don't see why these are
> encoded - it's not necessary).
>
Indeed these URI are suspect:
<http://fr.dbpedia.org/resource/Mode_d\u0027emploi> ,
<http://es.dbpedia.org/resource/Gu\u00EDa_del_usuario> .
<http://ja.dbpedia.org/resource/\u30DE\u30CB\u30E5\u30A2\u30EB> ,
<
http://cs.dbpedia.org/resource/U\u017Eivatelsk\u00E1_p\u0159\u00EDru\u010Dka>
,
<http://ko.dbpedia.org/resource/\uC0AC\uC6A9_\uC124\uBA85\uC11C> .
> Andy
>
> ...
> > at tdb.tdbquery.main(tdbquery.java:33)
> >
> > NOTE : no problem with apache-jena-3.10.0-SNAPSHOT !?
> >
> >
> > Jean-Marc Vanel
> > <
> http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me
> >
> > +33 (0)6 89 16 29 52
> > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
> > Chroniques jardin
> > <
> http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle
> >
> >
>
Some malediction in http://dbpedia.org/resource/User_guide => StringIndexOutOfBoundsException in TDB
How to reproduce with 3.14.0
bin/*tdbloader* --loc TDB --graph=http://dbpedia.org/resource/User_guide \
--verbose http://dbpedia.org/resource/User_guide
echo "
CONSTRUCT {
<http://dbpedia.org/resource/User_guide>
?P ?O . }
WHERE { GRAPH ?G {
<http://dbpedia.org/resource/User_guide>
?P ?O . } }
LIMIT
# 30 # OK
35 # KO !!!
" > /tmp/const.ql
bin/*tdbquery* --debug --loc=TDB --query /tmp/const.ql
And here is the *stack*:
16:14:23 ERROR BindingTDB :: get1(?O)
java.lang.StringIndexOutOfBoundsException: String index out of range: 39
at java.lang.String.charAt(String.java:658)
at org.apache.jena.atlas.lib.StrUtils.decodeHex(StrUtils.java:212)
at org.apache.jena.tdb.store.nodetable.NodecSSE.decode(NodecSSE.java:121)
at org.apache.jena.tdb.lib.NodeLib.decode(NodeLib.java:120)
at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:97)
at
org.apache.jena.tdb.store.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:182)
at
org.apache.jena.tdb.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:108)
at
org.apache.jena.tdb.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:67)
at
org.apache.jena.tdb.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:128)
at
org.apache.jena.tdb.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:82)
at
org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:50)
at
org.apache.jena.tdb.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
at org.apache.jena.tdb.solver.BindingTDB.get1(BindingTDB.java:126)
at
org.apache.jena.sparql.engine.binding.BindingBase.get(BindingBase.java:104)
at org.apache.jena.sparql.core.Var.lookup(Var.java:85)
at org.apache.jena.sparql.core.Var.lookup(Var.java:80)
at org.apache.jena.sparql.core.Substitute.substitute(Substitute.java:117)
at org.apache.jena.sparql.core.Substitute.substitute(Substitute.java:73)
at org.apache.jena.sparql.modify.TemplateLib.subst(TemplateLib.java:177)
at org.apache.jena.sparql.modify.TemplateLib$1.apply(TemplateLib.java:83)
at org.apache.jena.sparql.modify.TemplateLib$1.apply(TemplateLib.java:73)
at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:352)
at
org.apache.jena.ext.com.google.common.collect.Iterators$ConcatenatedIterator.hasNext(Iterators.java:1340)
at
org.apache.jena.sparql.engine.QueryExecutionBase.execConstruct(QueryExecutionBase.java:219)
at
org.apache.jena.sparql.engine.QueryExecutionBase.execConstruct(QueryExecutionBase.java:207)
at
org.apache.jena.sparql.util.QueryExecUtils.doConstructQuery(QueryExecUtils.java:162)
at
org.apache.jena.sparql.util.QueryExecUtils.executeQuery(QueryExecUtils.java:83)
at arq.query.lambda$queryExec$0(query.java:225)
at org.apache.jena.system.Txn.exec(Txn.java:77)
at org.apache.jena.system.Txn.executeRead(Txn.java:115)
at arq.query.queryExec(query.java:222)
at arq.query.exec(query.java:153)
at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
at tdb.tdbquery.main(tdbquery.java:33)
NOTE : no problem with apache-jena-3.10.0-SNAPSHOT !?
Jean-Marc Vanel
<http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
<http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: TDB blocked
Le mer. 25 mars 2020 à 17:17, Andy Seaborne a écrit : > > On 25/03/2020 14:36, Jean-Marc Vanel wrote: > > Le mer. 25 mars 2020 à 13:07, Andy Seaborne a écrit : > > ... > > >> What's the full stack trace? > >> (What's the blocking thread?) > I was looking for the other thread that is blocking this one at > being(write). Same JVM. > See details below. Tell me if you still want it. > >> Yes, another unended W transaction in the same process might be the > cause. > >> > > I thought rather, another unended W transaction in *another* process, > since > > the tests execution was parallel. > > If I understand correctly, > > parallelExecution in Test := false > > means parallel in the same JVM. > Yes indeed, that's for taking advantage of multicore CPU's. The parameter that actually created the freeze problem in the original post was fork = true and it clear that calling TDBFactory.createDataset() and close() , and other calls, randomly from different JVM's on the same TDB directory is bound to fail. On the other hand, the combination fork := false parallelExecution in Test := true is not good for this kind of tests. My understanding is that TDB is made for multi-thread access in reading and writing (and it works perfectly in my Play Framework application for years), but not made for creating and closing. All tests pass, but one, with this stack (complete!): [info] TimeSeriesTestJena: [info] - notifyDataEvent + getTimeSeries [info] deductions.runtime.semlogs.TimeSeriesTestJena *** ABORTED *** [info] java.util.*ConcurrentModificationException*: Reader = 0, Writer = 2 [info] at org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(DatasetControlMRSW.java:147) [info] at org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(DatasetControlMRSW.java:143) [info] at org.apache.jena.tdb.sys.DatasetControlMRSW.checkConcurrency(DatasetControlMRSW.java:75) [info] at org.apache.jena.tdb.sys.DatasetControlMRSW.startUpdate(DatasetControlMRSW.java:57) [info] at org.apache.jena.tdb.store.nodetupletable.NodeTupleTableConcrete.startWrite(NodeTupleTableConcrete.java:65) [info] at org.apache.jena.tdb.store.nodetupletable.NodeTupleTableConcrete.sync(NodeTupleTableConcrete.java:251) [info] at org.apache.jena.tdb.store.TableBase.sync(TableBase.java:51) [info] at org.apache.jena.tdb.store.DatasetGraphTDB.sync(DatasetGraphTDB.java:253) [info] at org.apache.jena.tdb.transaction.DatasetGraphTransaction.close(DatasetGraphTransaction.java:296) [info] at org.apache.jena.sparql.core.DatasetImpl.close(DatasetImpl.java:231) > Andy > > > > > > Test code: > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/test/scala/deductions/runtime/sparql_cache/TestRDFCache.scala > > = > > Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended) > > Unsafe.park(boolean, long) line: not available [native method] > > LockSupport.park(Object) line: 175 > > Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt() > > line: 836 > > > Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int) > > line: 997 > > > Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int) > > line: 1304 > > Semaphore.acquire() line: 312 > > TransactionManager.acquireWriterLock(boolean) line: 616 > > TransactionManager.beginInternal(TxnType, TxnType, String) line: 358 > > TransactionManager.begin(TxnType, String) line: 343 > > StoreConnection.begin(TxnType, String) line: 128 > > StoreConnection.begin(TxnType) line: 108 > > DatasetGraphTransaction.begin(TxnType) line: 169 > > DatasetGraphTransaction.begin(ReadWrite) line: 162 > > DatasetImpl.begin(ReadWrite) line: 122 > > JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23 > > 52027729.apply() line: not available > > Try$.apply(Function0) line: 213 > > JenaDatasetStore.rw(Dataset, Function0) line: 22 > > JenaDatasetStore.rw(Object, Function0) line: 10 > > TestRDFCache(RDFCacheAlgo).readStoreURI(Object, Object, > > DATASET) line: 368 > > RDFCacheAlgo.readStoreURI$(RDFCacheAlgo, Object, Object, > > Object) line: 366 > > TestRDFCache.readStoreURI(Object, Object, Object) line: 10 > > TestRDFCache(RDFCacheAlgo).readStoreURIinOwnGraph(Object) > > line: 334 > > TestRDFCache(RDFCacheAlgo).readStoreUriInNamedGraph(Object) > > line: 326 > > RDFCacheAlgo.readStoreUriInNamedGraph$(RDFCacheAlgo, Object) > > line: 325 > > TestRDFCache.readStoreUriInNamedGraph(Object) line: 10 > > TestRDFCache.$anonfun$new$4(TestRDFCache) line: 63 > > 1708361043.apply$mcV$sp() line: not available > > 1708361043(JFunction0$mcV$sp).apply() line: 23 &g
Re: TDB blocked
r, Set, Set, Selector[], boolean, ConfigMap, SummaryCounter, Option, LinkedBlockingQueue, Logger[], boolean, boolean, boolean, boolean, boolean, boolean, boolean, boolean, boolean, boolean, ExecutorService) line: 318 Framework$ScalaTestTask.execute(EventHandler, Logger[]) line: 513 ForkMain$Run.lambda$runTest$1(Task, ObjectOutputStream, Logger[]) line: 304 1574877131.call() line: not available FutureTask.run() line: 266 ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1149 ThreadPoolExecutor$Worker.run() line: 624 Thread.run() line: 748 = > Andy > > > On 25/03/2020 11:08, Jean-Marc Vanel wrote: > > I investigated on that. > > Now my test suite passes without Jena TDB trouble. I just added this in > > Scala SBT : > > parallelExecution in Test := false > > > > I didn't know that parallel execution was the default. > > So apparently this was an issue with 2 processes accessing the same TDB . > > Normally this is detected and an explicit Jena TDB message appears, but > not > > this time. > > > > Jean-Marc Vanel > > < > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > > +33 (0)6 89 16 29 52 > > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > > Chroniques jardin > > < > http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle > > > > > > > > Le mar. 24 mars 2020 à 18:56, Jean-Marc Vanel > a > > écrit : > > > >> Hi Jena users! > >> > >> After changing my test suite in* semantic_forms > >> <http://semantic-forms.cc:9112/ldp/semantic_forms>* , I get > >> consistently a thread blocked forever. I upgraded to latest Jena > Version = > >> "3.14.0" , and happily my Web application seems to be working fine on my > >> laptop, and the test suite sort of works, that is running tests one by > one. > >> In my test suite I used to recursively delete the TDB directories after > >> each test. What changed is that instead I kept the same TDB directories > for > >> each test, calling close() after each test. > >> This is with TDB 1. Is another unended transaction the issue? Then how > to > >> discover that ? > >> > >> Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended) > >> Unsafe.park(boolean, long) line: not available [native method] > >> LockSupport.park(Object) line: 175 > >> Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt() > >> line: 836 > >> > Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int) > >> line: 997 > >> > Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int) > >> line: 1304 > >> Semaphore.acquire() line: 312 > >> > >> * TransactionManager.acquireWriterLock(boolean) line: 616 * > >> TransactionManager.beginInternal(TxnType, TxnType, String) line: 358 > >> TransactionManager.begin(TxnType, String) line: 343 > >> StoreConnection.begin(TxnType, String) line: 128 > >> StoreConnection.begin(TxnType) line: 108 > >> DatasetGraphTransaction.begin(TxnType) line: 169 > >> DatasetGraphTransaction.begin(ReadWrite) line: 162 > >> DatasetImpl.begin(ReadWrite) line: 122 > >> JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23 > >> > >> > >> Jean-Marc Vanel > >> < > http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me > > > >> +33 (0)6 89 16 29 52 > >> Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > >> Chroniques jardin > >> < > http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle > > > >> > > >
Re: TDB blocked
I investigated on that. Now my test suite passes without Jena TDB trouble. I just added this in Scala SBT : parallelExecution in Test := false I didn't know that parallel execution was the default. So apparently this was an issue with 2 processes accessing the same TDB . Normally this is detected and an explicit Jena TDB message appears, but not this time. Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle> Le mar. 24 mars 2020 à 18:56, Jean-Marc Vanel a écrit : > Hi Jena users! > > After changing my test suite in* semantic_forms > <http://semantic-forms.cc:9112/ldp/semantic_forms>* , I get > consistently a thread blocked forever. I upgraded to latest Jena Version = > "3.14.0" , and happily my Web application seems to be working fine on my > laptop, and the test suite sort of works, that is running tests one by one. > In my test suite I used to recursively delete the TDB directories after > each test. What changed is that instead I kept the same TDB directories for > each test, calling close() after each test. > This is with TDB 1. Is another unended transaction the issue? Then how to > discover that ? > > Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended) > Unsafe.park(boolean, long) line: not available [native method] > LockSupport.park(Object) line: 175 > Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt() > line: 836 > Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int) > line: 997 > Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int) > line: 1304 > Semaphore.acquire() line: 312 > > * TransactionManager.acquireWriterLock(boolean) line: 616 * > TransactionManager.beginInternal(TxnType, TxnType, String) line: 358 > TransactionManager.begin(TxnType, String) line: 343 > StoreConnection.begin(TxnType, String) line: 128 > StoreConnection.begin(TxnType) line: 108 > DatasetGraphTransaction.begin(TxnType) line: 169 > DatasetGraphTransaction.begin(ReadWrite) line: 162 > DatasetImpl.begin(ReadWrite) line: 122 > JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23 > > > Jean-Marc Vanel > <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> > +33 (0)6 89 16 29 52 > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > Chroniques jardin > <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle> >
TDB blocked
Hi Jena users! After changing my test suite in* semantic_forms <http://semantic-forms.cc:9112/ldp/semantic_forms>* , I get consistently a thread blocked forever. I upgraded to latest Jena Version = "3.14.0" , and happily my Web application seems to be working fine on my laptop, and the test suite sort of works, that is running tests one by one. In my test suite I used to recursively delete the TDB directories after each test. What changed is that instead I kept the same TDB directories for each test, calling close() after each test. This is with TDB 1. Is another unended transaction the issue? Then how to discover that ? Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended) Unsafe.park(boolean, long) line: not available [native method] LockSupport.park(Object) line: 175 Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt() line: 836 Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int) line: 997 Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int) line: 1304 Semaphore.acquire() line: 312 * TransactionManager.acquireWriterLock(boolean) line: 616 * TransactionManager.beginInternal(TxnType, TxnType, String) line: 358 TransactionManager.begin(TxnType, String) line: 343 StoreConnection.begin(TxnType, String) line: 128 StoreConnection.begin(TxnType) line: 108 DatasetGraphTransaction.begin(TxnType) line: 169 DatasetGraphTransaction.begin(ReadWrite) line: 162 DatasetImpl.begin(ReadWrite) line: 122 JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23 Jean-Marc Vanel <http://semantic-forms.cc:9112/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/history?uri=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Storing a lot of strings in TDB store
First this a bad practice: http://people/JohnSmith http://www.w3.org/2001/vcard-rdf/3.0#Region "New York" . You should do http://people/JohnSmith, http://www.w3.org/2001/vcard-rdf/3.0#Region dbpedia:NewYork . that is , http://dbpedia.org/resource/New_York possibly with another object property like http://xmlns.com/foaf/0.1/based_near I understand that you have a database of Vcard stuff, but one must keep in mind that Semantic Web is all about creating links, filling strings is secondary. And then there is no trouble with string at all :) . Jean-Marc Vanel <http://163.172.179.125:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle> Le ven. 15 févr. 2019 à 13:02, Ekaterina Danilova < katja.danilov...@gmail.com> a écrit : > Hello > i would like to ask how TDB2 and Fuseki manages big amounts of string data > (especially repeating data) and what it the best practices. Does it > optimize it somehow? Or is it on us to do some improvements. > > For example, we have a TDB2 storage which we access via Fuseki and example > named graph like this: > [http://people/JohnSmith, http://www.w3.org/2001/vcard-rdf/3.0#Region, > "New > York"] > [http://people/JohnSmith, http://www.w3.org/2001/vcard-rdf/3.0#Other, > "long > long string"] > [http://people/JohnSmith, http://www.w3.org/2001/vcard-rdf/3.0#NAME, "JOHN > SMITH"] > > So, we have JohnSmith person with 2 properties - "Region" and "Other". One > of them is short string of New York, other is long string. > Assume we have 100 000 more people and many of them have same "Region" and > "other" properties. So, what would be the best approach to storing such > data? > > I created 10 000 more named graphs of people with different names but same > other properties and tested the performance. > First I checked 10 000 cases of reading the graphs like this and the > average time was around 4.4 ms (no matter how long are the strings). > > Other option I considered is making "New York" a resource and storing it in > "cities" named graph and doing the same thing with "long long string". So, > the idea is to store the actual string only once.I tested reading the > graphs again on 10 000 cases and didn't notice any change in performance. > The average load time was still 4.4 ms when instead of "New York" and "Long > long String" we had resources URIs. > However, to get the full data, we need to add the actual resources to our > original JohnSmith graph, which adds overhead since we have to get 2 more > named graphs. So, it causes quite expectable drop of performance. > > So, according to my tests the first case (the one described in the graph > example) performed the best, but it feels like we are storing too much > extra information. So, I still wanted to ask on your opinions to such > approach and learn if TDB store makes some inner optimization to the data. >
Re: CSV to rdf
I use my tool , based on Jena and Any23 : https://github.com/jmvanel/semantic_forms/blob/master/doc/en/administration.md#semantize-raw-stuff I you have questions or troubles, mail me privately , I'm willing to enhance the doc or the code, and making a pre-release . Jean-Marc Vanel <http://163.172.179.125:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle> Le jeu. 14 févr. 2019 à 14:59, elio hbeich a écrit : > Dear all > > Do you have any suggestion about tools or XSLT that can transform CSV to > RDF > > Thank you in advance, > Elio HBEICH >
Re: jena-csv
Not knowing that Jena-csv existed, I developed an extension to Any23, adding features for CSV. I use it, and there is documentation: https://github.com/jmvanel/semantic_forms/wiki/Semantization#step-1-direct-mappping Le lun. 14 janv. 2019 17:21, Andy Seaborne a écrit : > The project can't keep acquiring "stuff" so this is our first attempt to > clearup modules where we had little sign they were in use. > > > users@jena was sent an email on 2018-12-14. > Subject: "Retiring Jena modules" > > > https://lists.apache.org/thread.html/edd5876b070f24091e19b5c1dd274ef46c74a0f920d419a29a59f66b@%3Cusers.jena.apache.org%3E > > > On 14/01/2019 15:03, Piotr Nowara wrote:> Hi Bruno, > > > > thanks for your answer. I followed your github link but unfortunately I > > don't see any readme file with instructions there. > > > > It's a maven project - "mvn clean install" > > Or if you want to grab the source to incorporate into you project, then > the source is directly packaged as a sources jar file: > > > http://central.maven.org/maven2/org/apache/jena/jena-csv/3.9.0/jena-csv-3.9.0-sources.jar > > (that is the one that makes browsing Jena source in an IDE work) > > Andy > > On 14/01/2019 16:11, ajs6f wrote: > > The module was retired, as Bruno explained, for lack of any maintainer. > We can't keep code in the codebase when no one will take responsibility for > it. Do you perchance have some time available to commit to maintaining the > jena-csv module? > > > > ajs6f > > > >> On Jan 14, 2019, at 10:17 AM, Piotr Nowara > wrote: > >> > >> Hi Andy, > >> > >> thanks for those links. > >> > >> Please tell me was the jena-csv module retired just because that CSVW > >> standard or there are some bugs that no one has time or interest to fix? > >> > >> I was using jena-csv for loading CSV files and it was doing its job > pretty > >> well. I don't need another W3C standard to produce a ton of boiler-plate > >> code (pardon, set of "metadata" files fot the Web) and then parse it > with > >> Ruby scripts from my JAVA application. > >> > >> Thanks, > >> Piotr > >> > >> pon., 14 sty 2019 o 08:32 Andy Seaborne napisał(a): > >> > >>> CSVW is the W3C standard relating to converting CSV to RDF. There are > >>> tools (e.g. below) that will do that and output a file so it is not > >>> limited to java. > >>> > >>> Andy > >>> > >>> > >>> The W3C CSVW implementation report lists a few in the EARL files: > >>> > >>> > https://w3c.github.io/csvw/tests/reports/index.html#individual-test-results > >>> > >>> https://rubygems.org/gems/rdf-tabular (Ruby / The Unlicense) > >>> https://github.com/theodi/csvlint.rb (Ruby / MIT license) > >>> https://github.com/sebneu/csvw-parser (ruby / MIT) > >>> > >>> Also: > >>> http://www.greggkellogg.net/2015/04/implementing-csv-on-the-web/ > >>> > >>> Not CSVW: > >>> https://github.com/tarql/tarql > >>> > >>> On 14/01/2019 09:03, Bruno P. Kinoshita wrote: > Hi Piotr, > > I believe it was retired for the lack of developer bandwidth to > maintain > >>> it. > > > If you look at the GitHub repository, there is a README at the > jena-csv > >>> folder now. It instructs users to grab - if necessary - jena-csv from > jena > >>> 3.9.0 (https://github.com/apache/jena/tree/jena-3.9.0/jena-csv). > Cheers > Bruno > > > > > > From: Piotr Nowara > To: users@jena.apache.org > Sent: Monday, 14 January 2019 9:38 PM > Subject: jena-csv > > > > Hi, > > > I'm using jena-csv for importing CSV files to Jena. Now the 3.10 > release > > notes say it's no longer a part of the release. > > > What is the replacement for this retired module? > > Why it was retired? (I've read the linked JIRA and discussion but I > > couldn't find the reason) > > > Thanks, > > Piotr > > >>> > > >
Re: Build by program a Dataset that is both textual and spatial
Answers interleaved
Le lun. 24 déc. 2018 à 00:27, Bruno P. Kinoshita a
écrit :
> That's great news Jean! That's some fine investigation work you did!
>
> Couple questions
>
> Q1/ is it something that you think could help others doing the same? If so
> we could add it somewhere in the documentation.
>
Sure,
but, as I wrote previously, it probably only works statically, that is, TDB
updates are probably not reflected in both indices.
I put the new config on a site in production, and I will confirm that.
That being said, many people only use Jena on stable data, so it can suit
their needs..
NOTE that I fixed FIX spatial.EntityDefinition("uri","geo") everywhere in
code : in fact basically 2 variants of config. code work!
The only thing that does not work is sharing the Lucene directory for text
and spatial, because of LockObtainFailedException.
Q2/ just to confirm, that's not doable via assembler files, right? Only
> programmatically?
>
It is *probably* doable via assembler file; I made 2 attempts to reproduce
the code that works with an assembler file, but it does not work:
jena.spatial+text4.assembler.ttl
<https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext4.assembler.ttl>
jena.spatial+text5.assembler.ttl
<https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext5.assembler.ttl>
> Cheers!
> Bruno
>
>
>
>
>
> From: Jean-Marc Vanel
> To: Jena users
> Sent: Monday, 24 December 2018 9:36 AM
> Subject: Re: Build by program a Dataset that is both textual and spatial
>
>
>
> I finally got it to work by coding:
>
>
> https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/lucene/LuceneIndex.scala#L199
>
>
> A confusing documentation item was this in
>
>
> https://jena.apache.org/documentation/query/spatial-query.html#create-spatial-dataset
>
> :
>
>EntityDefinition entDef = new EntityDefinition("entityField",
>
> "geoField");
>
> when it should be:
>
>EntityDefinition entDef = new EntityDefinition("uri", "geo");
>
> as in
>
>
> https://jena.apache.org/documentation/query/spatial-query.html#create-spatial-dataset
>
>
> In fact it is not clear the role of these 2 parameters, but it works with
>
> these values !
>
>
>
>
>
>
>
>
> Le dim. 23 déc. 2018 à 15:49, ajs6f a écrit :
>
>
> > Hello, Jean-Marc,
>
> >
>
> > Thank you for investigating this question so carefully! Of course, it
>
> > should indeed be possible to build up a "text+spatial" dataset along the
>
> > lines you've proposed. But would it do what is needed? I can see how it
>
> > would feed changes to both indexes (which, I would note, I think you
> could
>
> > also do with a DatasetChanges implementation that did the same thing) but
>
> > how would it use both for queries? How to combine, for example, the
> `find`
>
> > methods?
>
> >
>
> > DatasetGraphSpatial seems to use its indexes via these overrides:
>
> >
>
> > // Intecept these and force the use of views.
>
> > @Override
>
> > public Graph getDefaultGraph()
>
> > { return GraphView.createDefaultGraph(this) ; }
>
> > @Override
>
> > public Graph getGraph(Node graphNode)
>
> > { return GraphView.createNamedGraph(this, graphNode) ; }
>
> > // -
>
> >
>
> > whereas DatasetGraphText does something similar, as well as introducing a
>
> > number of `search` methods. How to combine these?
>
> >
>
> > It's a good general problem. I can imagine other kinds of indexes that
>
> > people might want to use (image indexes, timeseries representations) and
>
> > our basic SPI should be able to support using multiple kinds of special
>
> > indexes...
>
> >
>
> > ajs6f
>
> >
>
> > > On Dec 23, 2018, at 9:37 AM, Jean-Marc Vanel >
>
> > wrote:
>
> > >
>
> > > I think it would not be difficult to set Jena for text and spatial, by
>
> > > adding the indexes in the so-called Context
> (org.apache.jena.sparql.util
>
> > > .Context) of the DatasetGraph .
>
> > > But achieving dynamic update of both indexes when TDB is updated will
>
> > > require extending the implementation along the line of preceding mail.
>
> > >
>
> > > I wonder if new GeoSPARQL implementation will be compatible with
> textual
>
> > > indexing .
>
> > >
>
> > >
>
> > >
>
&
Re: Build by program a Dataset that is both textual and spatial
I finally got it to work by coding:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/lucene/LuceneIndex.scala#L199
A confusing documentation item was this in
https://jena.apache.org/documentation/query/spatial-query.html#create-spatial-dataset
:
EntityDefinition entDef = new EntityDefinition("entityField",
"geoField");
when it should be:
EntityDefinition entDef = new EntityDefinition("uri", "geo");
as in
https://jena.apache.org/documentation/query/spatial-query.html#create-spatial-dataset
In fact it is not clear the role of these 2 parameters, but it works with
these values !
Le dim. 23 déc. 2018 à 15:49, ajs6f a écrit :
> Hello, Jean-Marc,
>
> Thank you for investigating this question so carefully! Of course, it
> should indeed be possible to build up a "text+spatial" dataset along the
> lines you've proposed. But would it do what is needed? I can see how it
> would feed changes to both indexes (which, I would note, I think you could
> also do with a DatasetChanges implementation that did the same thing) but
> how would it use both for queries? How to combine, for example, the `find`
> methods?
>
> DatasetGraphSpatial seems to use its indexes via these overrides:
>
> // Intecept these and force the use of views.
> @Override
> public Graph getDefaultGraph()
> { return GraphView.createDefaultGraph(this) ; }
> @Override
> public Graph getGraph(Node graphNode)
> { return GraphView.createNamedGraph(this, graphNode) ; }
> // -
>
> whereas DatasetGraphText does something similar, as well as introducing a
> number of `search` methods. How to combine these?
>
> It's a good general problem. I can imagine other kinds of indexes that
> people might want to use (image indexes, timeseries representations) and
> our basic SPI should be able to support using multiple kinds of special
> indexes...
>
> ajs6f
>
> > On Dec 23, 2018, at 9:37 AM, Jean-Marc Vanel
> wrote:
> >
> > I think it would not be difficult to set Jena for text and spatial, by
> > adding the indexes in the so-called Context (org.apache.jena.sparql.util
> > .Context) of the DatasetGraph .
> > But achieving dynamic update of both indexes when TDB is updated will
> > require extending the implementation along the line of preceding mail.
> >
> > I wonder if new GeoSPARQL implementation will be compatible with textual
> > indexing .
> >
> >
> >
> > Le dim. 23 déc. 2018 à 11:53, Jean-Marc Vanel
> a
> > écrit :
> >
> >> Thanks Bruno for sharing your trials; alas you're not further than I.
> >> Using gist.github.com is indeed better than the regular git, like I do,
> >> for sharing trials.
> >>
> >> I'm getting skeptical about doing spatial + textual as the Jena API
> >> stands.
> >> For one thing, nobody stood up and said "I did it" since 10 days.
> >> Second, spatial and textual indexing follow a similar design, but not
> >> readily mixable:
> >>
> >> - TextDatasetFactory 's create() method returns a DatasetGraphText
> >> - SpatialDatasetFactory create() method returns a DatasetGraphSpatial
> >>
> >> Both classes DatasetGraphText and DatasetGraphSpatial have their
> >> respective index as members and extend DatasetGraphMonitor so that they
> get
> >> notified of add or delete to the DatasetGraph.
> >> Moreover, alas, the DatasetGraphMonitor class does not follow the
> observer
> >> design pattern <https://en.wikipedia.org/wiki/Observer_pattern> , which
> >> would allow to attach several observer objects to the same DatasetGraph.
> >> The monitor in DatasetGraphMonitor class is interface DatasetChanges.
> >>
> >> So the solution I can think of at the moment is creating a mixin
> >> class DatasetGraphTextSpatial with delegation like this:
> >>
> >> class DatasetGraphTextSpatial extends DatasetGraphWrapper {
> >> DatasetGraphText datasetGraphText;
> >> DatasetGraphSpatial datasetGraphSpatial;
> >> public DatasetGraphTextSpatial(DatasetGraph dsg) {
> >>super(dsg);
> >> }
> >> // delegating all methods to both members:
> >> public void commit() {
> >>datasetGraphText.commit();
> >>datasetGraphSpatial.commit();
> >> }
> >> // etc (tedious) ...
> >> }
> >>
> >>
> >> Le dim. 23 déc. 2018 à 09:13, Bruno P. Kinoshita a
> >> écrit :
> >>
> >>> Hi,
> >>>
> >>> I had a crack at it, without luck.
Re: Build by program a Dataset that is both textual and spatial
I think it would not be difficult to set Jena for text and spatial, by
adding the indexes in the so-called Context (org.apache.jena.sparql.util
.Context) of the DatasetGraph .
But achieving dynamic update of both indexes when TDB is updated will
require extending the implementation along the line of preceding mail.
I wonder if new GeoSPARQL implementation will be compatible with textual
indexing .
Le dim. 23 déc. 2018 à 11:53, Jean-Marc Vanel a
écrit :
> Thanks Bruno for sharing your trials; alas you're not further than I.
> Using gist.github.com is indeed better than the regular git, like I do,
> for sharing trials.
>
> I'm getting skeptical about doing spatial + textual as the Jena API
> stands.
> For one thing, nobody stood up and said "I did it" since 10 days.
> Second, spatial and textual indexing follow a similar design, but not
> readily mixable:
>
>- TextDatasetFactory 's create() method returns a DatasetGraphText
>- SpatialDatasetFactory create() method returns a DatasetGraphSpatial
>
> Both classes DatasetGraphText and DatasetGraphSpatial have their
> respective index as members and extend DatasetGraphMonitor so that they get
> notified of add or delete to the DatasetGraph.
> Moreover, alas, the DatasetGraphMonitor class does not follow the observer
> design pattern <https://en.wikipedia.org/wiki/Observer_pattern> , which
> would allow to attach several observer objects to the same DatasetGraph.
> The monitor in DatasetGraphMonitor class is interface DatasetChanges.
>
> So the solution I can think of at the moment is creating a mixin
> class DatasetGraphTextSpatial with delegation like this:
>
> class DatasetGraphTextSpatial extends DatasetGraphWrapper {
> DatasetGraphText datasetGraphText;
> DatasetGraphSpatial datasetGraphSpatial;
> public DatasetGraphTextSpatial(DatasetGraph dsg) {
> super(dsg);
> }
> // delegating all methods to both members:
> public void commit() {
> datasetGraphText.commit();
> datasetGraphSpatial.commit();
> }
> // etc (tedious) ...
> }
>
>
> Le dim. 23 déc. 2018 à 09:13, Bruno P. Kinoshita a
> écrit :
>
>> Hi,
>>
>> I had a crack at it, without luck. Here's my set up:
>> https://gist.github.com/kinow/f391be7036b63f8ac0c08b9832e8bfa7
>>
>> I started with a Spatial only, using a configuration turtle file from
>> Jena Spatial tests (see gist's spatial1.ttl). Loaded the dataset (also from
>> tests), a query with spatial:nearby worked (geoarq-data-1.ttl &
>> geoarq-data-1-query.ttl in the gist above).
>>
>> Then started by modifying the spatial Lucene Index, adding the properties
>> necessary for a Jena Text dataset (see gist's spatial-text.ttl). Tried to
>> make both, the spatial lucene index and the text lucene index point to the
>> same directory.
>>
>>
>> Once I started Fuseki (see comments in gist for rest of set up), the
>> spatial query was still working, but even though I had a text:index
>> pointing to an index, when the query was executed, it would be delegated to
>> the DatasetGraphSpatial. This dataset knows nothing of text, and when the
>> TextQueryPF is executed, the textIndex is null. My logs have:
>>
>> [2018-12-23 21:03:57] TextQueryPF WARN Failed to find the text index :
>> tried context and as a text-enabled dataset
>> [2018-12-23 21:03:57] TextQueryPF WARN No text index - no text search
>> performed
>>
>> So alas I couldn't post a working example.
>>
>> Hope that helps anyway
>>
>> Bruno
>>
>> ps: Lucene locks its indexes, so I **guess** if you tried a different
>> approach of having another Fuseki pointing to the same directory, it would
>> probably raise an exception at some point
>>
>>
>>
>>
>> From: Jean-Marc Vanel
>> To: Jena users
>> Sent: Saturday, 22 December 2018 10:03 PM
>> Subject: Re: Build by program a Dataset that is both textual and spatial
>>
>>
>>
>> I reverted to trying configuration by code, with 2 more tries:
>>
>> https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/lucene/LuceneIndex.scala#L109
>> :
>> add to textual Dataset's Context a spatialIndex key to the spatialIndex.
>>
>> and line 135 :
>> Use same Lucene directory for both indexes (not forgetting to spatially
>> index the existing textual index).
>> WARN
>> org.apache.jena.query.spatial.pfunction.SpatialOperationWithCircleBase
>> - Failed to find the spatial index : tried context and as a
>> spatial-enabled dataset
>>
>> Both fai
Re: Build by program a Dataset that is both textual and spatial
Thanks Bruno for sharing your trials; alas you're not further than I.
Using gist.github.com is indeed better than the regular git, like I do, for
sharing trials.
I'm getting skeptical about doing spatial + textual as the Jena API stands.
For one thing, nobody stood up and said "I did it" since 10 days.
Second, spatial and textual indexing follow a similar design, but not
readily mixable:
- TextDatasetFactory 's create() method returns a DatasetGraphText
- SpatialDatasetFactory create() method returns a DatasetGraphSpatial
Both classes DatasetGraphText and DatasetGraphSpatial have their respective
index as members and extend DatasetGraphMonitor so that they get notified
of add or delete to the DatasetGraph.
Moreover, alas, the DatasetGraphMonitor class does not follow the observer
design pattern <https://en.wikipedia.org/wiki/Observer_pattern> , which
would allow to attach several observer objects to the same DatasetGraph.
The monitor in DatasetGraphMonitor class is interface DatasetChanges.
So the solution I can think of at the moment is creating a mixin
class DatasetGraphTextSpatial with delegation like this:
class DatasetGraphTextSpatial extends DatasetGraphWrapper {
DatasetGraphText datasetGraphText;
DatasetGraphSpatial datasetGraphSpatial;
public DatasetGraphTextSpatial(DatasetGraph dsg) {
super(dsg);
}
// delegating all methods to both members:
public void commit() {
datasetGraphText.commit();
datasetGraphSpatial.commit();
}
// etc (tedious) ...
}
Le dim. 23 déc. 2018 à 09:13, Bruno P. Kinoshita a
écrit :
> Hi,
>
> I had a crack at it, without luck. Here's my set up:
> https://gist.github.com/kinow/f391be7036b63f8ac0c08b9832e8bfa7
>
> I started with a Spatial only, using a configuration turtle file from Jena
> Spatial tests (see gist's spatial1.ttl). Loaded the dataset (also from
> tests), a query with spatial:nearby worked (geoarq-data-1.ttl &
> geoarq-data-1-query.ttl in the gist above).
>
> Then started by modifying the spatial Lucene Index, adding the properties
> necessary for a Jena Text dataset (see gist's spatial-text.ttl). Tried to
> make both, the spatial lucene index and the text lucene index point to the
> same directory.
>
>
> Once I started Fuseki (see comments in gist for rest of set up), the
> spatial query was still working, but even though I had a text:index
> pointing to an index, when the query was executed, it would be delegated to
> the DatasetGraphSpatial. This dataset knows nothing of text, and when the
> TextQueryPF is executed, the textIndex is null. My logs have:
>
> [2018-12-23 21:03:57] TextQueryPF WARN Failed to find the text index :
> tried context and as a text-enabled dataset
> [2018-12-23 21:03:57] TextQueryPF WARN No text index - no text search
> performed
>
> So alas I couldn't post a working example.
>
> Hope that helps anyway
>
> Bruno
>
> ps: Lucene locks its indexes, so I **guess** if you tried a different
> approach of having another Fuseki pointing to the same directory, it would
> probably raise an exception at some point
>
>
>
>
> From: Jean-Marc Vanel
> To: Jena users
> Sent: Saturday, 22 December 2018 10:03 PM
> Subject: Re: Build by program a Dataset that is both textual and spatial
>
>
>
> I reverted to trying configuration by code, with 2 more tries:
>
> https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/lucene/LuceneIndex.scala#L109
> :
> add to textual Dataset's Context a spatialIndex key to the spatialIndex.
>
> and line 135 :
> Use same Lucene directory for both indexes (not forgetting to spatially
> index the existing textual index).
> WARN
> org.apache.jena.query.spatial.pfunction.SpatialOperationWithCircleBase
> - Failed to find the spatial index : tried context and as a
> spatial-enabled dataset
>
> Both fail!
>
> My best hope is to look into Fuseki code,
> but can someone confirm that Fuseki does spatial WITH textual Indexing?
>
> I already looked into Fuseki code, but could not find calls to spatial or
> textual factory in jena-fuseki1/src/main/java/org/apache/jena/fuseki/ .
>
>
> Le ven. 21 déc. 2018 à 11:29, Jean-Marc Vanel a
> écrit :
>
> > I'm still clueless about how to configure Jena for textual AND spatial
> > indexes.
> > I tried yet another assembler file:
> >
> >
> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext3.assembler.ttl
> > but even less good as
> >
> >
> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext2.assembler.ttl
> > reported just before.
> >
> > It seems that nobody uses both indexes by program
Re: Retiring Jena modules
Greg , this looks nice, and when you need a tester , I can test . >From my point of view old and simple vocabulary have a great value in the Semantic Web, Especially, having atomic properties like oldgeo:long and lat , having well defined rdfs:domain and range are straight Semantic Web design, while datatype like geosparql force application to make a parsing. To be honest, old geo: vocabulary lacks rdfs:range declarations, but I added them in my app's database, as xsd:float . Le ven. 21 déc. 2018 à 15:45, Greg Albiston a écrit : > Hi Jean-Marc, > > The WGS84 Lat/Lon isn't part of the GeoSPARQL standard. > However, I'm looking into supporting them in the replacement module. > > For example: > > - converting WGS84 datasets to GeoSPARQL format. > - SPARQL function for conversion of WGS84 to WKT Point in query (making > the GeoSPARQL functions available for use). > - existing jena-spatial property functions for GeoSPARQL and WGS84. > > Any comments welcome. > > Thanks, > > Greg > > On 21/12/2018 10:56, Jean-Marc Vanel wrote: > > I hope it will not remove support for old and widespread vocabulary > > http://www.w3.org/2003/01/geo/wgs84_pos# > > > > https://github.com/galbiston/geosparql-jena does not seem to mention > that. > > > > > > Le ven. 21 déc. 2018 à 11:25, Andy Seaborne a écrit : > > > >> > >> On 21/12/2018 08:34, Nouwt, B. (Barry) wrote: > >>> Hi Andy, will the GeoSPARQL implementation be available before the > >> jena-spatial module is retired? > >> > >> The module is a contribution from Greg Albiston: > >> > >> > The project is now published on GitHub at > >> > https://github.com/galbiston/geosparql-jena and has been released > to > >> > Maven Central today. > >> > > >> > There is also a project which uses Fuseki's embedded server with the > >> > GeoSPARQL features at https://github.com/galbiston/geosparql-fuseki > . > >> > >> We're working with Greg to incorporate it in the Jena release (not the > >> next one, 3.10); it works now from Greg's repo / maven artifacts. > >> > >> Andy > >> > >>> Regards, Barry Nouwt > >>> > >>> -Original Message- > >>> From: Andy Seaborne > >>> Sent: zaterdag 15 december 2018 00:04 > >>> To: users@jena.apache.org > >>> Subject: Retiring Jena modules > >>> > >>> Hi Jena users, > >>> > >>> The project team is looking at retiring some lesser used modules. > >>> > >>> In the next release, 3.10.0, the modules > >>> > >>> jena-fuseki1 > >>> jena-csv > >>> > >>> will not be part of the release and there won't be maven artifacts. > >>> There haven't been any changes to these modules and the code will be > >> available from git history. > >>> Beyond that we are looking at the status of: > >>> > >>> jena-spatial - to be replaced by a GeoSPARQL implementation > >>> jena-sdb > >>> jena-maven-tools - command line schemagen is in jena-cmds. > >>> > >>> We're looking for feedback so please let us know if you use these > >> modules from a recent version of Jena. > >>>Andy > >>>on behalf of the Jena dev team. > >>> This message may contain information that is not intended for you. If > >> you are not the addressee or if this message was sent to you by mistake, > >> you are requested to inform the sender and delete the message. TNO > accepts > >> no liability for the content of this e-mail, for the manner in which you > >> use it and for damage of any kind resulting from the risks inherent to > the > >> electronic transmission of messages. > > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Retiring Jena modules
I hope it will not remove support for old and widespread vocabulary http://www.w3.org/2003/01/geo/wgs84_pos# https://github.com/galbiston/geosparql-jena does not seem to mention that. Le ven. 21 déc. 2018 à 11:25, Andy Seaborne a écrit : > > > On 21/12/2018 08:34, Nouwt, B. (Barry) wrote: > > Hi Andy, will the GeoSPARQL implementation be available before the > jena-spatial module is retired? > > The module is a contribution from Greg Albiston: > > > The project is now published on GitHub at > > https://github.com/galbiston/geosparql-jena and has been released to > > Maven Central today. > > > > There is also a project which uses Fuseki's embedded server with the > > GeoSPARQL features at https://github.com/galbiston/geosparql-fuseki. > > We're working with Greg to incorporate it in the Jena release (not the > next one, 3.10); it works now from Greg's repo / maven artifacts. > > Andy > > > > > Regards, Barry Nouwt > > > > -Original Message- > > From: Andy Seaborne > > Sent: zaterdag 15 december 2018 00:04 > > To: users@jena.apache.org > > Subject: Retiring Jena modules > > > > Hi Jena users, > > > > The project team is looking at retiring some lesser used modules. > > > > In the next release, 3.10.0, the modules > > > > jena-fuseki1 > > jena-csv > > > > will not be part of the release and there won't be maven artifacts. > > There haven't been any changes to these modules and the code will be > available from git history. > > > > Beyond that we are looking at the status of: > > > > jena-spatial - to be replaced by a GeoSPARQL implementation > > jena-sdb > > jena-maven-tools - command line schemagen is in jena-cmds. > > > > We're looking for feedback so please let us know if you use these > modules from a recent version of Jena. > > > > Andy > > on behalf of the Jena dev team. > > This message may contain information that is not intended for you. If > you are not the addressee or if this message was sent to you by mistake, > you are requested to inform the sender and delete the message. TNO accepts > no liability for the content of this e-mail, for the manner in which you > use it and for damage of any kind resulting from the risks inherent to the > electronic transmission of messages. > > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Build by program a Dataset that is both textual and spatial
I'm still clueless about how to configure Jena for textual AND spatial
indexes.
I tried yet another assembler file:
https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext3.assembler.ttl
but even less good as
https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext2.assembler.ttl
reported just before.
It seems that nobody uses both indexes by program;
otherwise I'd like to know how you do; you can send me privately your
assembler files.
Fuseki claims to "incorporate Jena text query
<https://jena.apache.org/documentation/query/text-query.html> and Jena
spatial query
<https://jena.apache.org/documentation/query/spatial-query.html>" in this
documentation:
https://jena.apache.org/documentation/fuseki2/
So there is a way to configure Jena for textual AND spatial indexes!
In absence of other help, I'll have to look into Fuseki code.
NOTE: the error message "*tried context and as a text-enabled dataset*"
maybe hints at a solution, but the word "context" is not clear.
Le lun. 17 déc. 2018 à 14:51, Jean-Marc Vanel a
écrit :
>
>
> Le lun. 17 déc. 2018 à 13:24, Andy Seaborne a écrit :
>
>> ...
>> :spatial_dataset rdf:type spatial:SpatialDataset ;
>> rdf:type text:TextDataset ;
>> ...
>>
>> Not sure but I think the system will create this twice.
>>
>> It would be better to have two declarations, one for spatial, one for
>> text. I'm not sure what happen if they share the same :directory - it
>> might work, it might not.
>>
>
> So, if one want to make queries that are both textual and spatial,
> having two declarations, one for spatial, one for text, sharing the same
> directory, appears as the only solution .
> I tested and report here.
> The current assembler file:
>
> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext2.assembler.ttl
> Results:
>
>- spatial query works, textual query has empty result, but no message
>in log, when *calling with spatial Dataset* in call
>to DatasetFactory.assemble()
>- spatial query works, textual query has result independant of the
>string searched, and messages in log "Failed to find the text index" , "No
>text index - no text search performed : tried context and as a text-enabled
>dataset", when *calling with textual Dataset *in call to
>DatasetFactory.assemble()
>
> The query tested for text:
> PREFIX text: <http://jena.apache.org/text#>
> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
> SELECT *
> WHERE {
> ?s a <http://dbpedia.org/ontology/Garden> ;
> text:query (rdfs:label 'Gardens' ) ;
> } LIMIT 11
>
> The spatial query tested:
>
> PREFIX spatial: <http://jena.apache.org/spatial#>
> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
> SELECT DISTINCT ?placeName ?place
> WHERE {
> ?place spatial:nearby (
> 45.75 4.85 # Lyon
> 1000 'km') .
> ?place rdfs:label ?placeName
> FILTER( LANG( ?placeName) = 'en' )
> } LIMIT 111
>
> Data: the 2600 dboGarden in dbPedia endpoint.
>
>
>
>> Andy
>>
>> On 16/12/2018 16:41, Jean-Marc Vanel wrote:
>> > Sorry , I sent a bad link on the list, my assembler file is really this:
>> >
>> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl
>> >
>> >
>> >
>> >
>> > Le dim. 16 déc. 2018 à 13:40, Marco Neumann a
>> > écrit :
>> >
>> >> you are missing the text index in the assembler.
>> >>
>> >> On Sun, Dec 16, 2018 at 12:08 PM Jean-Marc Vanel <
>> jeanmarc.va...@gmail.com
>> >>>
>> >> wrote:
>> >>
>> >>> Yes indeed,
>> >>> exactly with this assembler file:
>> >>>
>> >>>
>> >>
>> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial.assembler.ttl
>> >>>
>> >>> And, when the Jena based application is started with this same
>> assembler
>> >>> file, the spatial queries work , as said above in this thread.
>> >>>
>> >>>
>> >>> Le dim. 16 déc. 2018 à 12:18, Marco Neumann
>> a
>> >>> écrit :
>> >>>
>> >>>> did you create the text index with jena.textindexer
>> >>>> --desc=//config.ttl?
>> >>>>
>> >>>>
>> >>>> On Sun, Dec 16, 2018 at 8:48 AM Jean-Marc Vanel <
>> >>> jeanmarc.va...@gmail.com>
>> >>>> wro
Re: Build by program a Dataset that is both textual and spatial
Le lun. 17 déc. 2018 à 13:24, Andy Seaborne a écrit :
> ...
> :spatial_dataset rdf:type spatial:SpatialDataset ;
> rdf:type text:TextDataset ;
> ...
>
> Not sure but I think the system will create this twice.
>
> It would be better to have two declarations, one for spatial, one for
> text. I'm not sure what happen if they share the same :directory - it
> might work, it might not.
>
So, if one want to make queries that are both textual and spatial,
having two declarations, one for spatial, one for text, sharing the same
directory, appears as the only solution .
I tested and report here.
The current assembler file:
https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext2.assembler.ttl
Results:
- spatial query works, textual query has empty result, but no message
in log, when *calling with spatial Dataset* in call
to DatasetFactory.assemble()
- spatial query works, textual query has result independant of the
string searched, and messages in log "Failed to find the text index" , "No
text index - no text search performed : tried context and as a text-enabled
dataset", when *calling with textual Dataset *in call to
DatasetFactory.assemble()
The query tested for text:
PREFIX text: <http://jena.apache.org/text#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT *
WHERE {
?s a <http://dbpedia.org/ontology/Garden> ;
text:query (rdfs:label 'Gardens' ) ;
} LIMIT 11
The spatial query tested:
PREFIX spatial: <http://jena.apache.org/spatial#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?placeName ?place
WHERE {
?place spatial:nearby (
45.75 4.85 # Lyon
1000 'km') .
?place rdfs:label ?placeName
FILTER( LANG( ?placeName) = 'en' )
} LIMIT 111
Data: the 2600 dboGarden in dbPedia endpoint.
> Andy
>
> On 16/12/2018 16:41, Jean-Marc Vanel wrote:
> > Sorry , I sent a bad link on the list, my assembler file is really this:
> >
> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl
> >
> >
> >
> >
> > Le dim. 16 déc. 2018 à 13:40, Marco Neumann a
> > écrit :
> >
> >> you are missing the text index in the assembler.
> >>
> >> On Sun, Dec 16, 2018 at 12:08 PM Jean-Marc Vanel <
> jeanmarc.va...@gmail.com
> >>>
> >> wrote:
> >>
> >>> Yes indeed,
> >>> exactly with this assembler file:
> >>>
> >>>
> >>
> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial.assembler.ttl
> >>>
> >>> And, when the Jena based application is started with this same
> assembler
> >>> file, the spatial queries work , as said above in this thread.
> >>>
> >>>
> >>> Le dim. 16 déc. 2018 à 12:18, Marco Neumann
> a
> >>> écrit :
> >>>
> >>>> did you create the text index with jena.textindexer
> >>>> --desc=//config.ttl?
> >>>>
> >>>>
> >>>> On Sun, Dec 16, 2018 at 8:48 AM Jean-Marc Vanel <
> >>> jeanmarc.va...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> I fixed a stupid error in text index URI, resulting from pasting, in
> >>> new
> >>>>> file jena.spatial+text.assembler.ttl:
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl
> >>>>>
> >>>>> Now, it reads:
> >>>>> :spatial_dataset rdf:type spatial:SpatialDataset ;
> >>>>> rdf:type text:TextDataset ;
> >>>>> spatial:dataset <#dataset> ;
> >>>>> spatial:index <#indexLucene> ;
> >>>>> text:dataset <#dataset> ;
> >>>>> text:index* <#indexLuceneText>* ;
> >>>>> .
> >>>>>
> >>>>> But still it says "Failed to find the text index" .
> >>>>>
> >>>>>
> >>>>> Le dim. 16 déc. 2018 à 09:34, Jean-Marc Vanel <
> >>> jeanmarc.va...@gmail.com>
> >>>> a
> >>>>> écrit :
> >>>>>
> >>>>>> So I tried a new file jena.spatial+text.assembler.ttl with
> >> separate
> >>>>>> Lucene indices for spatial and text:
> >>>>>>
> >>>>>>
> >>>>>
> >>
Re: Build by program a Dataset that is both textual and spatial
Sorry , I sent a bad link on the list, my assembler file is really this: https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl Le dim. 16 déc. 2018 à 13:40, Marco Neumann a écrit : > you are missing the text index in the assembler. > > On Sun, Dec 16, 2018 at 12:08 PM Jean-Marc Vanel > > wrote: > > > Yes indeed, > > exactly with this assembler file: > > > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial.assembler.ttl > > > > And, when the Jena based application is started with this same assembler > > file, the spatial queries work , as said above in this thread. > > > > > > Le dim. 16 déc. 2018 à 12:18, Marco Neumann a > > écrit : > > > > > did you create the text index with jena.textindexer > > > --desc=//config.ttl? > > > > > > > > > On Sun, Dec 16, 2018 at 8:48 AM Jean-Marc Vanel < > > jeanmarc.va...@gmail.com> > > > wrote: > > > > > > > I fixed a stupid error in text index URI, resulting from pasting, in > > new > > > > file jena.spatial+text.assembler.ttl: > > > > > > > > > > > > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl > > > > > > > > Now, it reads: > > > > :spatial_dataset rdf:type spatial:SpatialDataset ; > > > > rdf:type text:TextDataset ; > > > > spatial:dataset <#dataset> ; > > > > spatial:index <#indexLucene> ; > > > > text:dataset <#dataset> ; > > > > text:index* <#indexLuceneText>* ; > > > > . > > > > > > > > But still it says "Failed to find the text index" . > > > > > > > > > > > > Le dim. 16 déc. 2018 à 09:34, Jean-Marc Vanel < > > jeanmarc.va...@gmail.com> > > > a > > > > écrit : > > > > > > > > > So I tried a new file jena.spatial+text.assembler.ttl with > separate > > > > > Lucene indices for spatial and text: > > > > > > > > > > > > > > > > > > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl > > > > > > > > > > It defines a Dataset with both Dataset types and both Lucene > indices: > > > > > > > > > > :spatial_dataset rdf:type spatial:SpatialDataset ; > > > > > rdf:type text:TextDataset ; > > > > > spatial:dataset <#dataset> ; > > > > > spatial:index <#indexLucene> ; > > > > > text:dataset <#dataset> ; > > > > > text:index <#indexLucene> ; > > > > > . > > > > > > > > > > Alas , when querying , it says "Failed to find the text index" : > > > > > > > > > > WARN org.apache.jena.query.text.TextQueryPF - Failed to find the > > text > > > > > index : tried context and as a text-enabled dataset > > > > > WARN org.apache.jena.query.text.TextQueryPF - No text index - no > > text > > > > > search performed > > > > > > > > > > Is it because rdf:type spatial:SpatialDataset is asserted first in > > > > > assembler file? > > > > > So, defining a hybrid Dataset does not work anymore by TTL > > > specification > > > > > than by JVM code specification. > > > > > I definitely need more experts' advice . > > > > > > > > > > > > > > > > > > > > Le sam. 15 déc. 2018 à 21:10, ajs6f a écrit : > > > > > > > > > >> > On Dec 15, 2018, at 9:59 AM, Marco Neumann < > > marco.neum...@gmail.com > > > > > > > > >> wrote: > > > > >> > > > > > >> >> *Question*: does that make sense to have a unique Lucene index > > for > > > > >> text and > > > > >> >> space ? > > > > >> >> It is possible at all? If yes, is it good in terms of disk > space > > > and > > > > >> >> performance? > > > > >> >> Like this: > > > > >> >> <#indexLucene> a text:TextIndexLucene ; > > > > >> >> a spatial:SpatialIndexLucene ; > >
Re: Build by program a Dataset that is both textual and spatial
Yes indeed, exactly with this assembler file: https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial.assembler.ttl And, when the Jena based application is started with this same assembler file, the spatial queries work , as said above in this thread. Le dim. 16 déc. 2018 à 12:18, Marco Neumann a écrit : > did you create the text index with jena.textindexer > --desc=//config.ttl? > > > On Sun, Dec 16, 2018 at 8:48 AM Jean-Marc Vanel > wrote: > > > I fixed a stupid error in text index URI, resulting from pasting, in new > > file jena.spatial+text.assembler.ttl: > > > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl > > > > Now, it reads: > > :spatial_dataset rdf:type spatial:SpatialDataset ; > > rdf:type text:TextDataset ; > > spatial:dataset <#dataset> ; > > spatial:index <#indexLucene> ; > > text:dataset <#dataset> ; > > text:index* <#indexLuceneText>* ; > > . > > > > But still it says "Failed to find the text index" . > > > > > > Le dim. 16 déc. 2018 à 09:34, Jean-Marc Vanel > a > > écrit : > > > > > So I tried a new file jena.spatial+text.assembler.ttl with separate > > > Lucene indices for spatial and text: > > > > > > > > > https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl > > > > > > It defines a Dataset with both Dataset types and both Lucene indices: > > > > > > :spatial_dataset rdf:type spatial:SpatialDataset ; > > > rdf:type text:TextDataset ; > > > spatial:dataset <#dataset> ; > > > spatial:index <#indexLucene> ; > > > text:dataset <#dataset> ; > > > text:index <#indexLucene> ; > > > . > > > > > > Alas , when querying , it says "Failed to find the text index" : > > > > > > WARN org.apache.jena.query.text.TextQueryPF - Failed to find the text > > > index : tried context and as a text-enabled dataset > > > WARN org.apache.jena.query.text.TextQueryPF - No text index - no text > > > search performed > > > > > > Is it because rdf:type spatial:SpatialDataset is asserted first in > > > assembler file? > > > So, defining a hybrid Dataset does not work anymore by TTL > specification > > > than by JVM code specification. > > > I definitely need more experts' advice . > > > > > > > > > > > > Le sam. 15 déc. 2018 à 21:10, ajs6f a écrit : > > > > > >> > On Dec 15, 2018, at 9:59 AM, Marco Neumann > > > >> wrote: > > >> > > > >> >> *Question*: does that make sense to have a unique Lucene index for > > >> text and > > >> >> space ? > > >> >> It is possible at all? If yes, is it good in terms of disk space > and > > >> >> performance? > > >> >> Like this: > > >> >> <#indexLucene> a text:TextIndexLucene ; > > >> >> a spatial:SpatialIndexLucene ; > > >> >> # etc ... > > >> > > >> I'm sure that would be ideal for many cases, but I'm not at all sure > > that > > >> the same index can answer queries of both kinds. Perhaps we can > combine > > >> fields from both, but are the relationships between tuple and index > > record > > >> the same in both cases? > > >> > > >> Otherwise, I know we had a conversation at some point in the past on > one > > >> of the lists about trying to factor out commonalities between > > jena-spatial > > >> and jena-text, but it didn't go very far at that time and I don't know > > what > > >> the intervening years have done to make it more or less feasible. > There > > >> have been many changes to jena-text in that time and the new spatial > > module > > >> is a whole new story. I'd put a link here but searching > > lists.apache.org > > >> hasn't brought it up for me. > > >> > > >> ajs6f > > >> > > >> > > > > > > -- > > > Jean-Marc Vanel > > > > > > > > > http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject > > > < > > > http://www.semantic-forms.cc:9111/display?displayuri=http://j
Re: sparql 1.4 billion triples
yphu, you didn't share your query. Maybe the query have questionable features. Did you try a simple but useful query, like getting the first 10 foaf:Person ? Le ven. 7 déc. 2018 à 15:53, y...@zju.edu.cn a écrit : > Dear jena, > I have built a graph with 1.4 billion triples and store it as a data set > in TDB through Fuseki upload system. > Now, I try to make some sparql search, the speed is very slow. > > For example, when I make the sqarql in Fuseki in the following, it takes > 50 seconds. > How can I improve the speed? > -- > Best wishes! > > > 胡云苹 > 浙江大学控制科学与工程学院 > 浙江省杭州市浙大路38号浙大玉泉校区CSC研究所 > Institute of Cyber-Systems and Control, College of Control Science and > Engineering, Zhejiang University, Hangzhou 310027,P.R.China > Email : y...@zju.edu.cn ;hyphy...@163.com > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Build by program a Dataset that is both textual and spatial
I fixed a stupid error in text index URI, resulting from pasting, in new file jena.spatial+text.assembler.ttl: https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl Now, it reads: :spatial_dataset rdf:type spatial:SpatialDataset ; rdf:type text:TextDataset ; spatial:dataset <#dataset> ; spatial:index <#indexLucene> ; text:dataset <#dataset> ; text:index* <#indexLuceneText>* ; . But still it says "Failed to find the text index" . Le dim. 16 déc. 2018 à 09:34, Jean-Marc Vanel a écrit : > So I tried a new file jena.spatial+text.assembler.ttl with separate > Lucene indices for spatial and text: > > https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl > > It defines a Dataset with both Dataset types and both Lucene indices: > > :spatial_dataset rdf:type spatial:SpatialDataset ; > rdf:type text:TextDataset ; > spatial:dataset <#dataset> ; > spatial:index <#indexLucene> ; > text:dataset <#dataset> ; > text:index <#indexLucene> ; > . > > Alas , when querying , it says "Failed to find the text index" : > > WARN org.apache.jena.query.text.TextQueryPF - Failed to find the text > index : tried context and as a text-enabled dataset > WARN org.apache.jena.query.text.TextQueryPF - No text index - no text > search performed > > Is it because rdf:type spatial:SpatialDataset is asserted first in > assembler file? > So, defining a hybrid Dataset does not work anymore by TTL specification > than by JVM code specification. > I definitely need more experts' advice . > > > > Le sam. 15 déc. 2018 à 21:10, ajs6f a écrit : > >> > On Dec 15, 2018, at 9:59 AM, Marco Neumann >> wrote: >> > >> >> *Question*: does that make sense to have a unique Lucene index for >> text and >> >> space ? >> >> It is possible at all? If yes, is it good in terms of disk space and >> >> performance? >> >> Like this: >> >> <#indexLucene> a text:TextIndexLucene ; >> >> a spatial:SpatialIndexLucene ; >> >> # etc ... >> >> I'm sure that would be ideal for many cases, but I'm not at all sure that >> the same index can answer queries of both kinds. Perhaps we can combine >> fields from both, but are the relationships between tuple and index record >> the same in both cases? >> >> Otherwise, I know we had a conversation at some point in the past on one >> of the lists about trying to factor out commonalities between jena-spatial >> and jena-text, but it didn't go very far at that time and I don't know what >> the intervening years have done to make it more or less feasible. There >> have been many changes to jena-text in that time and the new spatial module >> is a whole new story. I'd put a link here but searching lists.apache.org >> hasn't brought it up for me. >> >> ajs6f >> >> > > -- > Jean-Marc Vanel > > http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject > <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> > Rule-based programming, Semantic Web > +33 (0)6 89 16 29 52 > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > Chroniques jardin > <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle> > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Build by program a Dataset that is both textual and spatial
So I tried a new file jena.spatial+text.assembler.ttl with separate Lucene indices for spatial and text: https://github.com/jmvanel/semantic_forms/blob/master/scala/jena.spatial%2Btext.assembler.ttl It defines a Dataset with both Dataset types and both Lucene indices: :spatial_dataset rdf:type spatial:SpatialDataset ; rdf:type text:TextDataset ; spatial:dataset <#dataset> ; spatial:index <#indexLucene> ; text:dataset <#dataset> ; text:index <#indexLucene> ; . Alas , when querying , it says "Failed to find the text index" : WARN org.apache.jena.query.text.TextQueryPF - Failed to find the text index : tried context and as a text-enabled dataset WARN org.apache.jena.query.text.TextQueryPF - No text index - no text search performed Is it because rdf:type spatial:SpatialDataset is asserted first in assembler file? So, defining a hybrid Dataset does not work anymore by TTL specification than by JVM code specification. I definitely need more experts' advice . Le sam. 15 déc. 2018 à 21:10, ajs6f a écrit : > > On Dec 15, 2018, at 9:59 AM, Marco Neumann > wrote: > > > >> *Question*: does that make sense to have a unique Lucene index for text > and > >> space ? > >> It is possible at all? If yes, is it good in terms of disk space and > >> performance? > >> Like this: > >> <#indexLucene> a text:TextIndexLucene ; > >> a spatial:SpatialIndexLucene ; > >> # etc ... > > I'm sure that would be ideal for many cases, but I'm not at all sure that > the same index can answer queries of both kinds. Perhaps we can combine > fields from both, but are the relationships between tuple and index record > the same in both cases? > > Otherwise, I know we had a conversation at some point in the past on one > of the lists about trying to factor out commonalities between jena-spatial > and jena-text, but it didn't go very far at that time and I don't know what > the intervening years have done to make it more or less feasible. There > have been many changes to jena-text in that time and the new spatial module > is a whole new story. I'd put a link here but searching lists.apache.org > hasn't brought it up for me. > > ajs6f > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Build by program a Dataset that is both textual and spatial
I began to try using what I sketched in the preceding post: https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/lucene/LuceneIndex.scala#L88 But it does not work: that is, a spatial query returns nothing. To create the spatial index by program jena.spatialindexer , it is necessary to have an assembler file. I created one, following closely the model in the documentation: https://jena.apache.org/documentation/query/spatial-query.html#spatial-dataset-assembler Then a spatial query works. But of course, no textual query works. So I'll try to create an assembler file by mixing the examples for spatial above and textual from the doc. here: https://jena.apache.org/documentation/query/text-query.html#text-dataset-assembler *Question*: does that make sense to have a unique Lucene index for text and space ? It is possible at all? If yes, is it good in terms of disk space and performance? Like this: <#indexLucene> a text:TextIndexLucene ; a spatial:SpatialIndexLucene ; # etc ... Le jeu. 13 déc. 2018 à 11:08, Jean-Marc Vanel a écrit : > Hi all > > After looking the documentation, > > https://jena.apache.org/documentation/query/spatial-query.html#building-a-spatial-index > it is not clear which is the best way to build by program a Dataset that > is both textual and spatial. > > Both functions take an argument of type Dataset and return an "augmented" > Dataset: > TextDatasetFactory.create() > SpatialDatasetFactory.createLucene() > > So, in theory, I can create first the textual Dataset, and then from that > the spatial Dataset, or the reverse. > > Is that the way to go? > Or is there a way to create in one step a combined Dataset ? > Is there a recommended order of creating the "augmented" Datasets, > depending on data and usage ? > Is there some loss of efficiency compared to using textual or spatial > alone ? > > Happy Jenaing from Jean (Marc) ! > -- > Jean-Marc Vanel > > http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject > <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> > Rule-based programming, Semantic Web > +33 (0)6 89 16 29 52 > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > Chroniques jardin > <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle> > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Build by program a Dataset that is both textual and spatial
Hi all After looking the documentation, https://jena.apache.org/documentation/query/spatial-query.html#building-a-spatial-index it is not clear which is the best way to build by program a Dataset that is both textual and spatial. Both functions take an argument of type Dataset and return an "augmented" Dataset: TextDatasetFactory.create() SpatialDatasetFactory.createLucene() So, in theory, I can create first the textual Dataset, and then from that the spatial Dataset, or the reverse. Is that the way to go? Or is there a way to create in one step a combined Dataset ? Is there a recommended order of creating the "augmented" Datasets, depending on data and usage ? Is there some loss of efficiency compared to using textual or spatial alone ? Happy Jenaing from Jean (Marc) ! -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: Re-Using the GEO Namespace Prefix
Many dbPedia data use geo: namespace: http://www.w3.org/2003/01/geo/wgs84_pos# For instance the 2600 <http://dbpedia.org/ontology/Garden> . As for tools, my reusable visualizer in a static HTML page detects geo: coordinates: Example view with LeafLet and gardens from a SPARQL query <http://semantic-forms.cc:1952/assets/geo-map/geo-map.html?link-prefix=http://semantic-forms.cc:1952/display?displayuri==fr=http://semantic-forms.cc:1952/sparql?query=%0APREFIX+text%3A+%3Chttp%3A%2F%2Fjena.apache.org%2Ftext%23%3E+%0APREFIX+rdfs%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E+%0APREFIX+form%3A+%3Chttp%3A%2F%2Fraw.githubusercontent.com%2Fjmvanel%2Fsemantic_forms%2Fmaster%2Fvocabulary%2Fforms.owl.ttl%23%3E+%0APREFIX+geo%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23%3E+%0APREFIX+foaf%3A+%3Chttp%3A%2F%2Fxmlns.com%2Ffoaf%2F0.1%2F%3E+%0ACONSTRUCT+%7B%0A++%3Fthing+geo%3Along+%3FLONG+.%0A++%3Fthing+geo%3Alat+%3FLAT+.%0A++%3Fthing+rdfs%3Alabel+%3FLAB+.%0A++%3Fthing+foaf%3Adepiction+%3FIMG+.%0A%7D+WHERE+%7B%0A++%0A++%0A+graph+%3Fg1+%7B%0A+++%3Fthing+a+%3Chttp%3A%2F%2Fdbpedia.org%2Fontology%2FGarden%3E+.%0A+%7D+.%0A++graph+%3Fgrll+%7B%0A%3Fthing+geo%3Along+%3FLONG+.%0A%3Fthing+geo%3Alat+%3FLAT+.%0A++%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fgrlab+%7B%0A%3Fthing+rdfs%3Alabel+%3FLAB+%7D+%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fgrlab2+%7B%0A%3Fthing+%3Curn%3AdisplayLabel%3E+%3FLAB+%7D+%7D%0A%0A++OPTIONAL+%7B%0A+++graph+%3Fg3+%7B%0A%3Fthing+foaf%3Adepiction+%3FIMG+%7D+%7D%0A++OPTIONAL+%7B%0A+++graph+%3Fg4+%7B%0A%3Fthing+foaf%3Aimg+%3FIMG+%7D+%7D%0A%0AOPTIONAL+%7B%0A+++graph+%3FgrCount+%7B%0A%3Fthing+form%3AlinksCount+%3FCOUNT.%0A++%7D+%7D%0A%7D%0AORDER+BY+DESC%28%3FCOUNT%29%0A> . Also the pages and forms in semantic_forms that include geo: coordinates are populated with data coming from GPS: Example, create a botanic observation: https://semantic-forms.cc:1953/create?uri=http%3A%2F%2Fdeductions.github.io%2Fnature_observation.owl.ttl%23Observation Le mer. 12 déc. 2018 à 14:33, Andy Seaborne a écrit : > Hi Marco, > > Where is' geo' being used? > > Andy > > > On 12/12/2018 12:16, Marco Neumann wrote: > > What do people on the list think about re-using the namespace prefix geo? > > > > it's currently used for > > > > WGS84 Geo Positioning: an RDF vocabulary > > http://www.w3.org/2003/01/geo/wgs84_pos# > > > > but becomes more popular (e.g. see wikidata) as a namespace prefix for > > > > OGC GeoSPARQL 1.0 > > http://www.opengis.net/ont/geosparql# > > > > I would have preferred to see the OGC introduce a new namespace prefix > like > > geosparql but for some reason they settled on replacing geo (W3C ) with > > geosparql (OGC) and geof for functions. Technically not a big deal but > that > > said most people would have to change their existing data to refer to the > > new URI now to accommodate the change. > > > > Alternatively we could just introduce geosparql or ogcgeo as a new > > namespace prefix in jena projects for OGC gesparql to avoid confusion. I > am > > sure the re-purposing of the geo namespace prefix was a "mistake" at the > > time of OGC geosparql (Ed. Matthew Perry and John Herring Oracle) > > discussions. No need to perpetuate this change into the future here. > > > > What are your thoughts on this? > > > > > > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Data corruption problem with TDB
I draw yous attention on my message in Jira: https://issues.apache.org/jira/browse/JENA-1553 -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Lucene dependency could be upgraded
Jena text 3.9.0 is at Lucene 6.4.1 , while the last Lucene is 7.5.0 : https://mvnrepository.com/artifact/org.apache.jena/jena-text/3.9.0 Trying to upgrade brings ClassNotFoundException: org.apache.lucene.queryparser.analyzing.AnalyzingQueryParser This happens in class TextIndexLucene. -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui Chroniques jardin <http://semantic-forms.cc:1952/backlinks?q=http%3A%2F%2Fdbpedia.org%2Fresource%2FChronicle>
Re: How we store date after shutdowning fuseki server
I would use a TDB baked Fuseki.
The code to do this is not more complex than what you use.
First, the data can resist to a power shudown.
Second, it is easy to obtain a dump, by the utility program
tdb.tdbdump
or by an HTTP service that triggers the dump without having to stop the
server.
2018-07-04 16:04 GMT+02:00 jena melina :
> hello ,
>
> I'm using fuseki server (in-memory ) to manipulate RDF data
> And i don't use any TDB backend. My question is how to save modification
> that i do using sparql update ?
>
> 1/ if I need to write the data back to disk , how can 'i do it ?
> 2/ if i need to connect my Fuseki to a TDB how can 'i do it ?
>
> To upload tata into fuseki server i use :
>
> public static void uploadRDF(String rdf, String serviceURI) {
>
> Model m = ModelFactory.createDefaultModel();
> m.read(Insert.class.getResource("/" + rdf).getFile(), "RDF/XML");
> // upload the resulting model
> DatasetAccessor accessor = DatasetAccessorFactory.
> createHTTP(serviceURI);
> accessor.putModel(m);
>
>
> }
>
> and to query my Data i use :
>
> public static ResultSet execSelectAndPrintquery(String serviceURI,
> String query) {
> QueryExecution q = QueryExecutionFactory.sparqlService(serviceURI,
> Constants.SPARQL_PREFIX+query);
>
> ResultSet results = q.execSelect();
> ResultSetFormatter.out(System.out, results);
> return results;
> }
>
> and to update data i use :
>
> public static void performUpdate (String updateCommand,String
> datasetUrl) {
> String id = UUID.randomUUID().toString();
> System.out.println(String.format("Adding %s", id));
> UpdateProcessor upp =
> UpdateExecutionFactory.createRemote(UpdateFactory.create(String.format(
> EngineConstants.SPARQL_PREFIX+updateCommand
> , id)),datasetUrl );
> upp.execute();
> }
>
>
> And i need to save all modifications , so ho can i do it ?
>
> thanks for your help ..
>
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: [ANN] Apache Jena 3.8.0
Thanks for the release ! I tried it with semantic_forms : all tests pass ! Note, this implies that TDB file format did not change. I'l test TDB2 again and let you know. -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http:// jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: [ANN] Apache Jena 3.8.0
It is most probably as specified by HTTP protocol : https://en.wikipedia.org/wiki/HTTP_compression#Compression_scheme_negotiation so, something that you can request for every query. 2018-07-02 13:07 GMT+02:00 Laura Morales : > > JENA-1554 > > > > Support for bz2 compressed inputs and outputs > > Is this transparent to the user, or is something that I can request for > every query? > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Retrieving SPARQL-results xml using curl
Here is a SPARQL query using wget :
wget --method=POST \
--body-data="query=SELECT * WHERE {?S ?P ?O} LIMIT 10" \
--header='Accept:application/sparql-results+xml' \
http://localhost:9000/sparql2
Other useful scripts in
https://github.com/jmvanel/semantic_forms/tree/master/scala/forms_play/dist/scripts
2018-06-28 12:24 GMT+02:00 Brice Sommacal :
> Hello,
>
> I use the last distribution of Fuseki which is initialized with a TDB2
> dataset [1].
> Working with the user interface, everything os working well. I mean I am
> able to retrieve results from a select query and then download it.
>
> Going one step further, I would like to automate dataset update and dataset
> queries restitution.
> I am able to run SPARQL Update query within culr (using SPARQL INSERT
> inside a ttl file).
>
> I am also able to get SPARQL Result JSON objects using commands line like
> the following :
> curl
> http://localhost:3030/Metro/query?query=PREFIX+rdf%3A+%
> 3Chttp%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22[..
> .]
>
> However, if I specify the header to return XML result set, it keeps
> outputting the data in a JSON format.
>
> Here is my command : curl
> http://localhost:3030/Metro/query?query=PREFIX+rdf%3A+%
> 3Chttp%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22[..
> .] -X POST -H 'Accept:+application/sparql-results+xml'
>
> I don't know what could be wrong. I am not an expert using curl... But
> syntax that I have seen on forums looks like the same...
>
> > Do you have an idea of what I am doing wrong?
>
>
> Regards,
>
>
> Brice
>
>
> [1] there is a huge progress between TDB1 and TDB2 (also fuseki 1 and 2).
> Thanks a lot to the community to make it available.
>
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: RIOT filter to remove bad triples
Hi Laura, I would do this: - consider the .nt as a ttl file - run riot with a base URI - remove the bad lines with grep -v Example: cat relative.ttl "v" . java -$CP riotcmd.riot --base=http://nogood.net/ /home/jmv/data/TESTS/relative.ttl <http://nogood.net/rel> <http://nogood.net/p> "v" . 2018-04-27 13:10 GMT+02:00 Laura Morales <laure...@mail.com>: > Here's the problem: I'm downloading a few .nt from the Internet and they > contain some bad triples. "Bad" means that they are syntactically correct > but logically incorrect; for example they contain triples with relative > URLs instead of absolute, or maybe they contain an empty subject like this > <>. I don't know if it's possible to ask RIOT to treat these as Warnings > instead of Errors, but if I use RIOT --validate it only reports then as > errors. > Before you say anything, I know that the original sources should be fixed > and that it's not RIOT's job to fix any errors. But I don't really have > control over those graphs, so I was wondering if it's possible to use RIOT > as some sort of filter like this > > 1. read next triple > 2. if logically correct, append to output >if logically incorrect, do not append to output > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Fuseki service vs. webapp
OK , I'll added LICENSE & NOTICE files. Regarding advertising Jena-based software, some projets have a page or paragraph called "Powered by Jena" , Does it exist ? It would add to the visibility and fame of the Jena ecosystem, and help finding tools. 2018-04-26 17:57 GMT+02:00 Andy Seaborne <a...@apache.org>: > Jean-Marc, > > From my own POV, I don't have much problem with advertising Jena-based > software here but please do note for people that it is not under the Apache > License that people might assume as it is on an Apache mail list. > semantic_forms is GPL v3. > > Maybe add a LICENSE file and a NOTICE file to the repository and not rely > on README. > > Andy > > On 26/04/18 13:56, Jean-Marc Vanel wrote: > >> Hi Sorin, >> >> you might want to try semantic_forms, >> another SPARQL server based on Jena TDB. >> This one is developed in Scala, on top of PlayFramework, >> itself on top of Netty 4.0.41.Final . >> Semantic_forms is also a generic semantic web navigation and annotation >> application, and a framework to build business applications centered on >> forms (input forms or read only). >> >> A pre-release is here : >> https://github.com/jmvanel/semantic_forms/releases/tag/2.5.0 >> >> You can test the zip, >> and/or the sites: >> http://semantic-forms.cc:9112 : general site >> http://semantic-forms.cc:9111 : sandbox >> >> *NOTES:* >> >> - *currently , a Jena assembly file is not taken in account, but I can >> add that quickly if you ask* >> - *instead, a single TDB directory , named "TDB" is taken in account* >> >> >> >> >> 2018-04-26 13:55 GMT+02:00 Sorin Gheorghiu <sorin.gheorghiu@uni-konstanz. >> de> >> : >> >> Hi, >>> >>> Let's suppose Fuseki 3.6.0 is started as a service (and specific plugins >>> like reindex, percolator, mustache, netty3 & 4 are loaded). Then a script >>> which reads intensively data over SPARQL fails after the first ~400 >>> SELECT's with connection refused error (Net::HTTP::Persistent::Error). >>> It >>> seems due to too many client connections in use fuseki is not capable to >>> open new ones. >>> >>> If the log4j is set to DEBUG (instead INFO or WARN) the script won't >>> crash, but fuseki will generate a huge amount of debug data which >>> produces >>> a process delay which helps it to close the old connections. Of course, >>> this is not a reliable workaround on a production server. >>> >>> Alternatively, if fuseki is started as a Webapp (for the same assembler >>> file), then the same script runs (more than 8 SELECT's within 1 hour) >>> without failure. In conclusion, fuseki as a Web Application handles >>> better >>> the client connections. >>> >>> Regards, >>> Sorin >>> >>> >>> >>> >> >> -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http:/ /jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Fuseki service vs. webapp
Hi Sorin, you might want to try semantic_forms, another SPARQL server based on Jena TDB. This one is developed in Scala, on top of PlayFramework, itself on top of Netty 4.0.41.Final . Semantic_forms is also a generic semantic web navigation and annotation application, and a framework to build business applications centered on forms (input forms or read only). A pre-release is here : https://github.com/jmvanel/semantic_forms/releases/tag/2.5.0 You can test the zip, and/or the sites: http://semantic-forms.cc:9112 : general site http://semantic-forms.cc:9111 : sandbox *NOTES:* - *currently , a Jena assembly file is not taken in account, but I can add that quickly if you ask* - *instead, a single TDB directory , named "TDB" is taken in account* 2018-04-26 13:55 GMT+02:00 Sorin Gheorghiu <sorin.gheorg...@uni-konstanz.de> : > Hi, > > Let's suppose Fuseki 3.6.0 is started as a service (and specific plugins > like reindex, percolator, mustache, netty3 & 4 are loaded). Then a script > which reads intensively data over SPARQL fails after the first ~400 > SELECT's with connection refused error (Net::HTTP::Persistent::Error). It > seems due to too many client connections in use fuseki is not capable to > open new ones. > > If the log4j is set to DEBUG (instead INFO or WARN) the script won't > crash, but fuseki will generate a huge amount of debug data which produces > a process delay which helps it to close the old connections. Of course, > this is not a reliable workaround on a production server. > > Alternatively, if fuseki is started as a Webapp (for the same assembler > file), then the same script runs (more than 8 SELECT's within 1 hour) > without failure. In conclusion, fuseki as a Web Application handles better > the client connections. > > Regards, > Sorin > > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Apache Jena installation issues with JENAROOT (on Mac)
nmental variable settings: > > > > > > $ echo $JAVA_HOME > > > /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home > > > > > > $JENA_HOME > > > /Users/username/Documents/jena/apachejena370 > > > > > > echo $JENAROOT > > > /Users/username/Documents/jena/apachejena370 > > > > > > $ echo $CLASSPATH > > > /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/ > > Home/bin:/Users/username/Documents/jena/apachejena370/lib/* > > > > > > When I run the code: > > > > > > $ sudo java Tutorial01 > > > > > > Exception in thread "main" java.lang.NoClassDefFoundError: > > > org/apache/jena/rdf/model/ModelFactory > > > at Tutorial01.main(Tutorial01.java:34) > > > Caused by: java.lang.ClassNotFoundException: > > > org.apache.jena.rdf.model.ModelFactory > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > > ... 1 more > > > > > > I've also somehow broken something because javac is no longer working: > $ > > > sudo javac Tutorial01.java > > > > > > Tutorial01.java:21: error: package org.apache.jena.rdf.model does not > > exist > > > import org.apache.jena.rdf.model.*; > > > ^ > > > Tutorial01.java:22: error: package org.apache.jena.vocabulary does not > > exist > > > import org.apache.jena.vocabulary.*; > > > ^ > > > Tutorial01.java:34: error: cannot find symbol > > > Model model = ModelFactory.createDefaultModel(); > > > ^ > > > symbol: class Model > > > location: class Tutorial01 > > > Tutorial01.java:34: error: cannot find symbol > > > Model model = ModelFactory.createDefaultModel(); > > > ^ > > > symbol: variable ModelFactory > > > location: class Tutorial01 > > > Tutorial01.java:37: error: cannot find symbol > > >Resource johnSmith = model.createResource(personURI); > > >^ > > > symbol: class Resource > > > location: class Tutorial01 > > > Tutorial01.java:40: error: cannot find symbol > > > johnSmith.addProperty(VCARD.FN, fullName); > > > ^ > > > symbol: variable VCARD > > > location: class Tutorial01 > > > 6 errors > > > > > > > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Towards Jena 3.7.0
Latest Jena source works fine with semantic_forms <https://github.com/jmvanel/semantic_forms/wiki> application (uses Jena TDB API with Lucene connection ). According to latest commit, the Jena release is almost there : *commit b850a9cd118db450becb1568f52a3e2617833e61 (HEAD -> master, origin/master, origin/HEAD)* *Author: Andy Seaborne <a...@apache.org <a...@apache.org>>* *Date: Thu Mar 29 17:08:34 2018 +0100* *[maven-release-plugin] prepare for next development iteration* and jena version is already "3.8.0-SNAPSHOT" . 2018-03-13 19:53 GMT+01:00 Andy Seaborne <a...@apache.org>: > The development community is heading towards Jena 3.7.0 (no specific date > - as ever, it has to fit around the volunteers time). > > We'd like to encourage you to test it before the release using the > development code and builds. Drop over to dev@. > > > There is lots in this release, much more than listed here, but areas that > might be worth you investigating and testing include: > > > == Java9: Running on a Java9 platform is supported > Language requirement remain Java8. More below. > > == SPARQL: Evaluation change > > JENA-1507: Changes to GROUP BY when the WHERE pattern has no results. > > == Feature > > JENA-1461: Allow ARQ custom functions to be written in JavaScript > Draft documentation: > http://jena.apache.org/documentation/query/javascript-functions.html > > == API > > This release is not binary compatible (NB Releases are not guaranteed to > be binary compatible nor mix-and-match versions of jars). > This time there are specific API changes: > > JENA-1389: Return `this` rather than `void` from Dataset > JENA-1495: PR#368 "Return Model from PrefixMapping methods" > > > ** Java9 > > Java8 is the current LTS version of Java and receives security updates. > The next LTS is Java11. [Sched] > > [Sched] http://www.oracle.com/technetwork/java/eol-135779.html > > There are different cases for running with Java9: > > [1] Run on a Java9 JVM > [2] Build using Java9 JDK, output Java8 class files. > [3] Java9 required - language features and runtime library > [4] Java9 modules > > [1] at Jena 3.7.0, running with a Java9 platform (except jena-elephas) is > supported. > > [2] at Jena 3.7.0, running maven (3.5.0 or later) using Java9 to produce > Java8 bytecode is supported (except as noted). We now run a Jenkins job > daily to check this for the main jars. > > [3] The project has no current plans to require a Java9 language runtime. > The language requirement is still Java8. > > [4] Jena jars can work as automatic modules but Jena itself does not > provide a "modules" version. Proper migration needs all the dependencies to > be modules, and also has implications on the upstream. Contribution and > discussions are welcome! > > For discussion and background see > > [A] http://blog.joda.org/2017/05/java-se-9-jpms-automatic-modules.html > [B] http://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html > > And note: Java 9 is obsolete March 2018, as soon as Java10 comes out. > http://openjdk.java.net/projects/jdk/10/#Schedule > > Notes: > > jena-elephas depends on Hadoop that depends on jdk.tools which is not > available in Java9. > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Vocabularies for actions
Laura, your use case is not clear enough . Anyway, *ask for* an action on github to "create new by " is done by POSTing the right JSON on the github API URL . And I don't see much difference between : - asking for* an action on github to "create a new issue" , and - simply, "create a new issue" on github In both case the data about the issueis the same . As a side note, AFAIK , on github the REST API is not in RDF. But if ones writes a suitable JSON-LD @context , it would be as if the github REST API is in RDF . 2018-03-17 17:20 GMT+01:00 Laura Morales <laure...@mail.com>: > Hi, > > all the vocabularies that I know of (and that I can find) seem to > *describe* something, like a Person. Is there any vocabulary to *ask for* > an action? For instance in the context of a source code repository, is > there a dictionary to describe an "action to be performed" by a machine > such as "create new by " or "add to " or > "fork to "? > > Thanks. > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
FILTER (CONTAINS on a graph name : should order matter ?
Hi !
This works as expected:
SELECT DISTINCT ?thing
WHERE {
graph ?thing {
[] ?p ?O .
}
FILTER (CONTAINS( str(?thing),"cartopair"))
}
but this gives an empty result :
SELECT DISTINCT ?thing
WHERE {
graph ?thing {
[] ?p ?O .
FILTER (CONTAINS( str(?thing),"cartopair"))
}
}
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Recursive SPARQL ( aka arbitrary length path ) : performance
I tried both queries, with union Graph , and without , and the result is
interesting and not ambiguous ( measured with time wget ) :
- with union Graph : around 0.07 second
- without union Graph: 12 seconds
About the inheritance depth, the main use case currently is to get the
foaf:Person when asking for a foaf:Agent .
Then other cases are the dbPedia stuff like cites, places .
And taxonomic data about species where family, genus, species are modeled
by RDF classes, and the depth (8) is like this
Kingdom: Plantae <https://en.wikipedia.org/wiki/Plant>
(unranked): Angiosperms <https://en.wikipedia.org/wiki/Angiosperms>
(unranked): Eudicots <https://en.wikipedia.org/wiki/Eudicots>
(unranked): Asterids <https://en.wikipedia.org/wiki/Asterids>
Order: Lamiales <https://en.wikipedia.org/wiki/Lamiales>
Family: Lamiaceae <https://en.wikipedia.org/wiki/Lamiaceae>
Genus: *Salvia <https://en.wikipedia.org/wiki/Salvia>*
Species: *S. fruticosa*
I would say, generally no deep hierarchy.
Here is there the query with union Graph :
PREFIX text: <http://jena.apache.org/text#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX form: <
http://raw.githubusercontent.com/jmvanel/semantic_forms/master/vocabulary/forms.owl.ttl#
>
SELECT DISTINCT ?thing ?COUNT WHERE {
?thing text:query ( 'Jean*' ) .
GRAPH {
?thing a ?sub .
?sub rdfs:subClassOf* <http://xmlns.com/foaf/0.1/Person> .
OPTIONAL {
?thing form:linksCount ?COUNT.
}
}
}
ORDER BY DESC(?COUNT)
LIMIT 10
and the one originally posted, more complex, and less efficient ( but same
result):
PREFIX text: <http://jena.apache.org/text#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX form: <
http://raw.githubusercontent.com/jmvanel/semantic_forms/master/vocabulary/forms.owl.ttl#
>
SELECT DISTINCT ?thing ?COUNT WHERE {
?thing text:query ( 'Jean*' ) .
graph ?g1 {
?thing a ?sub .
}
graph ?g2 {
?sub rdfs:subClassOf* <http://xmlns.com/foaf/0.1/Person> .
} .
OPTIONAL {
graph ?grCount {
?thing form:linksCount ?COUNT.
} }
}
ORDER BY DESC(?COUNT)
LIMIT 10
2018-02-08 16:19 GMT+01:00 Andy Seaborne <a...@apache.org>:
>
>
> On 08/02/18 08:39, Jean-Marc Vanel wrote:
>
>> Hi
>>
>> I wonder about performance of arbitrary length path in Jena :
>> https://www.w3.org/TR/2013/REC-sparql11-query-20130321/#prop
>> ertypath-arbitrary-length
>>
>> For example , here is the query in semantic_forms for searching a string
>> with a type class. I added yesterday the rdfs:subClassOf* pattern.
>> I wonder if it would not be more efficient with the unionGraph instead of
>> numerous GRAPH blocks .
>>
>
> if you try it, please do report on the effect.
>
> GRAPH
>
> Are the rdfs:subClassOf hierarchies very deep?
>
> As you have it, there is a bit of cross product of ?g1 and ?g2 going on.
>
> Andy
>
>
>> PREFIX text: <http://jena.apache.org/text#>
>> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
>> PREFIX form: <
>> http://raw.githubusercontent.com/jmvanel/semantic_forms/mast
>> er/vocabulary/forms.owl.ttl#
>>
>>>
>>>
>> SELECT DISTINCT ?thing ?COUNT WHERE {
>> ?thing text:query ( 'Jean*' ) .
>>
>> graph ?g1 {
>> ?thing a ?sub .
>> }
>> graph ?g2 {
>> ?sub *rdfs:subClassOf** <http://xmlns.com/foaf/0.1/Person> .
>> } .
>> OPTIONAL {
>> graph ?grCount {
>> ?thing form:linksCount ?COUNT.
>>} }
>> }
>> ORDER BY DESC(?COUNT)
>> LIMIT 10
>>
>>
>>
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Recursive SPARQL ( aka arbitrary length path ) : performance
Hi
I wonder about performance of arbitrary length path in Jena :
https://www.w3.org/TR/2013/REC-sparql11-query-20130321/#propertypath-arbitrary-length
For example , here is the query in semantic_forms for searching a string
with a type class. I added yesterday the rdfs:subClassOf* pattern.
I wonder if it would not be more efficient with the unionGraph instead of
numerous GRAPH blocks .
PREFIX text: <http://jena.apache.org/text#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX form: <
http://raw.githubusercontent.com/jmvanel/semantic_forms/master/vocabulary/forms.owl.ttl#
>
SELECT DISTINCT ?thing ?COUNT WHERE {
?thing text:query ( 'Jean*' ) .
graph ?g1 {
?thing a ?sub .
}
graph ?g2 {
?sub *rdfs:subClassOf** <http://xmlns.com/foaf/0.1/Person> .
} .
OPTIONAL {
graph ?grCount {
?thing form:linksCount ?COUNT.
} }
}
ORDER BY DESC(?COUNT)
LIMIT 10
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
end() called for WRITE transaction without commit or abort having been called ; THEN "Impossibly large object"
Hi On my laptop, I fixed my quad dump, reloaded it, and then I had this . I cut the bottom of stack. I think the rw() generic transaction function in Banana-RDF could be improved, and in my code I could catch in the right places, but anyway this happens , and Jena should handle this gracefully. Afterwards, I get "Impossibly large object" often , and the database seems damaged. Restarting the app. does not help. org.apache.jena.sparql.JenaTransactionException: end() called for WRITE transaction without commit or abort having been called. This causes a forced abort. at org.apache.jena.tdb.transaction.Transaction.close(Transaction.java:245) ~[jena-tdb-3.6.0.jar:3.6.0] at org.apache.jena.tdb.transaction.DatasetGraphTxn.end(DatasetGraphTxn.java:70) ~[jena-tdb-3.6.0.jar:3.6.0] at org.apache.jena.tdb.transaction.DatasetGraphTransaction._end(DatasetGraphTransaction.java:207) ~[jena-tdb-3.6.0.jar:3.6.0] at org.apache.jena.sparql.core.DatasetGraphTrackActive.end(DatasetGraphTrackActive.java:58) ~[jena-arq-3.6.0.jar:3.6.0] at org.apache.jena.sparql.core.DatasetImpl.end(DatasetImpl.java:136) ~[jena-arq-3.6.0.jar:3.6.0] at org.w3.banana.jena.JenaDatasetStore$$anonfun$rw$1.apply(JenaDatasetStore.scala:28) ~[banana-jena_2.11.jar:0.8.4-SNAPSHOT] at scala.util.Try$.apply(Try.scala:192) ~[scala-library-2.11.12.jar:?] at org.w3.banana.jena.*JenaDatasetStore.rw*(JenaDatasetStore.scala:22) ~[banana-jena_2.11.jar:0.8.4-SNAPSHOT] at org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:10) ~[banana-jena_2.11.jar:0.8.4-SNAPSHOT] at deductions.runtime.sparql_cache.SPARQLHelpers$class.wrapInTransaction(SPARQLHelpers.scala:152) ~[classes/:?] at controllers.Application$.wrapInTransaction(Application.scala:8) ~[classes/:?] at deductions.runtime.services.DashboardHistoryUserActions$class.makeTableHistoryUserActions(DashboardHistoryUserActions.scala:92) ~[classes/:?] -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: SPARQL: binding to a graph URI results in NullPointerException
Thanks Andy for looking at the DB, I'll see if the bug creating literal RDF subjects is still there. Although in theory this is just a development database, I think it is several month old , which speaks well for Jena's robustness ! I start a fresh database, but I keep the damaged one; tell me if you want binary TDB files . -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: Extending Sparql with New Keywords
wrote: > > > > Hello, > > > > I am using jena 3.6.0 and I would like to extend the sparql 1.1 > grammar > > with some new keywords. Could you please help me what would be > the > > best way > > to start with? > > > > - As I understood, the grammar is generated by javaCC but I was > unable > > to > > find any documentation or any main classes that actually > generate the > > grammar. Are grammar classes generated as an out source and then > added > > to > > the project? > > > > I would be really glad if you could guide me on this. > > > > Best > > Berkin > > > > > > > > > > > > > > > > > > -- Jean-Marc Vanel http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject <http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me> Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: SPARQL: binding to a graph URI results in NullPointerException
2018-02-06 18:40 GMT+01:00 Andy Seaborne <a...@apache.org>:
> An oddity:
>
> The only way I can see to set with a null binding (and not get a warning)
> is on the path where the dataset is not a TDB one, but with a graph in a
> general dataset.
>
> Is that the setup here?
>
The TDB is created and opened always by the same application , by
TDBFactory.createDataset(Paths.get(database_location).toString())
I forgot to say that sometimes during the tuning of my app. , there were
uncaught exception that caused end() being called without commit or
rollback .
> (that said, tracing down and eliminating all other possible routes back
> isn't simple so I might have missed another route)
>
> Andy
>
>
> On 06/02/18 14:24, Andy Seaborne wrote:
>
>> Can you dump the database?
>>
>>
>> On 06/02/18 10:32, Jean-Marc Vanel wrote:
>>
>>> Hi
>>>
>>> I consistently get a NullPointerException from this query on my test TDB
>>> on laptop.
>>>
>>> SELECT DISTINCT ?thing
>>> WHERE {
>>>graph ?thing {
>>> [] ?p ?O .
>>>}
>>> } LIMIT 200
>>>
>>> Looking into code : BindingBase.java line 203 , it appears that a
>>> binding's
>>> node is null for one SELECT result.
>>>
>>
>> In which case there is a binding and the value (Node) is null.
>>
>> We can put in null checks to catch it earlier (that would be a good idea
>> anyway) but still not at the point where the problem starts because there
>> isn't one such point - it depends on cache flush order.
>>
>> This binding stands for a named graph.
>>>
>>
>> Just a though - do your named graph have URI names? or are any blank
>> nodes?
>>
>> So the database is probably broken. Of course, this is a work and test
>>> database whose content has no value, but I try to understand and prevent
>>> this to happen on production database.
>>> Is a there a bug somewhere ?
>>>
>>
>> Yes :-) Where? now that is the question!
>>
>> What is the likely cause ? During tests and debugs, transactions may have
>>> not been rolled back , application was killed, etc .
>>> How could the database be repaired in such cases ? By doing a tdbdump and
>>> recreate the TDB ?
>>>
>>
>> Non-transactional TDB needs to explicitly flush modification to disk
>> (TDB.sync). Now due to caching something are being written all the time
>> but not competently and consistently.
>>
>> If an index gets to disk, with new Nodes (not already used) nodes, then
>> the NodeTable also needs to be written. If it is not, a later lookup of
>> the NodeId can't find the node -> null.
>>
>> If it is transactional, and is always used transactionally, the problem
>> does not arise.
>>
>> You might try TDB2 - it does not allow non-transactional use.
>>
>> In the future, it might get an "autocommit" feature ... but then users@
>> will get questions about why it is so slow. Full SQL compatibility!
>>
>> Andy
>>
>>
>>> Running on Jena 3.6.0 and Ubuntu 2017.10 and java version "1.8.0_121" .
>>>
>>> java.lang.NullPointerException
>>> at
>>> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:203)
>>>
>>> at
>>> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:183)
>>>
>>> at java.util.HashMap.hash(HashMap.java:338)
>>> at java.util.HashMap.containsKey(HashMap.java:595)
>>> at java.util.HashSet.contains(HashSet.java:203)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.get
>>> InputNextUnseen(QueryIterDistinct.java:106)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.has
>>> NextBinding(QueryIterDistinct.java:70)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.has
>>> Next(QueryIteratorBase.java:114)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNex
>>> tBinding(QueryIterSlice.java:76)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.has
>>> Next(QueryIteratorBase.java:114)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.
>>> hasNextBinding(QueryIteratorWrapper.java:39)
>>> at
>>> org.apache.jena.sparql.engine.iterator.Query
Re: SPARQL: binding to a graph URI results in NullPointerException
Sorry Andy I did one or 2 more additions to the database, and after that I had those very bad messages "Impossibly large object" , during the dump : ERROR TDB ObjectFileStorage.read[nodes](14903272)[filesize=14989232][file.size()=14989232]: Impossibly large object : 1936684402 bytes > filesize-(loc+SizeOfInt)=85956 org.apache.jena.tdb.base.file.FileException: ObjectFileStorage.read[nodes](14903272)[filesize=14989232][file.size()=14989232]: Impossibly large object : 1936684402 bytes > filesize-(loc+SizeOfInt)=85956 at org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:348) at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78) at org.apache.jena.tdb.store.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:186) at org.apache.jena.tdb.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:111) at org.apache.jena.tdb.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:70) at org.apache.jena.tdb.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:128) at org.apache.jena.tdb.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:82) at org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:50) at org.apache.jena.tdb.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67) at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:129) at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:123) at org.apache.jena.tdb.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:59) at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270) at org.apache.jena.riot.system.StreamOps.sendQuadsToStream(StreamOps.java:140) at org.apache.jena.riot.writer.NQuadsWriter.write$(NQuadsWriter.java:62) at org.apache.jena.riot.writer.NQuadsWriter.write(NQuadsWriter.java:45) at org.apache.jena.riot.writer.NQuadsWriter.write(NQuadsWriter.java:91) at org.apache.jena.riot.RDFWriter.write$(RDFWriter.java:208) at org.apache.jena.riot.RDFWriter.output(RDFWriter.java:165) at org.apache.jena.riot.RDFWriter.output(RDFWriter.java:112) at org.apache.jena.riot.RDFWriterBuilder.output(RDFWriterBuilder.java:149) at org.apache.jena.riot.RDFDataMgr.write$(RDFDataMgr.java:1269) at org.apache.jena.riot.RDFDataMgr.write(RDFDataMgr.java:1162) at tdb.tdbdump.exec(tdbdump.java:67) at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) at jena.cmd.CmdMain.mainRun(CmdMain.java:58) at jena.cmd.CmdMain.mainRun(CmdMain.java:45) at tdb.tdbdump.main(tdbdump.java:36) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sbt.Run.invokeMain(Run.scala:67) at sbt.Run.run0(Run.scala:61) at sbt.Run.sbt$Run$$execute$1(Run.scala:51) at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:55) at sbt.Run$$anonfun$run$1.apply(Run.scala:55) at sbt.Run$$anonfun$run$1.apply(Run.scala:55) at sbt.Logger$$anon$4.apply(Logger.scala:84) at sbt.TrapExit$App.run(TrapExit.scala:248) at java.lang.Thread.run(Thread.java:745) Exception: sbt.TrapExitSecurityException thrown from the UncaughtExceptionHandler in thread "run-main-0" -rw-r--r-- 1 jmv jmv 64649634 févr. 6 19:39 dump.nq But, unexpectedly, after restarting once or twice , everything is well again . Answers interleaved below. 2018-02-06 15:24 GMT+01:00 Andy Seaborne <a...@apache.org>: > Can you dump the database? > I put it here: http://jmvanel.free.fr/tmp/dump2.nq.zip I has been checked by rapper . > On 06/02/18 10:32, Jean-Marc Vanel wrote: > >> Hi >> >> I consistently get a NullPointerException from this query on my test TDB >> on laptop. >> . > > >> Just a though - do your named graph have URI names? or are any blank > nodes? > No , not any graph named by a blank node; I don't do that, and moreover I checked by SPARQL . However, there a subject "Celtis pumila" that is not a resource. > So the database is probably broken. Of course, this is a work and test >> database whose content has no value, but I try to understand and prevent >> this to happen on production database. >> Is a there a bug somewhere ? >> > > Yes :-) Where? now that is the question! > > What is the likely cause ? During tests and debugs, transactions may have >> not been rolled back , application was killed, etc . >> How could the database be repaired in such cases ? By doing a tdbdump and >> recreate the TDB ? >> > > Non-transactional TDB needs to explicitly flush modification to disk > (TDB.sync). Now due to cachin
SPARQL: binding to a graph URI results in NullPointerException
Hi
I consistently get a NullPointerException from this query on my test TDB
on laptop.
SELECT DISTINCT ?thing
WHERE {
graph ?thing {
[] ?p ?O .
}
} LIMIT 200
Looking into code : BindingBase.java line 203 , it appears that a binding's
node is null for one SELECT result.
This binding stands for a named graph.
So the database is probably broken. Of course, this is a work and test
database whose content has no value, but I try to understand and prevent
this to happen on production database.
Is a there a bug somewhere ?
What is the likely cause ? During tests and debugs, transactions may have
not been rolled back , application was killed, etc .
How could the database be repaired in such cases ? By doing a tdbdump and
recreate the TDB ?
Running on Jena 3.6.0 and Ubuntu 2017.10 and java version "1.8.0_121" .
java.lang.NullPointerException
at
org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:203)
at
org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:183)
at java.util.HashMap.hash(HashMap.java:338)
at java.util.HashMap.containsKey(HashMap.java:595)
at java.util.HashSet.contains(HashSet.java:203)
at
org.apache.jena.sparql.engine.iterator.QueryIterDistinct.getInputNextUnseen(QueryIterDistinct.java:106)
at
org.apache.jena.sparql.engine.iterator.QueryIterDistinct.hasNextBinding(QueryIterDistinct.java:70)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:74)
*at
org.apache.jena.sparql.engine.ResultSetCheckCondition.hasNext(ResultSetCheckCondition.java:55)*
at
scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
at scala.collection.Iterator$class.toStream(Iterator.scala:1320)
at scala.collection.AbstractIterator.toStream(Iterator.scala:1334)
at
scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
at
scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
at
scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
at
scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
at scala.collection.generic.Growable$class.loop$1(Growable.scala:54)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:57)
at
scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:183)
at
scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45)
at scala.collection.TraversableLike$class.to(TraversableLike.scala:590)
at scala.collection.AbstractTraversable.to(Traversable.scala:104)
at
deductions.runtime.sparql_cache.SPARQLHelpers$$anonfun$sparqlSelectQueryVariablesNT$1.apply(SPARQLHelpers.scala:426)
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
