Re: Does order of JOIN clauses affect performance

[ajs6f]

Are you using TDB?

This page:

https://jena.apache.org/documentation/tdb/optimizer.html

gives some info about how TDB uses statistics to reorder queries. To my understanding (Andy can correct) graphs are not 
taken into account. TDB2 may be different...


For other cases-- the transactional in-memory store doesn't care very much about this issue because hashmaps are used. I 
am not familiar enough with the other in-memory implementations to comment.



ajs6f

Martynas Jusevičius wrote on 10/25/17 8:58 AM:

Maybe this article can help:
http://wwwconference.org/www2008/papers/pdf/p595-stocker1.pdf

It's about BGPs not graph patterns, but I guess selectivity still applies.

On Wed, 25 Oct 2017 at 03.17, Dimov, Stefan  wrote:


Let’s consider the following JOIN:

SELECT $subj
FROM NAMED ng1
FROM NAMED ng2
{
GRAPH ng1  { $subj  $pred0  $obj0 }
GRAPH ng2  { $subj  $pred1  $obj1 }
}

My question is: Does the order of the clauses affect the performance?
Let’s say that ng1 is much bigger than ng2. If SPARQL applies the first
clause first and then the second clause over the set result of the first
one, then it makes sense to exchange their places.

Is that right?

Regards,
Stefan

Re: [3.0.1] listSuperClasses() does not traverse?

[Dave Reynolds]

On 24/10/17 23:51, Martynas Jusevičius wrote:

Hi,

I thought I understood how OntClass.listSuperClasses() works, but maybe I
don't.

I have such a class structure in my ontology (superclass is at the top):

3. https://www.w3.org/ns/ldt/document-hierarchy/domain#Item
 2. http://atomgraph.com/ns/platform/domain#Item
 1. https://localhost/admin/ns#AgentItem

Yet when I'm debugging, I can see the pair-wise relationships, but not the
chain all the way up from 1 to 3:

1. getOntology().getOntModel().getOntClass("
https://localhost/admin/ns#AgentItem
").listSuperClasses(false).toList().toString()

[https://localhost/admin/ns#ItemOfAgentContainer,
http://atomgraph.com/ns/platform/domain#Item]

2. getOntology().getOntModel().getOntClass("
http://atomgraph.com/ns/platform/domain#Item
").listSuperClasses(false).toList().toString()

[https://www.w3.org/ns/ldt/document-hierarchy/domain#Item]

I can see that within the method hasPropertyValue(
getProfile().SUB_CLASS_OF(), "SUB_CLASS_OF", cls ) returns false.

Why is that so? Is my usage wrong?

Additional info:
getOntology().getOntModel().getSpecification().getProfile() == OWLProfile
getOntology().getOntModel().getSpecification().getReasoner() == null



If I recall correctly the OntModel API is designed to retrieve whatever 
is stated within the underlying model. The notion was that there was no 
point in having the OntModel API replicate what reasoning does.


So to see the subclass closure you need to have a sufficient reasoner 
configured.



Dave

Re: Does order of JOIN clauses affect performance

[Andy Seaborne]

Hi Stefan,

Within each GRAPH block there is a light amount of optimization (picking 
the first, most grounded tripe pattern; avoiding rdf:type) by default.


There isn't anything across GRAPH blocks.

TDB2 is the same as TDB1 in this regard (TDB2 is reworking the storage 
and transactions, not the execution).


Andy

On 25/10/17 08:04, aj...@apache.org wrote:

Are you using TDB?

This page:

https://jena.apache.org/documentation/tdb/optimizer.html

gives some info about how TDB uses statistics to reorder queries. To my 
understanding (Andy can correct) graphs are not taken into account. TDB2 
may be different...


For other cases-- the transactional in-memory store doesn't care very 
much about this issue because hashmaps are used. I am not familiar 
enough with the other in-memory implementations to comment.



ajs6f

Martynas Jusevičius wrote on 10/25/17 8:58 AM:

Maybe this article can help:
http://wwwconference.org/www2008/papers/pdf/p595-stocker1.pdf

It's about BGPs not graph patterns, but I guess selectivity still 
applies.


On Wed, 25 Oct 2017 at 03.17, Dimov, Stefan  wrote:


Let’s consider the following JOIN:

SELECT $subj
FROM NAMED ng1
FROM NAMED ng2
{
    GRAPH ng1  { $subj  $pred0  $obj0 }
GRAPH ng2  { $subj  $pred1  $obj1 }
}

My question is: Does the order of the clauses affect the performance?
Let’s say that ng1 is much bigger than ng2. If SPARQL applies the first
clause first and then the second clause over the set result of the first
one, then it makes sense to exchange their places.

Is that right?

Regards,
Stefan

Re: [3.0.1] listSuperClasses() does not traverse?

[Martynas Jusevičius]

Thanks Dave.

We are materializing inferences during ontology initialization to avoid
using reasoner subsequently (as it impacts performance).

So in that case I need to traverse the chain myself, correct?

On Wed, Oct 25, 2017 at 9:33 AM, Dave Reynolds 
wrote:

> On 24/10/17 23:51, Martynas Jusevičius wrote:
>
>> Hi,
>>
>> I thought I understood how OntClass.listSuperClasses() works, but maybe I
>> don't.
>>
>> I have such a class structure in my ontology (superclass is at the top):
>>
>> 3. https://www.w3.org/ns/ldt/document-hierarchy/domain#Item
>>  2. http://atomgraph.com/ns/platform/domain#Item
>>  1. https://localhost/admin/ns#AgentItem
>>
>> Yet when I'm debugging, I can see the pair-wise relationships, but not the
>> chain all the way up from 1 to 3:
>>
>> 1. getOntology().getOntModel().getOntClass("
>> https://localhost/admin/ns#AgentItem
>> ").listSuperClasses(false).toList().toString()
>>
>> [https://localhost/admin/ns#ItemOfAgentContainer,
>> http://atomgraph.com/ns/platform/domain#Item]
>>
>> 2. getOntology().getOntModel().getOntClass("
>> http://atomgraph.com/ns/platform/domain#Item
>> ").listSuperClasses(false).toList().toString()
>>
>> [https://www.w3.org/ns/ldt/document-hierarchy/domain#Item]
>>
>> I can see that within the method hasPropertyValue(
>> getProfile().SUB_CLASS_OF(), "SUB_CLASS_OF", cls ) returns false.
>>
>> Why is that so? Is my usage wrong?
>>
>> Additional info:
>> getOntology().getOntModel().getSpecification().getProfile() == OWLProfile
>> getOntology().getOntModel().getSpecification().getReasoner() == null
>>
>>
> If I recall correctly the OntModel API is designed to retrieve whatever is
> stated within the underlying model. The notion was that there was no point
> in having the OntModel API replicate what reasoning does.
>
> So to see the subclass closure you need to have a sufficient reasoner
> configured.
>
>
> Dave
>

Re: [3.0.1] listSuperClasses() does not traverse?

[Dave Reynolds]

Hi Martynas,

On 25/10/17 10:33, Martynas Jusevičius wrote:

Thanks Dave.

We are materializing inferences during ontology initialization to avoid
using reasoner subsequently (as it impacts performance).


Makes sense.


So in that case I need to traverse the chain myself, correct?


Not if you've materialized the inferences. If you have constructed the 
superClass closure as part of this materialization then the closure 
should be visible through the OntAPI.


If you haven't included that in your materialization then indeed you 
would need to traverse the chain yourself - either in the API or via 
SPARQL property paths.


Dave



On Wed, Oct 25, 2017 at 9:33 AM, Dave Reynolds 
wrote:


On 24/10/17 23:51, Martynas Jusevičius wrote:


Hi,

I thought I understood how OntClass.listSuperClasses() works, but maybe I
don't.

I have such a class structure in my ontology (superclass is at the top):

3. https://www.w3.org/ns/ldt/document-hierarchy/domain#Item
  2. http://atomgraph.com/ns/platform/domain#Item
  1. https://localhost/admin/ns#AgentItem

Yet when I'm debugging, I can see the pair-wise relationships, but not the
chain all the way up from 1 to 3:

1. getOntology().getOntModel().getOntClass("
https://localhost/admin/ns#AgentItem
").listSuperClasses(false).toList().toString()

[https://localhost/admin/ns#ItemOfAgentContainer,
http://atomgraph.com/ns/platform/domain#Item]

2. getOntology().getOntModel().getOntClass("
http://atomgraph.com/ns/platform/domain#Item
").listSuperClasses(false).toList().toString()

[https://www.w3.org/ns/ldt/document-hierarchy/domain#Item]

I can see that within the method hasPropertyValue(
getProfile().SUB_CLASS_OF(), "SUB_CLASS_OF", cls ) returns false.

Why is that so? Is my usage wrong?

Additional info:
getOntology().getOntModel().getSpecification().getProfile() == OWLProfile
getOntology().getOntModel().getSpecification().getReasoner() == null



If I recall correctly the OntModel API is designed to retrieve whatever is
stated within the underlying model. The notion was that there was no point
in having the OntModel API replicate what reasoning does.

So to see the subclass closure you need to have a sufficient reasoner
configured.


Dave

Re: [3.0.1] listSuperClasses() does not traverse?

[Dave Reynolds]

On 25/10/17 11:10, George News wrote:

On 2017-10-25 11:54, Dave Reynolds wrote:

Hi Martynas,

On 25/10/17 10:33, Martynas Jusevičius wrote:

Thanks Dave.

We are materializing inferences during ontology initialization to avoid
using reasoner subsequently (as it impacts performance).


Makes sense.


So in that case I need to traverse the chain myself, correct?


Not if you've materialized the inferences. If you have constructed the
superClass closure as part of this materialization then the closure
should be visible through the OntAPI.

If you haven't included that in your materialization then indeed you
would need to traverse the chain yourself - either in the API or via
SPARQL property paths.


What do you mean by materialize the inferences of subClass? 


That's not quite the way I phrased it.

All I meant was that the reasoners will in effect compute

(?a rdfs:subClassOf ?b) (?b rdfs:subClassOf ?c)
 -> (?a rdfs:subClassOf ?c)

[Though technically the builtin reasoners don't use rules for that.]

So if Martynas wants the OntClass.listSuperClasses query to work as he 
expected then he would need to include that in the materialization.


So, as you say, it would include things like:

ClassChild rdf:subClassOf ClassParent
ClassChildChild rdf:subClassOf ClassParent
ClassChildChild rdf:subClassOf ClassChild


It is clear
that you include the inferences for the individuals, like:

individual rdf:type ClassParent
individual rdf:type ClassChild
individual rdf:type ClassChildChild

But if I also include the materialization for the class definition, at
the end, I'm including the "full" ontology model.

ClassChild rdf:subClassOf ClassParent
ClassChildChild rdf:subClassOf ClassParent
ClassChildChild rdf:subClassOf ClassChild

Do you recommend to also include the second step?
To use the standard refrain "it depends what you are specifically trying 
to do".


The benefit is that you can get all superclasses with a simple query. 
The cost, apart from some size growth, is that finding direct 
superclasses becomes very painful. Which is why the built in reasoners 
have the support for "direct" versions which is then exposed in the 
OntAPI via all the "direct" flags. That distinction can get lost in the 
materialization.


Personally I would not include the subClassOf closure if materializing 
but would rely on query rewriting to such questions on demand. YMMV


Dave



I'm involved in a similar procedure as Martynas.

Thanks
Jorge



Dave



On Wed, Oct 25, 2017 at 9:33 AM, Dave Reynolds

wrote:


On 24/10/17 23:51, Martynas Jusevičius wrote:


Hi,

I thought I understood how OntClass.listSuperClasses() works, but
maybe I
don't.

I have such a class structure in my ontology (superclass is at the
top):

3. https://www.w3.org/ns/ldt/document-hierarchy/domain#Item
   2. http://atomgraph.com/ns/platform/domain#Item
   1. https://localhost/admin/ns#AgentItem

Yet when I'm debugging, I can see the pair-wise relationships, but
not the
chain all the way up from 1 to 3:

1. getOntology().getOntModel().getOntClass("
https://localhost/admin/ns#AgentItem
").listSuperClasses(false).toList().toString()

[https://localhost/admin/ns#ItemOfAgentContainer,
http://atomgraph.com/ns/platform/domain#Item]

2. getOntology().getOntModel().getOntClass("
http://atomgraph.com/ns/platform/domain#Item
").listSuperClasses(false).toList().toString()

[https://www.w3.org/ns/ldt/document-hierarchy/domain#Item]

I can see that within the method hasPropertyValue(
getProfile().SUB_CLASS_OF(), "SUB_CLASS_OF", cls ) returns false.

Why is that so? Is my usage wrong?

Additional info:
getOntology().getOntModel().getSpecification().getProfile() ==
OWLProfile
getOntology().getOntModel().getSpecification().getReasoner() == null



If I recall correctly the OntModel API is designed to retrieve
whatever is
stated within the underlying model. The notion was that there was no
point
in having the OntModel API replicate what reasoning does.

So to see the subclass closure you need to have a sufficient reasoner
configured.


Dave

Re: [3.0.1] listSuperClasses() does not traverse?

[Martynas Jusevičius]

Dave,

I think I get it now. As you mention, we do not include subClassOf closure
during materialization, as the size grows but most of the inferences are
irrelevant.

So we only have direct relationships in the data but are in fact looking
for an inferred one, which is not there.

On Wed, Oct 25, 2017 at 12:36 PM, Dave Reynolds 
wrote:

> On 25/10/17 11:10, George News wrote:
>
>> On 2017-10-25 11:54, Dave Reynolds wrote:
>>
>>> Hi Martynas,
>>>
>>> On 25/10/17 10:33, Martynas Jusevičius wrote:
>>>
 Thanks Dave.

 We are materializing inferences during ontology initialization to avoid
 using reasoner subsequently (as it impacts performance).

>>>
>>> Makes sense.
>>>
>>> So in that case I need to traverse the chain myself, correct?

>>>
>>> Not if you've materialized the inferences. If you have constructed the
>>> superClass closure as part of this materialization then the closure
>>> should be visible through the OntAPI.
>>>
>>> If you haven't included that in your materialization then indeed you
>>> would need to traverse the chain yourself - either in the API or via
>>> SPARQL property paths.
>>>
>>
>> What do you mean by materialize the inferences of subClass?
>>
>
> That's not quite the way I phrased it.
>
> All I meant was that the reasoners will in effect compute
>
> (?a rdfs:subClassOf ?b) (?b rdfs:subClassOf ?c)
>  -> (?a rdfs:subClassOf ?c)
>
> [Though technically the builtin reasoners don't use rules for that.]
>
> So if Martynas wants the OntClass.listSuperClasses query to work as he
> expected then he would need to include that in the materialization.
>
> So, as you say, it would include things like:
>
> ClassChild rdf:subClassOf ClassParent
> ClassChildChild rdf:subClassOf ClassParent
> ClassChildChild rdf:subClassOf ClassChild
>
> It is clear
>> that you include the inferences for the individuals, like:
>>
>> individual rdf:type ClassParent
>> individual rdf:type ClassChild
>> individual rdf:type ClassChildChild
>>
>> But if I also include the materialization for the class definition, at
>> the end, I'm including the "full" ontology model.
>>
>> ClassChild rdf:subClassOf ClassParent
>> ClassChildChild rdf:subClassOf ClassParent
>> ClassChildChild rdf:subClassOf ClassChild
>>
>> Do you recommend to also include the second step?
>>
> To use the standard refrain "it depends what you are specifically trying
> to do".
>
> The benefit is that you can get all superclasses with a simple query. The
> cost, apart from some size growth, is that finding direct superclasses
> becomes very painful. Which is why the built in reasoners have the support
> for "direct" versions which is then exposed in the OntAPI via all the
> "direct" flags. That distinction can get lost in the materialization.
>
> Personally I would not include the subClassOf closure if materializing but
> would rely on query rewriting to such questions on demand. YMMV
>
> Dave
>
>
>> I'm involved in a similar procedure as Martynas.
>>
>> Thanks
>> Jorge
>>
>>
>>
>> Dave
>>>
>>>
>>> On Wed, Oct 25, 2017 at 9:33 AM, Dave Reynolds
 
 wrote:

 On 24/10/17 23:51, Martynas Jusevičius wrote:
>
> Hi,
>>
>> I thought I understood how OntClass.listSuperClasses() works, but
>> maybe I
>> don't.
>>
>> I have such a class structure in my ontology (superclass is at the
>> top):
>>
>> 3. https://www.w3.org/ns/ldt/document-hierarchy/domain#Item
>>2. http://atomgraph.com/ns/platform/domain#Item
>>1. https://localhost/admin/ns#AgentItem
>>
>> Yet when I'm debugging, I can see the pair-wise relationships, but
>> not the
>> chain all the way up from 1 to 3:
>>
>> 1. getOntology().getOntModel().getOntClass("
>> https://localhost/admin/ns#AgentItem
>> ").listSuperClasses(false).toList().toString()
>>
>> [https://localhost/admin/ns#ItemOfAgentContainer,
>> http://atomgraph.com/ns/platform/domain#Item]
>>
>> 2. getOntology().getOntModel().getOntClass("
>> http://atomgraph.com/ns/platform/domain#Item
>> ").listSuperClasses(false).toList().toString()
>>
>> [https://www.w3.org/ns/ldt/document-hierarchy/domain#Item]
>>
>> I can see that within the method hasPropertyValue(
>> getProfile().SUB_CLASS_OF(), "SUB_CLASS_OF", cls ) returns false.
>>
>> Why is that so? Is my usage wrong?
>>
>> Additional info:
>> getOntology().getOntModel().getSpecification().getProfile() ==
>> OWLProfile
>> getOntology().getOntModel().getSpecification().getReasoner() == null
>>
>>
>> If I recall correctly the OntModel API is designed to retrieve
> whatever is
> stated within the underlying model. The notion was that there was no
> point
> in having the OntModel API replicate what reasoning does.
>
> So to see the subclass closure you need to have a sufficient reasoner
> configured.
>
>
> Dave
>
>

>>>

Re: [3.0.1] listSuperClasses() does not traverse?

[Martynas Jusevičius]

I ended up using JenaUtil.getAllSuperClasses() from SPINRDF:
https://github.com/spinrdf/spinrdf/blob/a7fc9a1ca7badecb9a8d858f7a8a33bb106e629f/src/main/java/org/spinrdf/util/JenaUtil.java#L401

On Wed, Oct 25, 2017 at 1:22 PM, Martynas Jusevičius  wrote:

> Dave,
>
> I think I get it now. As you mention, we do not include subClassOf closure
> during materialization, as the size grows but most of the inferences are
> irrelevant.
>
> So we only have direct relationships in the data but are in fact looking
> for an inferred one, which is not there.
>
> On Wed, Oct 25, 2017 at 12:36 PM, Dave Reynolds  > wrote:
>
>> On 25/10/17 11:10, George News wrote:
>>
>>> On 2017-10-25 11:54, Dave Reynolds wrote:
>>>
 Hi Martynas,

 On 25/10/17 10:33, Martynas Jusevičius wrote:

> Thanks Dave.
>
> We are materializing inferences during ontology initialization to avoid
> using reasoner subsequently (as it impacts performance).
>

 Makes sense.

 So in that case I need to traverse the chain myself, correct?
>

 Not if you've materialized the inferences. If you have constructed the
 superClass closure as part of this materialization then the closure
 should be visible through the OntAPI.

 If you haven't included that in your materialization then indeed you
 would need to traverse the chain yourself - either in the API or via
 SPARQL property paths.

>>>
>>> What do you mean by materialize the inferences of subClass?
>>>
>>
>> That's not quite the way I phrased it.
>>
>> All I meant was that the reasoners will in effect compute
>>
>> (?a rdfs:subClassOf ?b) (?b rdfs:subClassOf ?c)
>>  -> (?a rdfs:subClassOf ?c)
>>
>> [Though technically the builtin reasoners don't use rules for that.]
>>
>> So if Martynas wants the OntClass.listSuperClasses query to work as he
>> expected then he would need to include that in the materialization.
>>
>> So, as you say, it would include things like:
>>
>> ClassChild rdf:subClassOf ClassParent
>> ClassChildChild rdf:subClassOf ClassParent
>> ClassChildChild rdf:subClassOf ClassChild
>>
>> It is clear
>>> that you include the inferences for the individuals, like:
>>>
>>> individual rdf:type ClassParent
>>> individual rdf:type ClassChild
>>> individual rdf:type ClassChildChild
>>>
>>> But if I also include the materialization for the class definition, at
>>> the end, I'm including the "full" ontology model.
>>>
>>> ClassChild rdf:subClassOf ClassParent
>>> ClassChildChild rdf:subClassOf ClassParent
>>> ClassChildChild rdf:subClassOf ClassChild
>>>
>>> Do you recommend to also include the second step?
>>>
>> To use the standard refrain "it depends what you are specifically trying
>> to do".
>>
>> The benefit is that you can get all superclasses with a simple query. The
>> cost, apart from some size growth, is that finding direct superclasses
>> becomes very painful. Which is why the built in reasoners have the support
>> for "direct" versions which is then exposed in the OntAPI via all the
>> "direct" flags. That distinction can get lost in the materialization.
>>
>> Personally I would not include the subClassOf closure if materializing
>> but would rely on query rewriting to such questions on demand. YMMV
>>
>> Dave
>>
>>
>>> I'm involved in a similar procedure as Martynas.
>>>
>>> Thanks
>>> Jorge
>>>
>>>
>>>
>>> Dave


 On Wed, Oct 25, 2017 at 9:33 AM, Dave Reynolds
> 
> wrote:
>
> On 24/10/17 23:51, Martynas Jusevičius wrote:
>>
>> Hi,
>>>
>>> I thought I understood how OntClass.listSuperClasses() works, but
>>> maybe I
>>> don't.
>>>
>>> I have such a class structure in my ontology (superclass is at the
>>> top):
>>>
>>> 3. https://www.w3.org/ns/ldt/document-hierarchy/domain#Item
>>>2. http://atomgraph.com/ns/platform/domain#Item
>>>1. https://localhost/admin/ns#AgentItem
>>>
>>> Yet when I'm debugging, I can see the pair-wise relationships, but
>>> not the
>>> chain all the way up from 1 to 3:
>>>
>>> 1. getOntology().getOntModel().getOntClass("
>>> https://localhost/admin/ns#AgentItem
>>> ").listSuperClasses(false).toList().toString()
>>>
>>> [https://localhost/admin/ns#ItemOfAgentContainer,
>>> http://atomgraph.com/ns/platform/domain#Item]
>>>
>>> 2. getOntology().getOntModel().getOntClass("
>>> http://atomgraph.com/ns/platform/domain#Item
>>> ").listSuperClasses(false).toList().toString()
>>>
>>> [https://www.w3.org/ns/ldt/document-hierarchy/domain#Item]
>>>
>>> I can see that within the method hasPropertyValue(
>>> getProfile().SUB_CLASS_OF(), "SUB_CLASS_OF", cls ) returns false.
>>>
>>> Why is that so? Is my usage wrong?
>>>
>>> Additional info:
>>> getOntology().getOntModel().getSpecification().getProfile() ==
>>> OWLProfile
>>> getOntology().getOntModel().getSpecification().getReasoner() 

Binding graph names for WITH and USING

[Charles Greer]

Hi jena folks, I was surprised recently with a customer who was surprised that 
my jena connector did not properly

bind graph names as variables after WITH.


WITH ?g

INSERT ...

DELETE ...


When I looked back at the SPARQL specs, it looks indeed true that variables are 
inadmissable after WITH or USING.

I am curious about how to write a workaround, short of putting a literal in for 
?g


Thanks, Charles


Charles Greer
Lead Engineer
MarkLogic Corporation
cgr...@marklogic.com
@grechaw
www.marklogic.com

Re: Binding graph names for WITH and USING

[james anderson]

good evening;


> On 2017-10-25, at 18:48, Charles Greer  wrote:
> 
> Hi jena folks, I was surprised recently with a customer who was surprised 
> that my jena connector did not properly
> 
> bind graph names as variables after WITH.
> 
> 
> WITH ?g
> 
> INSERT ...
> 
> DELETE ...
> 
> 
> When I looked back at the SPARQL specs, it looks indeed true that variables 
> are inadmissable after WITH or USING.
> 
> I am curious about how to write a workaround, short of putting a literal in 
> for ?g

do you have a concrete example which you are in a position to share?

where did the customer express a binding for the ?g of which they thought the 
‘with’ clause was in its scope?

best regards, from berlin,

Re: Binding graph names for WITH and USING

[Andy Seaborne]

Hi Charles,

Indeed as the spec says:

[41]Modify::=
( 'WITH' iri )?
( DeleteClause InsertClause? | InsertClause ) UsingClause*
'WHERE' GroupGraphPattern

Where does the value for $g come from in the example?

Andy

On 25/10/17 17:48, Charles Greer wrote:

Hi jena folks, I was surprised recently with a customer who was surprised that 
my jena connector did not properly

bind graph names as variables after WITH.


WITH ?g

INSERT ...

DELETE ...


When I looked back at the SPARQL specs, it looks indeed true that variables are 
inadmissable after WITH or USING.

I am curious about how to write a workaround, short of putting a literal in for 
?g


Thanks, Charles


Charles Greer
Lead Engineer
MarkLogic Corporation
cgr...@marklogic.com
@grechaw
www.marklogic.com

Re: Binding graph names for WITH and USING

[Charles Greer]

The error from Jena when I try to reproduce is fairly clear:


@Test
public void testGraphBinding() {
MarkLogicDatasetGraph dsg = getMarkLogicDatasetGraph();
String updateQuery = "WITH ?g INSERT {  
 \"1\" }";
BindingMap updateBindings = new BindingHashMap();
updateBindings.add(Var.alloc("g"),
NodeFactory.createURI("http://example.org/g";));
UpdateRequest update = new UpdateRequest();
update.add(updateQuery);
UpdateAction.execute(update, dsg, updateBindings);
}



org.apache.jena.query.QueryParseException: Encountered "  "?g "" at line 
1, column 6.
Was expecting one of:
 ...
 ...
 ...



So I guess to reframe my question -- given that the expectation of this 
customer is that ?g should be bindable here,

can I give them a rationale?  Sometimes "the spec prohibits it" is not what 
people want to hear.




Charles Greer
Lead Engineer
MarkLogic Corporation
cgr...@marklogic.com
@grechaw
www.marklogic.com


From: james anderson 
Sent: Wednesday, October 25, 2017 9:59:14 AM
To: users@jena.apache.org
Subject: Re: Binding graph names for WITH and USING

good evening;


> On 2017-10-25, at 18:48, Charles Greer  wrote:
>
> Hi jena folks, I was surprised recently with a customer who was surprised 
> that my jena connector did not properly
>
> bind graph names as variables after WITH.
>
>
> WITH ?g
>
> INSERT ...
>
> DELETE ...
>
>
> When I looked back at the SPARQL specs, it looks indeed true that variables 
> are inadmissable after WITH or USING.
>
> I am curious about how to write a workaround, short of putting a literal in 
> for ?g

do you have a concrete example which you are in a position to share?

where did the customer express a binding for the ?g of which they thought the 
‘with’ clause was in its scope?

best regards, from berlin,

Re: Binding graph names for WITH and USING

[Andy Seaborne]

What are they trying to achieve?

The Update has a second error the parsers didn't get to.

No WHERE clause - it's mandatory

Did they mean:

INSERT {
   GRAPH ?g {   "1" }
}
WHERE
 { GRAPH ?g { } }

GRAPH ?g { } returns all graph names.

Where does USING come into this?

> Sometimes "the spec prohibits it" is not what people want to hear.

That's what the commercial support has to answer!

Andy


On 25/10/17 19:02, Charles Greer wrote:

The error from Jena when I try to reproduce is fairly clear:


@Test
public void testGraphBinding() {
 MarkLogicDatasetGraph dsg = getMarkLogicDatasetGraph();
 String updateQuery = "WITH ?g INSERT {   
\"1\" }";
 BindingMap updateBindings = new BindingHashMap();
 updateBindings.add(Var.alloc("g"),
 NodeFactory.createURI("http://example.org/g";));
 UpdateRequest update = new UpdateRequest();
 update.add(updateQuery);
 UpdateAction.execute(update, dsg, updateBindings);
}



org.apache.jena.query.QueryParseException: Encountered "  "?g "" at line 
1, column 6.
Was expecting one of:
  ...
  ...
  ...



So I guess to reframe my question -- given that the expectation of this 
customer is that ?g should be bindable here,

can I give them a rationale?  Sometimes "the spec prohibits it" is not what 
people want to hear.




Charles Greer
Lead Engineer
MarkLogic Corporation
cgr...@marklogic.com
@grechaw
www.marklogic.com


From: james anderson 
Sent: Wednesday, October 25, 2017 9:59:14 AM
To: users@jena.apache.org
Subject: Re: Binding graph names for WITH and USING

good evening;



On 2017-10-25, at 18:48, Charles Greer  wrote:

Hi jena folks, I was surprised recently with a customer who was surprised that 
my jena connector did not properly

bind graph names as variables after WITH.


WITH ?g

INSERT ...

DELETE ...


When I looked back at the SPARQL specs, it looks indeed true that variables are 
inadmissable after WITH or USING.

I am curious about how to write a workaround, short of putting a literal in for 
?g


do you have a concrete example which you are in a position to share?

where did the customer express a binding for the ?g of which they thought the 
‘with’ clause was in its scope?

best regards, from berlin,

Run standalone Fuseki with no logs

[Laura Morales]

When I execute ./fuseki-server, all output is printed to screen. Fuseki prints 
all queries as "INFO".
How can I disable all logs? Or maybe only display ERROR/WARN but no INFO?

Re: Run standalone Fuseki with no logs

[Andy Seaborne]

http://jena.apache.org/documentation/fuseki2/fuseki-logging.html

On 25/10/17 20:15, Laura Morales wrote:

When I execute ./fuseki-server, all output is printed to screen. Fuseki prints all 
queries as "INFO".
How can I disable all logs? Or maybe only display ERROR/WARN but no INFO?

Re: Run standalone Fuseki with no logs

[ajs6f]

Have you tried the locations for log configurations as described here:

https://jena.apache.org/documentation/fuseki2/fuseki-logging.html#setting-logging

?


ajs6f

Laura Morales wrote on 10/25/17 9:15 PM:

When I execute ./fuseki-server, all output is printed to screen. Fuseki prints all 
queries as "INFO".
How can I disable all logs? Or maybe only display ERROR/WARN but no INFO?

Re: Fuseki+HDT graceful "File not found"

[ajs6f]

A service is a dataset (which could be on-disk or in-memory) along with some HTTP endpoints that support the protocols 
like SPARQL Query, SPARQL Update, Graph Store, etc.


https://jena.apache.org/documentation/fuseki2/fuseki-data-services.html

That page is super-rough and we really need to polish and rewrite it. (Contributions welcome, and remember, to work on 
Jena documentation you need nothing other than a browser and a basic knowledge of Markdown.)



ajs6f

Laura Morales wrote on 10/24/17 9:43 PM:

Yup, I think I get this. What I don't get is what a "service" is?



Sent: Tuesday, October 24, 2017 at 9:36 PM
From: "Andy Seaborne" 
To: users@jena.apache.org
Subject: Re: Fuseki+HDT graceful "File not found"

On 24/10/17 11:57, Laura Morales wrote:


"each data service is an RDF datasets and a configurable set of endpoints for various 
operations such as SPARQL query, SPARQL update and file upload" How is this different than a 
"normal" standalone installation?



It's not - the standalone server looks in run/configurartion/ directory
and the config.ttl server file.

If you create a dataset (and hence its endpoints) via the UI, that's
where the persistent configuration goes.

Andy

Re: Run standalone Fuseki with no logs

[Laura Morales]

Thanks. I've just downloaded the default lof4j.preperties file and it seems to 
work.

 
 

Sent: Wednesday, October 25, 2017 at 9:18 PM
From: aj...@apache.org
To: users@jena.apache.org
Subject: Re: Run standalone Fuseki with no logs
Have you tried the locations for log configurations as described here:

https://jena.apache.org/documentation/fuseki2/fuseki-logging.html#setting-logging

?


ajs6f

Laura Morales wrote on 10/25/17 9:15 PM:
> When I execute ./fuseki-server, all output is printed to screen. Fuseki 
> prints all queries as "INFO".
> How can I disable all logs? Or maybe only display ERROR/WARN but no INFO?
>

Re: Binding graph names for WITH and USING

[Charles Greer]

Hi Andy,


I was trying to just recreate the scenario -- the original query did have a 
WHERE.

I found it:


prefix raf: 
with $resourceMetadataGraph
delete {?urn raf:latest ?latest.}
where {?urn raf:latest ?latest.}


It was my attempt to present a workaround that ended up with a USING - from 
SPARQL 1.1


"""

To illustrate the use of the WITH clause, an operation of the general form:

WITH  DELETE { a b c } INSERT { x y z } WHERE { ... }

is considered equivalent to:

DELETE { GRAPH  { a b c } } INSERT { GRAPH  { x y z } } USING  
WHERE { ... }

"""


We have taken on binding after USING and WITH as an enhancement request, but 
given your response I'm going to push back

until SPARQL 2.1


Thank you, Charles



From: Andy Seaborne 
Sent: Wednesday, October 25, 2017 11:10:12 AM
To: users@jena.apache.org
Subject: Re: Binding graph names for WITH and USING

What are they trying to achieve?

The Update has a second error the parsers didn't get to.

No WHERE clause - it's mandatory

Did they mean:

INSERT {
GRAPH ?g {   "1" }
}
WHERE
  { GRAPH ?g { } }

GRAPH ?g { } returns all graph names.

Where does USING come into this?

 > Sometimes "the spec prohibits it" is not what people want to hear.

That's what the commercial support has to answer!

 Andy


On 25/10/17 19:02, Charles Greer wrote:
> The error from Jena when I try to reproduce is fairly clear:
>
>
> @Test
> public void testGraphBinding() {
>  MarkLogicDatasetGraph dsg = getMarkLogicDatasetGraph();
>  String updateQuery = "WITH ?g INSERT {  
>  \"1\" }";
>  BindingMap updateBindings = new BindingHashMap();
>  updateBindings.add(Var.alloc("g"),
>  NodeFactory.createURI("http://example.org/g";));
>  UpdateRequest update = new UpdateRequest();
>  update.add(updateQuery);
>  UpdateAction.execute(update, dsg, updateBindings);
> }
>
>
>
> org.apache.jena.query.QueryParseException: Encountered "  "?g "" at 
> line 1, column 6.
> Was expecting one of:
>   ...
>   ...
>   ...
>
>
>
> So I guess to reframe my question -- given that the expectation of this 
> customer is that ?g should be bindable here,
>
> can I give them a rationale?  Sometimes "the spec prohibits it" is not what 
> people want to hear.
>
>
>
>
> Charles Greer
> Lead Engineer
> MarkLogic Corporation
> cgr...@marklogic.com
> @grechaw
> www.marklogic.com
Best Database for Integrating Data From Silos | 
MarkLogic
www.marklogic.com
Learn why MarkLogic Enterprise NoSQL is the world's best database for 
integrating data from silos.


>
> 
> From: james anderson 
> Sent: Wednesday, October 25, 2017 9:59:14 AM
> To: users@jena.apache.org
> Subject: Re: Binding graph names for WITH and USING
>
> good evening;
>
>
>> On 2017-10-25, at 18:48, Charles Greer  wrote:
>>
>> Hi jena folks, I was surprised recently with a customer who was surprised 
>> that my jena connector did not properly
>>
>> bind graph names as variables after WITH.
>>
>>
>> WITH ?g
>>
>> INSERT ...
>>
>> DELETE ...
>>
>>
>> When I looked back at the SPARQL specs, it looks indeed true that variables 
>> are inadmissable after WITH or USING.
>>
>> I am curious about how to write a workaround, short of putting a literal in 
>> for ?g
>
> do you have a concrete example which you are in a position to share?
>
> where did the customer express a binding for the ?g of which they thought the 
> ‘with’ clause was in its scope?
>
> best regards, from berlin,
>
>

Re: Binding graph names for WITH and USING

[Andy Seaborne]

USING is the like  SELECT-FROM -- the WG decided a different name was 
better. It sets up the dataset to query in the WHERE clause.  Again, 
there is no variable binding to use.


It looks to me like you really want $resourceMetadataGraph to come from 
outside - a parametrized update.  If so, the unsubtle way is to do 
string replacement.  I don't know if ParameterizedQueryString will work.


Don't forget

DELETE WHERE  QuadPattern

saves writing template and pattern

Andy

On 25/10/17 21:04, Charles Greer wrote:


Hi Andy,


I was trying to just recreate the scenario -- the original query did have a 
WHERE.

I found it:


prefix raf: 
with $resourceMetadataGraph
delete {?urn raf:latest ?latest.}
where {?urn raf:latest ?latest.}


It was my attempt to present a workaround that ended up with a USING - from 
SPARQL 1.1


"""

To illustrate the use of the WITH clause, an operation of the general form:

WITH  DELETE { a b c } INSERT { x y z } WHERE { ... }

is considered equivalent to:

DELETE { GRAPH  { a b c } } INSERT { GRAPH  { x y z } } USING  
WHERE { ... }

"""


We have taken on binding after USING and WITH as an enhancement request, but 
given your response I'm going to push back

until SPARQL 2.1


Thank you, Charles



From: Andy Seaborne 
Sent: Wednesday, October 25, 2017 11:10:12 AM
To: users@jena.apache.org
Subject: Re: Binding graph names for WITH and USING

What are they trying to achieve?

The Update has a second error the parsers didn't get to.

No WHERE clause - it's mandatory

Did they mean:

INSERT {
 GRAPH ?g {   "1" }
}
WHERE
   { GRAPH ?g { } }

GRAPH ?g { } returns all graph names.

Where does USING come into this?

  > Sometimes "the spec prohibits it" is not what people want to hear.

That's what the commercial support has to answer!

  Andy


On 25/10/17 19:02, Charles Greer wrote:

The error from Jena when I try to reproduce is fairly clear:


@Test
public void testGraphBinding() {
  MarkLogicDatasetGraph dsg = getMarkLogicDatasetGraph();
  String updateQuery = "WITH ?g INSERT {   
\"1\" }";
  BindingMap updateBindings = new BindingHashMap();
  updateBindings.add(Var.alloc("g"),
  NodeFactory.createURI("http://example.org/g";));
  UpdateRequest update = new UpdateRequest();
  update.add(updateQuery);
  UpdateAction.execute(update, dsg, updateBindings);
}



org.apache.jena.query.QueryParseException: Encountered "  "?g "" at line 
1, column 6.
Was expecting one of:
   ...
   ...
   ...



So I guess to reframe my question -- given that the expectation of this 
customer is that ?g should be bindable here,

can I give them a rationale?  Sometimes "the spec prohibits it" is not what 
people want to hear.




Charles Greer
Lead Engineer
MarkLogic Corporation
cgr...@marklogic.com
@grechaw
www.marklogic.com

Best Database for Integrating Data From Silos | 
MarkLogic
www.marklogic.com
Learn why MarkLogic Enterprise NoSQL is the world's best database for 
integrating data from silos.





From: james anderson 
Sent: Wednesday, October 25, 2017 9:59:14 AM
To: users@jena.apache.org
Subject: Re: Binding graph names for WITH and USING

good evening;



On 2017-10-25, at 18:48, Charles Greer  wrote:

Hi jena folks, I was surprised recently with a customer who was surprised that 
my jena connector did not properly

bind graph names as variables after WITH.


WITH ?g

INSERT ...

DELETE ...


When I looked back at the SPARQL specs, it looks indeed true that variables are 
inadmissable after WITH or USING.

I am curious about how to write a workaround, short of putting a literal in for 
?g


do you have a concrete example which you are in a position to share?

where did the customer express a binding for the ?g of which they thought the 
‘with’ clause was in its scope?

best regards, from berlin,

Using Fuseki over HTTP

[Jeffrey Brown]

I want to rewrite Digraphs with Text[1] to use an external Datalog server,
rather than the native Haskell module for graphs that it currently uses.
Jena looks like the best open-source choice. I can run ./fuseki, visit
http://localhost:3030, upload data and manually query it. I want to do
those things over HTTP.

I have found a number of sources that describe Jena's RDF and Sparql APIs,
but nothing describing its web API. My rudimentary understanding of web
services is that there's some URL of the form "http://localhost:3030//
dataset/query?magic", where an appropriate value for "magic" would produce
the results of a query. Is that correct?

How does one translate data entry (or removal) commands from Jena's RDF API
to Jena's Web RDF API? How does one translate queries from Jena's Sparql
API to Jena's Web Sparql API?

Last, the Jena documentation says that it contains two inference engines.
Is Datalog one of them and Sparql the other? I'm finding a lot of Jena
Sparql documentation, but no Jena Datalog documentation.

Many thanks.


[1] Digraphs with Text
 is a simple
app that provides a generalization of the graph (relationships of arbitrary
arity, arbitrarily nested) and a language for input and queries that is
friendly to non-programmers.


-- 
Jeff Brown | Jeffrey Benjamin Brown
Website    |   Facebook
   |   LinkedIn
(spammy, so I often miss
messages here)   |   Github 

Re: Binding graph names for WITH and USING

[Claude Warren]

Another template solution is the query builder where you can bind the ?g
just before query.  Though I am not certain that you can specify a var for
the graph name when building the query.  If you can then using the setVar()
method to replace it just before execution.

Claude

On Wed, Oct 25, 2017 at 10:35 PM, Andy Seaborne  wrote:

> USING is the like  SELECT-FROM -- the WG decided a different name was
> better. It sets up the dataset to query in the WHERE clause.  Again, there
> is no variable binding to use.
>
> It looks to me like you really want $resourceMetadataGraph to come from
> outside - a parametrized update.  If so, the unsubtle way is to do string
> replacement.  I don't know if ParameterizedQueryString will work.
>
> Don't forget
>
> DELETE WHERE  QuadPattern
>
> saves writing template and pattern
>
> Andy
>
>
> On 25/10/17 21:04, Charles Greer wrote:
>
>>
>> Hi Andy,
>>
>>
>> I was trying to just recreate the scenario -- the original query did have
>> a WHERE.
>>
>> I found it:
>>
>>
>> prefix raf: 
>> with $resourceMetadataGraph
>> delete {?urn raf:latest ?latest.}
>> where {?urn raf:latest ?latest.}
>>
>>
>> It was my attempt to present a workaround that ended up with a USING -
>> from SPARQL 1.1
>>
>>
>> """
>>
>> To illustrate the use of the WITH clause, an operation of the general
>> form:
>>
>> WITH  DELETE { a b c } INSERT { x y z } WHERE { ... }
>>
>> is considered equivalent to:
>>
>> DELETE { GRAPH  { a b c } } INSERT { GRAPH  { x y z } } USING
>>  WHERE { ... }
>>
>> """
>>
>>
>> We have taken on binding after USING and WITH as an enhancement request,
>> but given your response I'm going to push back
>>
>> until SPARQL 2.1
>>
>>
>> Thank you, Charles
>>
>>
>> 
>> From: Andy Seaborne 
>> Sent: Wednesday, October 25, 2017 11:10:12 AM
>> To: users@jena.apache.org
>> Subject: Re: Binding graph names for WITH and USING
>>
>> What are they trying to achieve?
>>
>> The Update has a second error the parsers didn't get to.
>>
>> No WHERE clause - it's mandatory
>>
>> Did they mean:
>>
>> INSERT {
>>  GRAPH ?g {   "1" }
>> }
>> WHERE
>>{ GRAPH ?g { } }
>>
>> GRAPH ?g { } returns all graph names.
>>
>> Where does USING come into this?
>>
>>   > Sometimes "the spec prohibits it" is not what people want to hear.
>>
>> That's what the commercial support has to answer!
>>
>>   Andy
>>
>>
>> On 25/10/17 19:02, Charles Greer wrote:
>>
>>> The error from Jena when I try to reproduce is fairly clear:
>>>
>>>
>>> @Test
>>> public void testGraphBinding() {
>>>   MarkLogicDatasetGraph dsg = getMarkLogicDatasetGraph();
>>>   String updateQuery = "WITH ?g INSERT {  <
>>> http://example.org/p> \"1\" }";
>>>   BindingMap updateBindings = new BindingHashMap();
>>>   updateBindings.add(Var.alloc("g"),
>>>   NodeFactory.createURI("http://example.org/g";));
>>>   UpdateRequest update = new UpdateRequest();
>>>   update.add(updateQuery);
>>>   UpdateAction.execute(update, dsg, updateBindings);
>>> }
>>>
>>>
>>>
>>> org.apache.jena.query.QueryParseException: Encountered "  "?g ""
>>> at line 1, column 6.
>>> Was expecting one of:
>>>...
>>>...
>>>...
>>>
>>>
>>>
>>> So I guess to reframe my question -- given that the expectation of this
>>> customer is that ?g should be bindable here,
>>>
>>> can I give them a rationale?  Sometimes "the spec prohibits it" is not
>>> what people want to hear.
>>>
>>>
>>>
>>>
>>> Charles Greer
>>> Lead Engineer
>>> MarkLogic Corporation
>>> cgr...@marklogic.com
>>> @grechaw
>>> www.marklogic.com
>>>
>> Best Database for Integrating Data From Silos | MarkLogic<
>> http://www.marklogic.com/>
>> www.marklogic.com
>> Learn why MarkLogic Enterprise NoSQL is the world's best database for
>> integrating data from silos.
>>
>>
>>
>>> 
>>> From: james anderson 
>>> Sent: Wednesday, October 25, 2017 9:59:14 AM
>>> To: users@jena.apache.org
>>> Subject: Re: Binding graph names for WITH and USING
>>>
>>> good evening;
>>>
>>>
>>> On 2017-10-25, at 18:48, Charles Greer 
 wrote:

 Hi jena folks, I was surprised recently with a customer who was
 surprised that my jena connector did not properly

 bind graph names as variables after WITH.


 WITH ?g

 INSERT ...

 DELETE ...


 When I looked back at the SPARQL specs, it looks indeed true that
 variables are inadmissable after WITH or USING.

 I am curious about how to write a workaround, short of putting a
 literal in for ?g

>>>
>>> do you have a concrete example which you are in a position to share?
>>>
>>> where did the customer express a binding for the ?g of which they
>>> thought the ‘with’ clause was in its scope?
>>>
>>> best regards, from berlin,
>>>
>>>
>>>
>>


-- 
I like: Like Like - The likeliest