Re: [Virtuoso-users] Virtuoso 37000 Error

[Quentin]

 

RE: [VIRTUOSO-USERS] VIRTUOSO 37000 ERROR

From
q [11]

To
virtuoso-users@lists.sourceforge.net [11]

Date
Today 09:56

<- [12] <<- [12] -> [12] [] [13] 

Hi S.M.Shamimul, 

I'm not with openlink so my advice is only from a fellow developer. I
have a few minutes though so here's my thoughts on your problem. 

The error you're getting looks like a bug to me, maybe in the query
optimiser since it's occurring after SQL compilation. I can't tell you
much about that bug except that it seems to occur while exploring
transitives but I can give you some tips that might help you avoid it.
Transitives in Virtuoso Open Source are a relatively unstable feature 

Your SPARQL query could be pre-optimised a bit which might simplify it
enough for the query optimiser to avoid whatever situation is causing
that error. 

Your first subquery has three clauses but the first two are not related
to each other. Reordering these might give some clues to the query
optimiser and potentially prevent a full self-join on the graph. 

May as well use a prefix too. 

prefix abc:  

Instead of:
 ?begin  ?o1.
 ?end * ?mid.
 ?mid * ?begin. 

Do: 

 ?begin abc:getInfectedBy [3] ?o1.
 ?mid abc:getInfectedBy [3]* ?begin.
 ?end abc:getInfectedBy [3]* ?mid. 

That way, each step is related to something previous. It's not always
true but cause ordering can be important sometimes. At least this way,
you don't have to trust that the query optimiser (which is a black box
to us) will work it out. 

Since you're using property paths, the ?mid node might be redundant, at
least you probably don't need two property paths in that clause. This
might be what you intended:
 ?begin abc:getInfectedBy ?o1.
 ?mid abc:getInfectedBy ?begin.
 ?end abc:getInfectedBy* ?mid. 

That will still experience the error. Modifying the query as below will
not experience the error but probably doesn't do what you want.
Interesting point for the developers who look into it though. This
version requires at least one instance of ?mid to exist. I think the
above lets ?end and ?mid bind to the same node though I'm not certain
without having data to test it against.
 ?begin abc:getInfectedBy ?o1.
 ?mid abc:getInfectedBy* ?begin.
 ?end abc:getInfectedBy ?mid. 

Another issue, not the actual cause of error, is that the second
subquery selects a bunch of variables that don't seem to be used. Is
your intent to filter ?o1 to only include the nodes satisfying the
clauses in the second subquery? 

Be aware that you're not filtering ?g and ?g1 so these can be satisfied
with any graph in your DB, including the same graph. If this is intended
then you can remove the graph clauses entirely and combine the two
subqueries into one query which will operate over the entire DB. Such a
query will probably not experience the error you're encountering at all.


Regarding the literals in your second subquery, these can be more
efficiently processed as filtered variables than as literals. Filtering
variables is nearly always more efficient than satisfying literals.
That's probably not the source of the error but it's a good idea
generally. 

?s 
'0'^^xsd:decimal. 

becomes 

?s abc:dendrogram_iteration [6] ?iter. 

filter ( ?iter = '0'^^xsd:decimal ) 

You also seem to be ordering the query and then limiting it to 1 row
which seems redundant, the results from the query aren't selected in the
main query so it's only purpose is to filter ?o1 I guess. 

The last two lines of the second subquery are duplicates and one can be
removed. 

You should probably have a closer look at your second query and make
sure it does what you intend it to do. Work with it alone to get it
working before you include it in the main query. 

Tinkering with your query, it appears that the mere presence of the
second subquery causes the error, I guess that the transitive predicates
don't play well with subqueries, that's probably a bug. 

Regards, 

Quentin 

Guiding Hand Solutions 

On 2016-06-11 07:08, S.M.Shamimul Hasan wrote: 

> Greetings,
> 
> I am running the following query on Virtuoso (7.1.0) SPARQL endpoint. 
> 
> 
> select (?o1 as ?initialseed), ?begin, ?end, (count(?mid)+1 as ?length) 
> where {
> GRAPH ?g
> {
> ?begin  ?o1.
> ?end * ?mid.
> ?mid * ?begin.
> }
> 
> {
> select ?o1, ?o2, ?e
> where{
> GRAPH ?g1
> {
> ?s  ?o1.
> ?s  ?o2.
> ?s  

Re: [Virtuoso-users] Virtuoso 37000 Error

[Quentin]

 

I'd suggest reviewing the documentation on property paths as well. 

https://www.w3.org/2009/sparql/wiki/Feature:PropertyPaths 

http://docs.openlinksw.com/virtuoso/sparqlextensions.html#rdfsparqlaggregatepathexpressions


On 2016-06-11 07:08, S.M.Shamimul Hasan wrote: 

> Greetings,
> 
> I am running the following query on Virtuoso (7.1.0) SPARQL endpoint. 
> 
> 
> select (?o1 as ?initialseed), ?begin, ?end, (count(?mid)+1 as ?length) 
> where {
> GRAPH ?g
> {
> ?begin  ?o1.
> ?end * ?mid.
> ?mid * ?begin.
> }
> 
> {
> select ?o1, ?o2, ?e
> where{
> GRAPH ?g1
> {
> ?s  ?o1.
> ?s  ?o2.
> ?s  '0'^^xsd:decimal.
> ?s  ?e.
> ?s1  ?o1.
> ?s1  ?o2.
> ?s1  '5'^^xsd:decimal.
> ?s1  '5'^^xsd:decimal
> }
> }ORDER BY ASC(?e) LIMIT 1 
> }
> 
> }GROUP BY ?o1 ?begin ?end
> 
> ***
> It is giving me following error.
> 
> 
> 
> Virtuoso 37000 Error SP031: SPARQL compiler: Variable '_::trans_subj_7_1' is 
> used in subexpressions of the query but not assigned
> 
> ***
> 
> In the query instead of "GRAPH ?g" if I use FROM clause with graph URI then 
> query works fine. However I need generic "GRAPH ?g." Please let me know how I 
> can fix it.
> 
> Thank you.
> 
> Regards,
> S.M.Shamimul Hasan
> 
> --
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity 
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e [1]
> 
> ___
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users [2]

 

Links:
--
[1] https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
[2] https://lists.sourceforge.net/lists/listinfo/virtuoso-users
[3] http://test.edu/abc/vocab/getInfectedBy
[4] http://test.edu/abc/vocab/dendrogram_infector_pid
[5] http://test.edu/abc/vocab/dendrogram_infectee_pid
[6] http://test.edu/abc/vocab/dendrogram_iteration
[7] http://test.edu/abc/vocab/dendrogram_exposureday
[8] http://test.edu/abc/vocab/contactnetwork_pid1
[9] http://test.edu/abc/vocab/contactnetwork_pid2
[10] http://test.edu/abc/vocab/contactnetwork_acttype1
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Virtuoso 37000 Error

[S.M.Shamimul Hasan]

Greetings,

I am running the following query on Virtuoso (7.1.0) SPARQL endpoint.


select  (?o1 as ?initialseed), ?begin, ?end, (count(?mid)+1 as ?length)
where {
GRAPH ?g
{
?begin  ?o1.
?end * ?mid.
?mid * ?begin.
}

 {
select ?o1, ?o2, ?e
where{
GRAPH ?g1
{
?s  ?o1.
?s  ?o2.
?s 
'0'^^xsd:decimal.
?s  ?e.
?s1  ?o1.
?s1  ?o2.
?s1 
'5'^^xsd:decimal.
?s1 
'5'^^xsd:decimal
 }
}ORDER BY ASC(?e) LIMIT 1
 }

}GROUP BY ?o1 ?begin ?end

***
It is giving me following error.



Virtuoso 37000 Error SP031: SPARQL compiler: Variable '_::trans_subj_7_1'
is used in subexpressions of the query but not assigned

***

In the query instead of "GRAPH  ?g" if I use FROM clause with graph URI
then query works fine.  However I need generic "GRAPH ?g." Please let me
know how I can fix it.

Thank you.

Regards,
S.M.Shamimul Hasan
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Replication on a schedule

[Hugh Williams]

Hi Daniel,

This is an open source mailing list for which cluster is not available as it is 
a commercial only feature, thus not likely to get much response on it …

Note sure what you mean by "replicate on a schedule rather than as deltas are 
flushed” ?

You seem to be seeking to use Virtuoso scale out Elastic Cluster [1][2] which 
is for horizontal scaling across multiple machine to pool resources (storage, 
memory, cores) for hosting large datasets that could not otherwise be hosting 
on a single machine. Virtuoso also has normal Database replication although 
this is a commercial feature only also and is also used for providing Virtuoso 
RDF Graph Replication clustering[3] of of single or scale out cluster farms. 
Based on the email you sent to support is would appear the Virtuoso RDF Graph 
Replication Clustering is most suitable for you use case.

See also the different cluster configuration architectures at [4] …

[1] http://docs.openlinksw.com/virtuoso/clusteroperation.html
[2] http://docs.openlinksw.com/virtuoso/clusterstcnf.html
[3] http://docs.openlinksw.com/virtuoso/rdfgraphreplication.html
[4] 
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtClusteringDiagrams

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

> On 10 Jun 2016, at 17:17, Davis, Daniel (NIH/NLM) [C]  
> wrote:
> 
> So, I’m not sure whether these are available in the Conductor interface, but 
> the cluster control utility (clctl) seems to support what I want to do if I 
> have scheduled jobs which turn on a cluster member for replication and then 
> stop the replication at some point:
>  
> At 2 am, clctl rebuild 2
> …
> At 4 am, check that node 2 is up-to-date, and then disable it - 
> clctl disable 2
>  
> This is probably an anti-pattern however.  Running with row-level autocommit 
> has been good because I don’t have clustering, and am manually replicating 
> via scripts.  I should probably rewack my stored procedures and systems 
> architecture so that my update jobs run efficiently with transaction 
> isolation.
>  
> From: Davis, Daniel (NIH/NLM) [C] 
> Sent: Friday, June 10, 2016 10:55 AM
> To: virtuoso-users 
> Subject: [Virtuoso-users] Replication on a schedule
>  
> Hi, does anyone out there with the Cluster edition of Virtuoso replicate on a 
> schedule rather than as deltas are flushed?  Is this desirable?
>  
> The issue is that in a star topology, replication represents an increased 
> risk of the replicant failing.  It may therefore be better to replicate only 
> during certain time frames so that the cluster remains redundant.   I know I 
> could do this scripts subscribing and subscribing to deltas, but it would be 
> more hands-off if Virtuoso has a simple concept of when to apply deltas.
>  
> Dan Davis, Systems/Applications Architect (Contractor),
> Office of Computer and Communications Systems,
> National Library of Medicine, NIH
>  
> --
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity 
> planning reports. 
> https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users



smime.p7s
Description: S/MIME cryptographic signature
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Replication on a schedule

[Davis, Daniel (NIH/NLM) [C]]

So, I'm not sure whether these are available in the Conductor interface, but 
the cluster control utility (clctl) seems to support what I want to do if I 
have scheduled jobs which turn on a cluster member for replication and then 
stop the replication at some point:

At 2 am, clctl rebuild 2
...
At 4 am, check that node 2 is up-to-date, and then disable it - 
clctl disable 2

This is probably an anti-pattern however.  Running with row-level autocommit 
has been good because I don't have clustering, and am manually replicating via 
scripts.  I should probably rewack my stored procedures and systems 
architecture so that my update jobs run efficiently with transaction isolation.

From: Davis, Daniel (NIH/NLM) [C]
Sent: Friday, June 10, 2016 10:55 AM
To: virtuoso-users 
Subject: [Virtuoso-users] Replication on a schedule

Hi, does anyone out there with the Cluster edition of Virtuoso replicate on a 
schedule rather than as deltas are flushed?  Is this desirable?

The issue is that in a star topology, replication represents an increased risk 
of the replicant failing.  It may therefore be better to replicate only during 
certain time frames so that the cluster remains redundant.   I know I could do 
this scripts subscribing and subscribing to deltas, but it would be more 
hands-off if Virtuoso has a simple concept of when to apply deltas.

Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Replication on a schedule

[Davis, Daniel (NIH/NLM) [C]]

Hi, does anyone out there with the Cluster edition of Virtuoso replicate on a 
schedule rather than as deltas are flushed?  Is this desirable?

The issue is that in a star topology, replication represents an increased risk 
of the replicant failing.  It may therefore be better to replicate only during 
certain time frames so that the cluster remains redundant.   I know I could do 
this scripts subscribing and subscribing to deltas, but it would be more 
hands-off if Virtuoso has a simple concept of when to apply deltas.

Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users