Re: [Virtuoso-users] deadlocks in bulk loading using rdf_loader_run function

[Davis, Daniel (NIH/NLM) [C]]

One way to achieve this is to use RDF Graph 
Groups
 and read-only users, e.g. users in the SPARQL_SELECT group.   The idea is to 
load data into a graph that your normal, read-only user will not see due to RDF 
Graph Security, and 
then move the triples into a graph that they can see.

Of course, the data is ultimately in RDF_QUAD.The advantage of this is that 
a more privileged user can access the data using SPARQL and SPARUL:

PREFIX …
INSERT { GRAPH  { ?s ?p ?o } }
WHERE { GRAPH  { ?s ?p ?o } }

This also allows for some transformation as data is copied.

Another advantage is that Virtuoso has assigned an IRI_ID to the triples, and 
so when changes are made, they can be as limited as possible (in SQL):

UPDATE RDF_QUAD
   SET g = rdf_make_iid_of_qname(newgraph_iri)
 WHERE g = rdf_make_iid_of_qname(oldgraph_iri);

Hope this helps,

Dan Davis

From: Gang Fu [mailto:gangfu1...@gmail.com]
Sent: Monday, December 21, 2015 9:28 AM
To: virtuoso-users 
Subject: Re: [Virtuoso-users] deadlocks in bulk loading using rdf_loader_run 
function

Is it possible to load files into staging tables first, then merge them into 
RDF_QUAD?

Best,
Gang

On Fri, Dec 18, 2015 at 2:34 PM, Gang Fu 
mailto:gangfu1...@gmail.com>> wrote:
Hi,

I have deadlock problem with function 'rdf_loader_run' running in parallel, and 
I have tried many ways to avoid deadlocks, including

1)  Preprocess files using sort and uniq

2)  Loading without indexes

3)  Split big files into very small chunks

But deadlocks problem is still there. It seems the only way to avoid this is 
loading in single thread, which is very slow… any other solutions?

Best,
Gang


--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] deadlocks in bulk loading using rdf_loader_run function

[Gang Fu]

Is it possible to load files into staging tables first, then merge them
into RDF_QUAD?

Best,
Gang

On Fri, Dec 18, 2015 at 2:34 PM, Gang Fu  wrote:

> Hi,
>
> I have deadlock problem with function 'rdf_loader_run' running in
> parallel, and I have tried many ways to avoid deadlocks, including
>
> 1)  Preprocess files using sort and uniq
>
> 2)  Loading without indexes
>
> 3)  Split big files into very small chunks
>
>
>
> But deadlocks problem is still there. It seems the only way to avoid this
> is loading in single thread, which is very slow… any other solutions?
>
>
>
> Best,
>
> Gang
>
>
>
--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users