date:20160817

Re: [Virtuoso-users] Deleting large number of triples

2016-08-17 Thread Quentin

 

Hi Pantelis, 

If you want fine control over processing of results then Daniel's answer
is the way to go, PLSQL allows you to do basically anything with your
results. 

However it's not necessary in this instance I think. We can probably do
what you want with a subquery. Since you only want to process a select
number of records (subject to some memory constraint that you'll find
out by trial-and-error) then we can do this with a subquery in SPARQL.
Additionally, select statements have (or used to have) a hard limit of
10,000 results that might complicate the process. 

-- 

SPARQL DEFINE sql:log-enable 3 

WITH GRAPH 
DELETE {
?S ?P ?O .
} WHERE {
{ SELECT ?S ?P ?O 
{ ?S ?P ?O .
FILTER ( ?P =  )
} LIMIT 10 }
} 

-- 

This will delete 10 matching records at a time from the target graph,
filtering on the target predicate. 

Upscale as required and rerun as many times as required. 

If you want to do it more elegantly, put it in a PLSQL function similar
to Daniel's below and just loop it together with a select query and stop
the function when the select query finds zero count of matching
predicates. 

-- 

SELECT PCount FROM (
SPARQL
SELECT COUNT(*) as ?PCount FROM 
{
?S ?P ?O .
FILTER ( ?P =  )
} ) AS PCount... 

-- 

Make that a SELECT INTO and store the result, loop with the delete until
the result is zero. 

If you want to track progress, write the count at each iteration to the
debug log (http://docs.openlinksw.com/virtuoso/fn_dbg_printf/) and
restart Virtuoso in debug&foreground mode to display the debug
statements. 

That will eventually clean up your table. 

Regards, 

Quentin. 

Guiding Hand Solutions 

On 2016-08-17 22:54, Pantelis Natsiavas wrote: 

> Thank you for your advice Daniel. 
> 
> Actually I want to delete only the statements containing the specific 
> predicate. I don't want to delete all the triples containing the subject of 
> the predicate. As I have already said, I don't feel comfortable with the 
> DELETE queries. 
> Is my query wrong? Could you suggest the correct query? 
> 
> Kind regards, 
> Pantelis Natsiavas 
> 
> 2016-08-17 17:19 GMT+03:00 Davis, Daniel (NIH/NLM) [C] :
> 
> So, this has nothing to do with the large vector size, but just to be sure 
> the SPARQL is correct - do you wish to delete the subjects (and all their 
> triples) where the subject has the predicate, or just the predicate itself? 
> 
> As far as avoiding the maximum vector size, I think your best approach is to 
> limit the number of matches and repeat the query until there are no results, 
> maybe with a count query in-between. I have had to do similar sorts of 
> work-arounds to avoid the maximum # of results and maximum size of string 
> issues. For instance, my first attempts to export large NTriples files after 
> processing failed due to these issues. You may be able to adapt the code 
> below, but I think that a repeated deleted query limited to a # of triples 
> will be best in your case. 
> 
> Anyway, the code: 
> 
> CREATE PROCEDURE meshrdf_export(in graph_uri varchar, in file_name varchar) { 
> 
> DECLARE banner any; 
> 
> DECLARE env, ses any; 
> 
> DECLARE ses_len, max_ses_len any; 
> 
> SET isolation = 'uncommitted'; 
> 
> max_ses_len := 1000; 
> 
> -- 
> 
> -- Truncate file and write a comment line indicating the graph and datetime 
> of export. 
> 
> -- 
> 
> --no_c_escapes- 
> 
> banner := sprintf('# <%s> exported at %sn', graph_uri, datestring(now())); 
> 
> string_to_file (file_name, banner, -2); 
> 
> env := vector (0, 0, 0); 
> 
> ses := string_output (); 
> 
> FOR (SELECT * FROM (SPARQL 
> 
> define input:storage "" 
> 
> SELECT ?s ?p ?o WHERE { 
> 
> GRAPH `iri(?:graph_uri)` { 
> 
> ?s ?p ?o 
> 
> } 
> 
> } ORDER BY ?s ?p ?o) AS sub OPTION (loop)) DO { 
> 
> http_nt_triple (env, "s", "p", "o", ses); 
> 
> ses_len := length (ses); 
> 
> IF (ses_len > max_ses_len) { 
> 
> string_to_file (file_name, ses, -1); 
> 
> ses := string_output (); 
> 
> } 
> 
> } 
> 
> IF (length (ses)) { 
> 
> string_to_file (file_name, ses, -1); 
> 
> } 
> 
> } 
> 
> Dan Davis, Systems/Applications Architect (Contractor), 
> 
> Office of Computer and Communications Systems, 
> 
> National Library of Medicine, NIH 
> 
> FROM: Pantelis Natsiavas [mailto:natsia...@gmail.com] 
> SENT: Wednesday, August 17, 2016 4:36 AM
> TO: virtuoso-users 
> SUBJECT: [Virtuoso-users] Deleting large number of triples 
> 
> Hi everybody. 
> 
> I am trying to delete a large number of triples of a very big graph. The 
> graph contains 217.609.545 triples and I want to delete all the triples 
> having a specific predicate (64.884.016 triples). 
> 
> I am trying to do it through the isql-v command line interface, using the 
> command: 
> 
> SPARQL DEFINE sql:log-enable 3 
> 
> WITH  
> 
> DELETE { ?s  ?o } 
> 
> WHERE{ ?s  ?o } 
> 
> After some time (I don't know exactly how much) I got the error 
> 
> *** Error 4200

Re: [Virtuoso-users] Page set checksum error on backup_online.

2016-08-17 Thread Hugh Williams

Hi Piret,

I would suggest run a database integrity check to ensure there is no detected 
corruption in the database with the command:

backup '/dev/null';

and possibly consider performing backup-dump and restore or even crash-dump and 
restore to correct possible corruption/inconsistency in the database as 
detailed at:

http://docs.openlinksw.com/virtuoso/backup_recovery/

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

> On 17 Aug 2016, at 09:39, Piret Lattikas  wrote:
> 
> Hi all.
> 
> We setup a separate location for virtuoso backup files on a different 
> volume. First I did a full backup by stopping all processes writing to 
> virtuoso and executed commands in ISQL terminal:
> SQL> checkpoint_interval(-1);
> SQL> backup_context_clear();
> SQL> checkpoint;
> SQL> 
> backup_online('virt-inc_dump_17082106#',100,0,vector('/mnt/backups/virtuoso'));
> SQL> checkpoint_interval(60);
> 
> So I disabled checkpoints while doing backups and enabled after backups 
> finished. This worked fine.
> 
> Then I setup a cron job to execute sql file with same commands at 
> midnight every day. Not using virtuoso built in solution as needed to 
> specify different location for backup files, by default they are created 
> to the virtuoso work directory.
> 
> The midnight backup failed right before end with page checksum error. At 
> midnight the processes writing to virtuoso were running, when the backup 
> started. And virtuoso service stopped after that error. The virtuoso.log 
> about the error:
> 
> 21:00:38 Starting online backup from page 1 to 16455680, current log is: 
> /var/lib/virtuoso-opensource-7/db/virtuoso.trx
> 21:00:39 Backing up page 1
> 21:00:39 Backing up page 2
> ...
> 21:04:04 Backing up page 1645
> 21:04:05 page set checksum ck=2ace7ae3 pck=2aca7ae3 xor = 4
> 21:04:05 /usr/bin/virtuoso-t() [0xb07467]
> 21:04:05 /usr/bin/virtuoso-t() [0xb0750f]
> 21:04:05 /usr/bin/virtuoso-t() [0x4a2bd1]
> 21:04:05 /usr/bin/virtuoso-t() [0x4a87ef]
> 21:04:05 /usr/bin/virtuoso-t() [0x4a8cd8]
> 21:04:05 /usr/bin/virtuoso-t() [0x4bc9bb]
> 21:04:05 /usr/bin/virtuoso-t() [0x593e12]
> 21:04:05 /usr/bin/virtuoso-t() [0x4fd830]
> 21:04:05 /usr/bin/virtuoso-t() [0x7456c7]
> 21:04:05 /usr/bin/virtuoso-t() [0x746185]
> 21:04:05 /usr/bin/virtuoso-t() [0x746b15]
> 21:04:05 /usr/bin/virtuoso-t() [0x746c06]
> 21:04:05 /usr/bin/virtuoso-t() [0x747cc7]
> 21:04:05 /usr/bin/virtuoso-t() [0x747e0e]
> 21:04:05 /usr/bin/virtuoso-t() [0x747f86]
> 21:04:05 /usr/bin/virtuoso-t() [0x6515b8]
> 21:04:05 /usr/bin/virtuoso-t() [0x65a055]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c18b]
> 21:04:05 /usr/bin/virtuoso-t() [0x6aaecb]
> 21:04:05 /usr/bin/virtuoso-t() [0x652864]
> 21:04:05 /usr/bin/virtuoso-t() [0x65a01e]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c18b]
> 21:04:05 /usr/bin/virtuoso-t() [0x6aaecb]
> 21:04:05 /usr/bin/virtuoso-t() [0x652864]
> 21:04:05 /usr/bin/virtuoso-t() [0x653d36]
> 21:04:05 /usr/bin/virtuoso-t() [0x65f822]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c29f]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c33d]
> 21:04:05 /usr/bin/virtuoso-t() [0x6edcb8]
> 21:04:05 /usr/bin/virtuoso-t() [0x590c1e]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c1a4]
> 21:04:05 /usr/bin/virtuoso-t() [0x6a6264]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c1a4]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c455]
> 21:04:05 /usr/bin/virtuoso-t() [0x6ed7b4]
> 21:04:05 /usr/bin/virtuoso-t() [0x59319c]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c1a4]
> 21:04:05 /usr/bin/virtuoso-t() [0x6550f1]
> 21:04:05 /usr/bin/virtuoso-t() [0x655321]
> 21:04:05 /usr/bin/virtuoso-t() [0x65a085]
> 21:04:05 /usr/bin/virtuoso-t() [0x69c18b]
> 21:04:05 /usr/bin/virtuoso-t() [0x6a9007]
> 21:04:05 /usr/bin/virtuoso-t() [0x6b2f3e]
> 21:04:05 /usr/bin/virtuoso-t() [0x6b3375]
> 21:04:05 /usr/bin/virtuoso-t() [0xb097dc]
> 21:04:05 /usr/bin/virtuoso-t() [0xb15a4c]
> 21:04:05 /lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5472cf6182]
> 21:04:05 /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f547230047d]
> 21:04:05 GPF: disk.c:298 page set checksum error
> 
> Did a full backup today again with processes writing to virtuoso stopped 
> and it passed successfully.
> 
> What can cause this? Is some other session/connection causing checkpoint 
> or changing the page while backup is running? Am I missing some command 
> that virtuoso built in incremental backup is executing to disable 
> changes to pages meant for backup?
> 
> 
> Sincerely.
> 
> Piret
> 
> 
> --
> ___
> Vir

Re: [Virtuoso-users] Transaction error in lod cache endpoint

2016-08-17 Thread Hugh Williams

HI Ghislain,

I get the same error, we are looking into it …

Note the query does run against http://linkedgeodata.org/sparql 

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

> On 17 Aug 2016, at 08:44, Ghislain ATEMEZING  
> wrote:
> 
> Hi,
> 
> I've just tried this query [1] in the LOD cache 
> http://lod.openlinksw.com/sparql  .
> Unfortunately I get this error: "Virtuoso 4000X Error SR176: Transaction 
> rolled back due to previous SQL error" 
> What I am missing?
> 
> TIA
> Ghislain
> 
> 
> [1] https://gist.github.com/gatemezing/28ed5e8d01f65f32a818c433817412cd 
>  
> -- 
> ---
> "Love all, trust a few, do wrong to none" (W. Shakespeare)
> Web: http://atemezing.org 
> --
> ___
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users

smime.p7s
Description: S/MIME cryptographic signature
--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Deleting large number of triples

2016-08-17 Thread Pantelis Natsiavas

Thank you for your advice Daniel.

Actually I want to delete only the statements containing the specific
predicate. I don't want to delete all the triples containing the subject of
the predicate. As I have already said, I don't feel comfortable with the
DELETE queries.
Is my query wrong? Could you suggest the correct query?

Kind regards,
Pantelis Natsiavas

2016-08-17 17:19 GMT+03:00 Davis, Daniel (NIH/NLM) [C] :

> So, this has nothing to do with the large vector size, but just to be sure
> the SPARQL is correct - do you wish to delete the subjects (and all their
> triples) where the subject has the predicate, or just the predicate itself?
>
>
>
> As far as avoiding the maximum vector size, I think your best approach is
> to limit the number of matches and repeat the query until there are no
> results, maybe with a count query in-between.   I have had to do similar
> sorts of work-arounds to avoid the maximum # of results and maximum size of
> string issues.   For instance, my first attempts to export large NTriples
> files after processing failed due to these issues.   You may be able to
> adapt the code below, but I think that a repeated deleted query limited to
> a # of triples will be best in your case.
>
>
>
> Anyway, the code:
>
>
>
> CREATE PROCEDURE meshrdf_export(in graph_uri varchar, in file_name
> varchar) {
>
> DECLARE banner any;
>
> DECLARE env, ses any;
>
> DECLARE ses_len, max_ses_len any;
>
>
>
> SET isolation = 'uncommitted';
>
>
>
> max_ses_len := 1000;
>
>
>
> --
>
> -- Truncate file and write a comment line indicating the graph and
> datetime of export.
>
> --
>
> --no_c_escapes-
>
> banner := sprintf('# <%s> exported at %s\n', graph_uri,
> datestring(now()));
>
> string_to_file (file_name, banner, -2);
>
>
>
> env := vector (0, 0, 0);
>
> ses := string_output ();
>
>
>
> FOR (SELECT * FROM (SPARQL
>
> define input:storage ""
>
> SELECT ?s ?p ?o WHERE {
>
>   GRAPH `iri(?:graph_uri)` {
>
> ?s ?p ?o
>
>   }
>
> } ORDER BY ?s ?p ?o) AS sub OPTION (loop)) DO {
>
> http_nt_triple (env, "s", "p", "o", ses);
>
> ses_len := length (ses);
>
>
>
> IF (ses_len > max_ses_len) {
>
> string_to_file (file_name, ses, -1);
>
> ses := string_output ();
>
> }
>
> }
>
> IF (length (ses)) {
>
> string_to_file (file_name, ses, -1);
>
> }
>
> }
>
>
>
> Dan Davis, Systems/Applications Architect (Contractor),
>
> Office of Computer and Communications Systems,
>
> National Library of Medicine, NIH
>
>
>
>
>
> *From:* Pantelis Natsiavas [mailto:natsia...@gmail.com]
> *Sent:* Wednesday, August 17, 2016 4:36 AM
> *To:* virtuoso-users 
> *Subject:* [Virtuoso-users] Deleting large number of triples
>
>
>
> Hi everybody.
>
>
>
> I am trying to delete a large number of triples of a very big graph. The
> graph contains *217.609.545* triples and I want to delete all the triples
> having a specific predicate (*64.884.016* triples).
>
>
>
> I am trying to do it through the isql-v command line interface, using the
> command:
>
>
>
> SPARQL DEFINE sql:log-enable 3
>
> WITH 
>
> DELETE { ?s  ?o }
>
> WHERE{ ?s  ?o }
>
>
>
> After some time (I don't know exactly how much) I got the error
>
>
>
> *** Error 42000: [Virtuoso Driver][Virtuoso Server]FRVEC: array in for
> vectored over max vector length 200 > 100
> at line 1 of Top-Level:
>
>
>
>  I checked the virtuoso.log and I see nothing related to the specific
> error.
>
>
>
> I changed the parameters in virtuoso.ini:
>
> MaxQueryMem  = 8G  ; from 2G
> VectorSize = 1000   ; not changed
> MaxVectorSize = 200  ; from 100
> AdjustVectorSize = 1   ; from 0
>
>
>
> I am not very confident about these changes in virtuoso settings, but
> checking the http://docs.openlinksw.com/virtuoso/dbadm.html these changes
> seemed the right thing to do.
>
>
>
> I restarted the VM and retried the whole process. After one hour, the
> memory consumed by Virtuoso got around 100% and got an error:
>
> *** Error 08S01: [Virtuoso Driver]CL065: Lost connection to server
>
>
>
> Please note that from previous similar errors, I already have the
> following virtuoso.ini settings:
>
> NumberOfBuffers = 136
> MaxDirtyBuffers = 100
> ThreadCleanupInterval= 1
> ResourcesCleanupInterval = 1
>
>
>
> My questions:
>
> 1. Is there any way to improve my query in order to facilitate its
> processing? It is the first time I am doing a DELETE query and I am not
> comfortable with it.
>
> 2. Is there any way to "split" the query so that it doesn't need to handle
> all these triples at once?
>
> 3. Alternatively, is there any configuration change that might improve
> memory handling in order to handle such big queries?
>
>
>
> Kind regards,
>
> Pantelis Natsiavas
>
>
>
>
>
-

Re: [Virtuoso-users] Deleting large number of triples

2016-08-17 Thread Davis, Daniel (NIH/NLM) [C]

So, this has nothing to do with the large vector size, but just to be sure the 
SPARQL is correct - do you wish to delete the subjects (and all their triples) 
where the subject has the predicate, or just the predicate itself?

As far as avoiding the maximum vector size, I think your best approach is to 
limit the number of matches and repeat the query until there are no results, 
maybe with a count query in-between.   I have had to do similar sorts of 
work-arounds to avoid the maximum # of results and maximum size of string 
issues.   For instance, my first attempts to export large NTriples files after 
processing failed due to these issues.   You may be able to adapt the code 
below, but I think that a repeated deleted query limited to a # of triples will 
be best in your case.

Anyway, the code:

CREATE PROCEDURE meshrdf_export(in graph_uri varchar, in file_name varchar) {
DECLARE banner any;
DECLARE env, ses any;
DECLARE ses_len, max_ses_len any;

SET isolation = 'uncommitted';

max_ses_len := 1000;

--
-- Truncate file and write a comment line indicating the graph and datetime 
of export.
--
--no_c_escapes-
banner := sprintf('# <%s> exported at %s\n', graph_uri, datestring(now()));
string_to_file (file_name, banner, -2);

env := vector (0, 0, 0);
ses := string_output ();

FOR (SELECT * FROM (SPARQL
define input:storage ""
SELECT ?s ?p ?o WHERE {
  GRAPH `iri(?:graph_uri)` {
?s ?p ?o
  }
} ORDER BY ?s ?p ?o) AS sub OPTION (loop)) DO {
http_nt_triple (env, "s", "p", "o", ses);
ses_len := length (ses);

IF (ses_len > max_ses_len) {
string_to_file (file_name, ses, -1);
ses := string_output ();
}
}
IF (length (ses)) {
string_to_file (file_name, ses, -1);
}
}

Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH


From: Pantelis Natsiavas [mailto:natsia...@gmail.com]
Sent: Wednesday, August 17, 2016 4:36 AM
To: virtuoso-users 
Subject: [Virtuoso-users] Deleting large number of triples

Hi everybody.

I am trying to delete a large number of triples of a very big graph. The graph 
contains 217.609.545 triples and I want to delete all the triples having a 
specific predicate (64.884.016 triples).

I am trying to do it through the isql-v command line interface, using the 
command:

SPARQL DEFINE sql:log-enable 3
WITH 
DELETE { ?s  ?o }
WHERE{ ?s  ?o }

After some time (I don't know exactly how much) I got the error

*** Error 42000: [Virtuoso Driver][Virtuoso Server]FRVEC: array in for vectored 
over max vector length 200 > 100
at line 1 of Top-Level:

 I checked the virtuoso.log and I see nothing related to the specific error.

I changed the parameters in virtuoso.ini:
MaxQueryMem  = 8G  ; from 2G
VectorSize = 1000   ; not changed
MaxVectorSize = 200  ; from 100
AdjustVectorSize = 1   ; from 0

I am not very confident about these changes in virtuoso settings, but checking 
the http://docs.openlinksw.com/virtuoso/dbadm.html these changes seemed the 
right thing to do.

I restarted the VM and retried the whole process. After one hour, the memory 
consumed by Virtuoso got around 100% and got an error:
*** Error 08S01: [Virtuoso Driver]CL065: Lost connection to server

Please note that from previous similar errors, I already have the following 
virtuoso.ini settings:
NumberOfBuffers = 136
MaxDirtyBuffers = 100
ThreadCleanupInterval= 1
ResourcesCleanupInterval = 1

My questions:
1. Is there any way to improve my query in order to facilitate its processing? 
It is the first time I am doing a DELETE query and I am not comfortable with it.
2. Is there any way to "split" the query so that it doesn't need to handle all 
these triples at once?
3. Alternatively, is there any configuration change that might improve memory 
handling in order to handle such big queries?

Kind regards,
Pantelis Natsiavas


--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Page set checksum error on backup_online.

2016-08-17 Thread Piret Lattikas

Hi all.

We setup a separate location for virtuoso backup files on a different 
volume. First I did a full backup by stopping all processes writing to 
virtuoso and executed commands in ISQL terminal:
SQL> checkpoint_interval(-1);
SQL> backup_context_clear();
SQL> checkpoint;
SQL> 
backup_online('virt-inc_dump_17082106#',100,0,vector('/mnt/backups/virtuoso'));
SQL> checkpoint_interval(60);

So I disabled checkpoints while doing backups and enabled after backups 
finished. This worked fine.

Then I setup a cron job to execute sql file with same commands at 
midnight every day. Not using virtuoso built in solution as needed to 
specify different location for backup files, by default they are created 
to the virtuoso work directory.

The midnight backup failed right before end with page checksum error. At 
midnight the processes writing to virtuoso were running, when the backup 
started. And virtuoso service stopped after that error. The virtuoso.log 
about the error:

21:00:38 Starting online backup from page 1 to 16455680, current log is: 
/var/lib/virtuoso-opensource-7/db/virtuoso.trx
21:00:39 Backing up page 1
21:00:39 Backing up page 2
...
21:04:04 Backing up page 1645
21:04:05 page set checksum ck=2ace7ae3 pck=2aca7ae3 xor = 4
21:04:05 /usr/bin/virtuoso-t() [0xb07467]
21:04:05 /usr/bin/virtuoso-t() [0xb0750f]
21:04:05 /usr/bin/virtuoso-t() [0x4a2bd1]
21:04:05 /usr/bin/virtuoso-t() [0x4a87ef]
21:04:05 /usr/bin/virtuoso-t() [0x4a8cd8]
21:04:05 /usr/bin/virtuoso-t() [0x4bc9bb]
21:04:05 /usr/bin/virtuoso-t() [0x593e12]
21:04:05 /usr/bin/virtuoso-t() [0x4fd830]
21:04:05 /usr/bin/virtuoso-t() [0x7456c7]
21:04:05 /usr/bin/virtuoso-t() [0x746185]
21:04:05 /usr/bin/virtuoso-t() [0x746b15]
21:04:05 /usr/bin/virtuoso-t() [0x746c06]
21:04:05 /usr/bin/virtuoso-t() [0x747cc7]
21:04:05 /usr/bin/virtuoso-t() [0x747e0e]
21:04:05 /usr/bin/virtuoso-t() [0x747f86]
21:04:05 /usr/bin/virtuoso-t() [0x6515b8]
21:04:05 /usr/bin/virtuoso-t() [0x65a055]
21:04:05 /usr/bin/virtuoso-t() [0x69c18b]
21:04:05 /usr/bin/virtuoso-t() [0x6aaecb]
21:04:05 /usr/bin/virtuoso-t() [0x652864]
21:04:05 /usr/bin/virtuoso-t() [0x65a01e]
21:04:05 /usr/bin/virtuoso-t() [0x69c18b]
21:04:05 /usr/bin/virtuoso-t() [0x6aaecb]
21:04:05 /usr/bin/virtuoso-t() [0x652864]
21:04:05 /usr/bin/virtuoso-t() [0x653d36]
21:04:05 /usr/bin/virtuoso-t() [0x65f822]
21:04:05 /usr/bin/virtuoso-t() [0x69c29f]
21:04:05 /usr/bin/virtuoso-t() [0x69c33d]
21:04:05 /usr/bin/virtuoso-t() [0x6edcb8]
21:04:05 /usr/bin/virtuoso-t() [0x590c1e]
21:04:05 /usr/bin/virtuoso-t() [0x69c1a4]
21:04:05 /usr/bin/virtuoso-t() [0x6a6264]
21:04:05 /usr/bin/virtuoso-t() [0x69c1a4]
21:04:05 /usr/bin/virtuoso-t() [0x69c455]
21:04:05 /usr/bin/virtuoso-t() [0x6ed7b4]
21:04:05 /usr/bin/virtuoso-t() [0x59319c]
21:04:05 /usr/bin/virtuoso-t() [0x69c1a4]
21:04:05 /usr/bin/virtuoso-t() [0x6550f1]
21:04:05 /usr/bin/virtuoso-t() [0x655321]
21:04:05 /usr/bin/virtuoso-t() [0x65a085]
21:04:05 /usr/bin/virtuoso-t() [0x69c18b]
21:04:05 /usr/bin/virtuoso-t() [0x6a9007]
21:04:05 /usr/bin/virtuoso-t() [0x6b2f3e]
21:04:05 /usr/bin/virtuoso-t() [0x6b3375]
21:04:05 /usr/bin/virtuoso-t() [0xb097dc]
21:04:05 /usr/bin/virtuoso-t() [0xb15a4c]
21:04:05 /lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5472cf6182]
21:04:05 /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f547230047d]
21:04:05 GPF: disk.c:298 page set checksum error

Did a full backup today again with processes writing to virtuoso stopped 
and it passed successfully.

What can cause this? Is some other session/connection causing checkpoint 
or changing the page while backup is running? Am I missing some command 
that virtuoso built in incremental backup is executing to disable 
changes to pages meant for backup?


Sincerely.

Piret


--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Deleting large number of triples

2016-08-17 Thread Pantelis Natsiavas

Hi everybody.

I am trying to delete a large number of triples of a very big graph. The
graph contains *217.609.545* triples and I want to delete all the triples
having a specific predicate (*64.884.016* triples).

I am trying to do it through the isql-v command line interface, using the
command:

SPARQL DEFINE sql:log-enable 3

WITH 

DELETE { ?s  ?o }

WHERE{ ?s  ?o }


After some time (I don't know exactly how much) I got the error

*** Error 42000: [Virtuoso Driver][Virtuoso Server]FRVEC: array in for
> vectored over max vector length 200 > 100
> at line 1 of Top-Level:


 I checked the virtuoso.log and I see nothing related to the specific error.

I changed the parameters in virtuoso.ini:

> MaxQueryMem  = 8G  ; from 2G
> VectorSize = 1000   ; not changed
> MaxVectorSize = 200  ; from 100
> AdjustVectorSize = 1   ; from 0


I am not very confident about these changes in virtuoso settings, but
checking the http://docs.openlinksw.com/virtuoso/dbadm.html these changes
seemed the right thing to do.

I restarted the VM and retried the whole process. After one hour, the
memory consumed by Virtuoso got around 100% and got an error:

> *** Error 08S01: [Virtuoso Driver]CL065: Lost connection to server


Please note that from previous similar errors, I already have the following
virtuoso.ini settings:

> NumberOfBuffers = 136
> MaxDirtyBuffers = 100
> ThreadCleanupInterval= 1
> ResourcesCleanupInterval = 1


My questions:
1. Is there any way to improve my query in order to facilitate its
processing? It is the first time I am doing a DELETE query and I am not
comfortable with it.
2. Is there any way to "split" the query so that it doesn't need to handle
all these triples at once?
3. Alternatively, is there any configuration change that might improve
memory handling in order to handle such big queries?

Kind regards,
Pantelis Natsiavas
--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Transaction error in lod cache endpoint

2016-08-17 Thread Ghislain ATEMEZING

Hi,

I've just tried this query [1] in the LOD cache
http://lod.openlinksw.com/sparql .
Unfortunately I get this error: "Virtuoso 4000X Error SR176: Transaction
rolled back due to previous SQL error"
What I am missing?

TIA
Ghislain


[1] https://gist.github.com/gatemezing/28ed5e8d01f65f32a818c433817412cd
-- 
---
"Love all, trust a few, do wrong to none" (W. Shakespeare)
Web: http://atemezing.org
--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Deleting large number of triples

Re: [Virtuoso-users] Page set checksum error on backup_online.

Re: [Virtuoso-users] Transaction error in lod cache endpoint

Re: [Virtuoso-users] Deleting large number of triples

Re: [Virtuoso-users] Deleting large number of triples

[Virtuoso-users] Page set checksum error on backup_online.

[Virtuoso-users] Deleting large number of triples

[Virtuoso-users] Transaction error in lod cache endpoint

8 matches

Site Navigation

Mail list logo

Footer information