hello Hugh,

some news :  we can now reproduce the issue with a significant amount of triples !

I uploaded a sample file to reproduce the "Broken index DB_DBA_RDF_IRI_UNQC_RI_ID and Broken index RDF_IRI" on +backup-dump, here

https://github.com/openlink/virtuoso-opensource/files/1625829/gsl20171003000001.zip

It seems to much "IRI is not in Unicode Normal Form" triples will silently break the RDF_IRI and DB_DBA_RDF_IRI_UNQC_RI_ID indices.


Thanks


Le 22/12/2017 à 18:21, Hugh Williams a écrit :
Hi Thomas,

This works of me with Virtuoso open source and commercial builds I have on Mac OS X. What is the version of the Virtuoso Windows open source binary you are using ( output of "virtuoso-t -?” command ) ?

Below is the output from my Open source test:

De-iMac-232:database hwilliams$ ~/Downloads/isql 1111
Connected to OpenLink Virtuoso
Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> ld_dir('.', 'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');

Done. -- 5 msec.
SQL> select * from load_list;
ll_file               ll_graph                                     ll_state    ll_started         ll_done              ll_host     ll_work_time ll_error VARCHAR NOT NULL               VARCHAR                                     INTEGER     TIMESTAMP         TIMESTAMP            INTEGER     INTEGER     VARCHAR
_______________________________________________________________________________

./badiritest.nt http://hub.abes.fr/journals/wiley2011/20170921                                   0           NULL       NULL                 NULL        NULL        NULL

1 Rows. -- 1 msec.
SQL> rdf_loader_run();

Done. -- 8 msec.
SQL> select * from load_list;               ll_file                                     ll_graph ll_state    ll_started           ll_done ll_host     ll_work_time  ll_error VARCHAR NOT NULL               VARCHAR                                     INTEGER     TIMESTAMP         TIMESTAMP            INTEGER     INTEGER     VARCHAR
_______________________________________________________________________________

./badiritest.nt http://hub.abes.fr/journals/wiley2011/20170921                                   2           2017.12.22 17:16.5 309717000  2017.12.22 17:16.5 313513000  0 NULL        NULL

1 Rows. -- 1 msec.
SQL> sparql select * from <http://hub.abes.fr/journals/wiley2011/20170921> where {?s ?p ?o};
s                     p                                           o
LONG VARCHAR                 LONG VARCHAR                                       LONG VARCHAR
_______________________________________________________________________________

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://xmlns.com/foaf/0.1/Person

1 Rows. -- 1 msec.
SQL> checkpoint;

Done. -- 44 msec.
SQL> shutdown;
De-iMac-232:database hwilliams$ ~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t +backup-dump

Fri Dec 22 2017
17:16:44 { Loading plugin 1: Type `plain', file `wikiv' in `/Users/hwilliams/vos-7-install/hosting'
17:16:44   FAILED  plugin 1: Unable to locate file }
17:16:44 { Loading plugin 2: Type `plain', file `mediawiki' in `/Users/hwilliams/vos-7-install/hosting'
17:16:44   FAILED  plugin 2: Unable to locate file }
17:16:44 { Loading plugin 3: Type `plain', file `creolewiki' in `/Users/hwilliams/vos-7-install/hosting'
17:16:44   FAILED  plugin 3: Unable to locate file }
17:16:44 OpenLink Virtuoso Universal Server
17:16:44 Version 07.20.3217-pthreads for Darwin as of Oct 31 2017
17:16:44 uses parts of OpenSSL, PCRE, Html Tidy
17:16:44 Database version 3126
17:16:44 SQL Optimizer enabled (max 1000 layouts)
17:16:45 Compiler unit is timed at 0.000284 msec
17:16:46 Roll forward started
17:16:46 Roll forward complete
17:16:47 PL LOG: Can't get list of vad packages in /Users/hwilliams/vos-7-install/vad/
17:16:47 Database dump started
17:16:47 Checkpoint finished, log reused
17:16:47 Dumping the schema tables
17:16:47 Dumping the registry
17:16:47 Dumping the schema done
17:16:47 Database dump complete
17:16:47 Server exiting
De-iMac-232:database hwilliams$ rm virtuoso.db
De-iMac-232:database hwilliams$ ~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t -R -f

Fri Dec 22 2017
17:17:05 { Loading plugin 1: Type `plain', file `wikiv' in `/Users/hwilliams/vos-7-install/hosting'
17:17:05   FAILED  plugin 1: Unable to locate file }
17:17:05 { Loading plugin 2: Type `plain', file `mediawiki' in `/Users/hwilliams/vos-7-install/hosting'
17:17:05   FAILED  plugin 2: Unable to locate file }
17:17:05 { Loading plugin 3: Type `plain', file `creolewiki' in `/Users/hwilliams/vos-7-install/hosting'
17:17:05   FAILED  plugin 3: Unable to locate file }
17:17:05 OpenLink Virtuoso Universal Server
17:17:05 Version 07.20.3217-pthreads for Darwin as of Oct 31 2017
17:17:05 uses parts of OpenSSL, PCRE, Html Tidy
17:17:05 SQL Optimizer enabled (max 1000 layouts)
17:17:05 Roll forward started
17:17:06 Roll forward complete
17:17:06 Checkpoint started
17:17:06 Checkpoint finished, log reused
17:17:06 Checkpoint started
17:17:06 Checkpoint finished, log reused
17:17:06 Server exiting
De-iMac-232:database hwilliams$ ~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t
De-iMac-232:database hwilliams$ tail -f virtuoso.log
17:17:14 uses parts of OpenSSL, PCRE, Html Tidy
17:17:14 Database version 3126
17:17:14 SQL Optimizer enabled (max 1000 layouts)
17:17:15 Compiler unit is timed at 0.000266 msec
17:17:16 Roll forward started
17:17:16     3 transactions, 185 bytes replayed (100 %)
17:17:16 Roll forward complete
17:17:16 PL LOG: Can't get list of vad packages in /Users/hwilliams/vos-7-install/vad/
17:17:16 Checkpoint started
17:17:16 Checkpoint finished, log reused
17:17:18 HTTP/WebDAV server online at 8890
17:17:18 Server online at 1111 (pid 10513)
^C
De-iMac-232:database hwilliams$ ~/Downloads/isql 1111
Connected to OpenLink Virtuoso
Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> sparql select * from <http://hub.abes.fr/journals/wiley2011/20170921> where {?s ?p ?o};
s                     p                                           o
LONG VARCHAR                 LONG VARCHAR                                       LONG VARCHAR
_______________________________________________________________________________

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://xmlns.com/foaf/0.1/Person

1 Rows. -- 1 msec.
SQL>

De-iMac-232:database hwilliams$ ~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t -?
Virtuoso Open Source Edition (Column Store) (multi threaded)
Version 7.2.5-dev.3217-pthreads as of Oct 31 2017
Compiled for Darwin (x86_64-apple-darwin15.6.0)
Copyright (C) 1998-2017 OpenLink Softwar

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.      // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 22 Dec 2017, at 15:56, Thomas Michaux <mich...@abes.fr <mailto:mich...@abes.fr>> wrote:

Hello,

we may have found a case where bulk load doesn't report error but invalid data has a side effect on the "backup-dump" process, step to reproduce :

1) - in an empty database : use

ld_dir('e:/virtuosoBACKUP/test', 'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');

with these data in file badiritest.nt

<http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana\u00A0c\u00A0g\u00A0a> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .

rdf_loader_run();

checkpoint;

shutdown;

2) then you can't success to process the "backup-dump" procedure :

virtuoso-t +foreground +configfile "c:\Program Files\OpenLink Software\VOS7\virtuoso-opensource\database\virtuoso.ini" +backup-dump


even if data is really in store :

SQL> SPARQL select ?s ?p ?o from <http://hub.abes.fr/journals/wiley2011/20170921> {?s ?p ?o};
s p o
LONG VARCHAR LONG VARCHAR LONG VARCHAR
_______________________________________________________________________________

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://xmlns.com/foaf/0.1/Person

1 Rows. -- 0 msec.


tested on windows , need to reproduce under linux.


Thanks

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://Slashdot.org>! http://sdm.link/slashdot_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net <mailto:Virtuoso-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/virtuoso-users


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Reply via email to