Hi Roland, If you can provide your dataset to test with that would be best as that is what you are having the issue with ...
We will also test with the rdf/xml version as that should load also unless there are genuine issue with the datasets in this form which the creators of the dataset should fix ... Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers On 29 Apr 2013, at 16:58, Roland <[email protected]> wrote: > Hi Hugh, > > No, there were no errors reported in virtuoso.log. > In the meantime I have repeated the same procedure on stable/vos6 where all > went well; all data loaded and available. > The dataset I am talking about is VIAF RDF data, publicly available here [1]. > I have preprocessed this set to erase some errors and converted the data to > n-triples. If you want to use it I could send you the converted dataset. > > Thanks, > Roland > > [1] http://viaf.org/viaf/data/viaf-20130417-clusters-rdf.xml.gz > > On 04/28/2013 01:57 PM, Hugh Williams wrote: >> Hi Roland, >> >> Are any error reported in the "virtuoso.log" file when the load is in >> progress ? Also, is this a publicly available datasets your are loading that >> we could try locally ? >> >> Best Regards >> Hugh Williams >> Professional Services >> OpenLink Software, Inc. // http://www.openlinksw.com/ >> Weblog -- http://www.openlinksw.com/blogs/ >> LinkedIn -- http://www.linkedin.com/company/openlink-software/ >> Twitter -- http://twitter.com/OpenLink >> Google+ -- http://plus.google.com/100570109519069333827/ >> Facebook -- http://www.facebook.com/OpenLinkSoftware >> Universal Data Access, Integration, and Management Technology Providers >> >> On 23 Apr 2013, at 22:39, Roland <[email protected]> wrote: >> >>> Hi Hugh, >>> >>> That saved me precious time. In the meantime I have upgraded to VOS7 and >>> try to load the large set using Bulk Loader. >>> But it is not working. See log below, this is what I get and Virtuoso >>> slowly dies after accumulating about 1GB in db, the service is not >>> available any more, I have to kill the process. >>> >>> After killing the process I restore the initial situation, just >>> virtuoso.ini and restart the server that creates a new db. Do I have to >>> restore some more, or better; how can I get it to stop waiting and proceed >>> loading? >>> >>> Thanks, >>> Roland >>> >>> >>> SQL> ld_dir ('/home/roland/Documents/metamatter/bieb/datalab/', >>> 'output.nt', 'http://viaf.org/'); >>> >>> Done. -- 4 msec. >>> SQL> rdf_loader_run(); >>> 23:26:24 PL LOG: Loader started >>> 23:26:44 Write wait on column page 23801. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:26:44 Write wait on column page 30567. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:26:44 Write wait on column page 30605. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:26:45 Write wait on column page 30606. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:26:45 Write wait on column page 31262. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:26:48 Write wait on column page 2051. Waits should be on the index leaf >>> page, except when col page is held for read by background write >>> 23:26:54 Write wait on column page 2823. Waits should be on the index leaf >>> page, except when col page is held for read by background write >>> 23:26:57 Write wait on column page 39246. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:06 Write wait on column page 4867. Waits should be on the index leaf >>> page, except when col page is held for read by background write >>> 23:27:09 missed delete of name id cache >>> http://viaf.org/viaf/sourceID/LC%7Cn+2001057591# 0d >>> 23:27:14 Write wait on column page 37357. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:14 Write wait on column page 50069. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:15 Write wait on column page 30589. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:15 Write wait on column page 50730. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:22 Write wait on column page 50807. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:26 Write wait on column page 50751. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:28 Write wait on column page 2414. Waits should be on the index leaf >>> page, except when col page is held for read by background write >>> 23:27:46 Write wait on column page 2819. Waits should be on the index leaf >>> page, except when col page is held for read by background write >>> 23:27:46 Write wait on column page 50090. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:46 Write wait on column page 79639. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:27:59 Write wait on column page 97536. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:28:02 Write wait on column page 86017. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:28:06 Write wait on column page 95255. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:28:19 missed delete of name id cache 0d >>> 23:28:20 Write wait on column page 30626. Waits should be on the index >>> leaf page, except when col page is held for read by background write >>> 23:28:23 Write wait on column page 2568. Waits should be on the index leaf >>> page, except when col page is held for read by background write >>> >>> >>> >>> On 04/22/2013 02:33 PM, Hugh Williams wrote: >>>> Hi Roland, >>>> >>>> You should use the Virtuoso RDF Bulik Loader for loading such large >>>> datasets as detailed at: >>>> >>>> >>>> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader >>>> >>>> From the error you report it would appear a checkpoint is being performed >>>> in the middle of your load, so checking should be disabled which is what >>>> the bulkloader does automatically and also turns of transaction logging >>>> with log_enable(3) to prevent the log from running out of memory, which >>>> once again the bulk loader does automatically and is probably the next >>>> error you would hit ... >>>> >>>> Best Regards >>>> Hugh Williams >>>> Professional Services >>>> OpenLink Software, Inc. // http://www.openlinksw.com/ >>>> Weblog -- http://www.openlinksw.com/blogs/ >>>> LinkedIn -- http://www.linkedin.com/company/openlink-software/ >>>> Twitter -- http://twitter.com/OpenLink >>>> Google+ -- http://plus.google.com/100570109519069333827/ >>>> Facebook -- http://www.facebook.com/OpenLinkSoftware >>>> Universal Data Access, Integration, and Management Technology Providers >>>> >>>> On 22 Apr 2013, at 13:07, Roland Cornelissen <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I am trying to load a large dataset (~50GB) and bumped into: >>>>> >>>>> *** Error 40001: [Virtuoso Driver][Virtuoso Server]SR325: Transaction >>>>> aborted due to a database checkpoint or database-wide atomic operation. >>>>> Please retry transaction >>>>> at line 10 of Top-Level: >>>>> ttlp_mt (file_to_string_output ('/usr/viaf/viaf_fixed.nt'), '', >>>>> 'http://viaf.org/', 255) >>>>> >>>>> I have 16GB memory in my box, and set: >>>>> NumberOfBuffers = 680000 >>>>> MaxDirtyBuffers = 500000 >>>>> and: >>>>> MaxCheckpointRemap = 2000 >>>>> >>>>> So i guess the MaxCheckpointRemap is the problem. At what value should I >>>>> set it? >>>>> Here [1] it says 1/4 of the database size in pages, 8K per page. >>>>> So when I calculate with a DB of 50GB that would be 1562500, is that >>>>> correct? >>>>> >>>>> Am I missing something else I should check before kicking of another run? >>>>> >>>>> Thanks, >>>>> Roland >>>>> >>>>> [1] http://www.openlinksw.com/OdbcRails/main/Main/VirtRDFPerformanceTuning >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Precog is a next-generation analytics platform capable of advanced >>>>> analytics on semi-structured data. The platform includes APIs for building >>>>> apps and a phenomenal toolset for data science. Developers can use >>>>> our toolset for easy data analysis & visualization. Get a free account! >>>>> http://www2.precog.com/precogplatform/slashdotnewsletter >>>>> _______________________________________________ >>>>> Virtuoso-users mailing list >>>>> [email protected] >>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users >>>> >>> >>> ------------------------------------------------------------------------------ >>> Try New Relic Now & We'll Send You this Cool Shirt >>> New Relic is the only SaaS-based application performance monitoring service >>> that delivers powerful full stack analytics. Optimize and monitor your >>> browser, app, & servers with just a few lines of code. Try New Relic >>> and get this awesome Nerd Life shirt! >>> http://p.sf.net/sfu/newrelic_d2d_apr_______________________________________________ >>> Virtuoso-users mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users >> > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! > http://p.sf.net/sfu/newrelic_d2d_apr_______________________________________________ > Virtuoso-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/virtuoso-users
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ AlienVault Unified Security Management (USM) platform delivers complete security visibility with the essential security capabilities. Easily and efficiently configure, manage, and operate all of your security controls from a single console and one unified framework. Download a free trial. http://p.sf.net/sfu/alienvault_d2d
_______________________________________________ Virtuoso-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/virtuoso-users
