-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Hugh,
Thanks for the recommendations. However, getting hold of a 8GB 64-bit Linux machine or anything of the like is not practical (or affordable) for me. A public SPARQL endpoint may be the solution, if I can only get around the restriction of 1000 result triples. Regards, Alex Hugh Williams wrote: > Hi Alex, > > I would definitely say your Windows Vista 32bit machine is under > resourced for hosting a fully loaded DBpedia instance in Virtuoso, being > basically a desktop spec machine. I would recommend a higher spec. > machine and a Server based OS ie Windows Server 2003 or 2008 or Linux > which is what most would typically setup DBpedia on. You could also > consider what DBpedia datasets are required for you need to load a > minimum set requiring less resources. > > Assuming you are using the settings in the default dbpeda.ini file the > NumberOfBuffers parameter is set to 100000 meaning some 800MB of memory > will be used by the Virtuoso Server, and it that much memory is not > available when the server attempts to allocate it this would account for > the out of memory errors you have been receiving. > > Personally I would recommend if possible you setup a 64bit Linux or > Windows Server machine with at least 8GB of RAM as receommended. You > also need a new set of binaries with the fixes for the "Can not set NULL > to not nullable column 'DB.DBA.RDF_QUAD.O" errors. If you intend to use > a Linux build we have a patched archive snap shot that can be made > available requiring that you compile and build the binaries. On Windows > you would need to wait for us to schedule a new set of open source > binaries for this OS. > > Best Regards > Hugh Williams > Professional Services > OpenLink Software > Web: http://www.openlinksw.com > Support: http://support.openlinksw.com > Forums: http://boards.openlinksw.com/support > > > > On 20 Sep 2009, at 01:39, Alex wrote: > > Hi Hugh, > > Yeah, on second thought I was rather ignoring those errors mainly > because I didn't want to have to deal with them! > > Here are the details of the machine on which I am installing: > 1. Windows Vista Ultimate 32-bit > 2. 2GB physical RAM > 3. Intel(R) Core(TM)2 CPU @ 2.13 GHz > > Now that you say the minimum for Amazon EC2 hosting is a machine with > 8GB memory, I'm starting to see that the specs may be a problem. It > didn't actually cross my mind initially, since this machine was > relatively high-end for 3 years ago when I bought it. Optimism again, > perhaps... > > Finally, here is the version info I got from running `virtuoso-t --help`. > > Virtuoso Open Source Edition (multi threaded) > Version 5.0.11.3039-threads as of Apr 21 2009 > Compiled for 32 Bit Windows Operating Environments > Copyright (C) 1999-2009 OpenLink Software > > Anyway, thanks for continued speedy replies and sticking with me on all > the issues. It has certainly not been a straightforward matter to > troubleshoot, and I'm just hoping there will be some fruits to these > labours. If it helps at all, I should probably state that my overall aim > is simply to set up a DBpedia SPARQL endpoint that can serve over 1000 > results per query (the current restriction on the DBpedia.org endpoint). > > Regards, > Alex > > Hugh Williams wrote: >>>> Hi Alex, >>>> >>>> I would say the "out of memory" errors in the installer log are a bit >>>> more than minor, as the system would appear not to have the resources >>>> for hosting the datasets being loaded. >>>> >>>> What are the details of the machine you are attempting to host your >>>> DBpedia instance on: >>>> >>>> 1. What is OS name and bit'ness ? You mentioned using cygwin, thus I >>>> presume you are on Windows ? What windows variant and is it 32 or 64 >>>> bit ? >>>> 2. What is available memory consumption ? I believe you said 2GB in a >>>> previous post ? >>>> 3. What is type and speed of Processor ? >>>> >>>> For hosting the DBpedia datasets on Amazon EC2 the 3.2 datasets we have >>>> currently recommend a minimum Linux x86_64 machine with 8GB memory. >>>> >>>> I also not a few "Can not set NULL to not nullable column >>>> 'DB.DBA.RDF_QUAD.O" errors in your load log which is an issue that has >>>> been fixed in our latest builds thus you would benefit from an updated >>>> build also, as those datasets have probably failed to load. >>>> >>>> Please confirm the version number of build date of the server you are >>>> using. >>>> >>>> Best Regards >>>> Hugh Williams >>>> Professional Services >>>> OpenLink Software >>>> Web: http://www.openlinksw.com >>>> Support: http://support.openlinksw.com >>>> Forums: http://boards.openlinksw.com/support >>>> >>>> >>>> >>>> On 19 Sep 2009, at 18:31, Alex wrote: >>>> >>>> Hi Hugh, >>>> >>>> I suspected you'd ask for them... no idea how pertinent they are to the >>>> issue though. I've attached the console log and the load log that was >>>> generated. >>>> >>>> Cheers, >>>> Alex >>>> >>>> Hugh Williams wrote: >>>>>>> Hi Alex, >>>>>>> >>>>>>> What are the minor errors you indicate having occurred ? >>>>>>> >>>>>>> The SPARQL query timeout is set to 60 by default, thus you might >>>>>>> want to >>>>>>> try increasing the MaxQueryExecutionTime paramter in the [SPARQL] >>>>>>> section of the dbpedia.ini file which controls the timeouts and >>>>>>> restart >>>>>>> the server. >>>>>>> >>>>>>> Should the problems persist please try enabling tracing on the server >>>>>>> as detailed at: >>>>>>> >>>>>>> http://docs.openlinksw.com/virtuoso/fn_trace_on.html >>>>>>> >>>>>>> This will write trace info to the dbpedia.log file, please provide >>>>>>> this >>>>>>> and the dbpedia.ini file for analysis. >>>>>>> >>>>>>> You should also try running the same query via the Virtuoso isql >>>>>>> problem, preceded with the "sparql" keyword to direct it to the >>>>>>> SPARQL >>>>>>> processor ... >>>>>>> >>>>>>> Best Regards >>>>>>> Hugh Williams >>>>>>> Professional Services >>>>>>> OpenLink Software >>>>>>> Web: http://www.openlinksw.com >>>>>>> Support: http://support.openlinksw.com >>>>>>> Forums: http://boards.openlinksw.com/support >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 19 Sep 2009, at 16:19, Alex wrote: >>>>>>> >>>>>>> Hi Hugh, >>>>>>> >>>>>>> The installation seems to have completed fine now (a few minor >>>>>>> errors), >>>>>>> and I've just tested the SPARQL endpoint in the browser, but the >>>>>>> server >>>>>>> seems to be complaining. >>>>>>> >>>>>>> S1T00 Error SR171: Transaction timed out >>>>>>> >>>>>>> SPARQL query: >>>>>>> ... >>>>>>> >>>>>>> I received this message for the recommended sample query in >>>>>>> README.txt, >>>>>>> as well as several others. Any suggestions please? >>>>>>> >>>>>>> Regards, >>>>>>> Alex > Started. > Checking files > load_nt.sh : present > dbpedia_dav.vad : present > dbpedia_post.sql : present > dbpedia-ontology.owl : present > umbel_class_hierarchy_v071.n3 : present > umbel_abstract_concepts.n3 : present > umbel_external_ontologies_linkage.n3 : present > yago-class-hierarchy_en.nt : present > umbel_subject_concepts.n3 : present > opencyc-2008-06-10.owl : present > opencyc-2008-06-10-readable.owl : present > Installing dbpedia_dav.vad > Starting... > Logging into: load_nt.sh.log > ======================================= > Loading started. > ======================================= > Loading data/externallinks_en.nt (3653630 lines) 13:25:50 > Retryng from line 1235724 > Loaded. > Loading data/geo_en.nt (1110708 lines) 13:41:43 > Loaded. > Loading data/homepage_en.nt (65178 lines) 13:47:38 > Loaded. > Loading data/image_en.nt (2775360 lines) 13:49:16 > Loaded. > Loading data/infobox-mappingbased-loose.nt (8238555 lines) 14:11:35 > Loaded. > Loading data/infoboxproperties_en.nt (98244 lines) 15:07:53 > Loaded. > Loading data/infobox_en.nt (39855823 lines) 15:12:25 > Loaded. > Loading data/instancetype_en.nt (3790668 lines) 16:04:50 > Loaded. > Loading data/longabstract_en.nt (2788812 lines) 16:31:05 > Loaded. > Loading data/pagelinks_en.nt (77092350 lines) 19:11:57 > Retryng from line 16209818 > Loaded. > Loading data/persondata_en.nt (118002 lines) 14:44:55 > Loaded. > Loading data/redirect_en.nt (3494567 lines) 15:14:27 > Loaded. > Loading data/shortabstract_en.nt (2787499 lines) 16:36:46 > Loaded. > Loading data/skoscategories_en.nt (1844055 lines) 23:28:05 > Loaded. > Loading data/wikipage_en.nt (3201391 lines) 00:35:10 > Loaded. > Loading data/yagolink_en.nt (1889985 lines) 02:44:03 > Loaded. > Loading data/yago_en.nt (4645108 lines) 08:32:36 > Loaded. > ======================================= > Final checkpoint. > ======================================= > Check bad.nt file for skipped triples. > ======================================= > Running post-install scripts > PASSED: RDF_OBJ_FT_RULE_ADD : STATE=OK MESSAGE=OK > PASSED: VT_INC_INDEX_DB_DBA_RDF_OBJ : STATE=OK MESSAGE=OK > PASSED: RDF_QUAD_POGS : STATE=OK MESSAGE=OK > PASSED: RDF_QUAD_PSOG : STATE=OK MESSAGE=OK > PASSED: RDF_QUAD_SOPG : STATE=OK MESSAGE=OK > PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK > PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK > PASSED: DBpedia inference : STATE=OK MESSAGE=OK > ***FAILED: Load umbel_class_hierarchy_v071 : STATE=23000 > MESSAGE=[OpenLink][Virt > uoso ODBC Driver][Virtuoso Server]SR133: Can not set NULL to not > nullable column > 'DB.DBA.RDF_QUAD.O' > PASSED: Load umbel_subject_concepts : STATE=OK MESSAGE=OK > PASSED: Load umbel_abstract_concepts : STATE=OK MESSAGE=OK > PASSED: Load umbel_external_ontologies_linkage : STATE=OK MESSAGE=OK > PASSED: UMBEL inference : STATE=OK MESSAGE=OK > PASSED: Load yago-class-hierarchy_en : STATE=OK MESSAGE=OK > PASSED: Yago inference : STATE=OK MESSAGE=OK > PASSED: Load opencyc_class_hierarchy : STATE=OK MESSAGE=OK > ***FAILED: Opencyc inference : STATE=40006 MESSAGE=[OpenLink][Virtuoso > ODBC Driv > er][Virtuoso Server]SR337: Transaction aborted because the server is > out of memo > ry > ***FAILED: opencyc-2008-06-10-readable : STATE=40006 > MESSAGE=[OpenLink][Virtuoso > ODBC Driver][Virtuoso Server]SR337: Transaction aborted because the > server is o > ut of memory > ***FAILED: DBpedia property inference : STATE=40006 > MESSAGE=[OpenLink][Virtuoso > ODBC Driver][Virtuoso Server]SR337: Transaction aborted because the > server is ou > t of memory > =============================== > Install finished. > Sat Sep 19 06:39:55 GMTDT 2009 > =============================== > The MAIL_SERVER environment variable is not set, e.g. export > MAIL_SERVER=mail.do > main:25 > =============================== > Install started > Tue Sep 15 13:25:15 GMTDT 2009 > =============================== > Checking for VOS setup > Starting Virtuoso server, please wait ... > Cannot start Virtuoso server, please consult dbpedia.log file > Started. > Checking files > load_nt.sh : present > dbpedia_dav.vad : present > dbpedia_post.sql : present > dbpedia-ontology.owl : present > umbel_class_hierarchy_v071.n3 : present > umbel_abstract_concepts.n3 : present > umbel_external_ontologies_linkage.n3 : present > yago-class-hierarchy_en.nt : present > umbel_subject_concepts.n3 : present > opencyc-2008-06-10.owl : present > opencyc-2008-06-10-readable.owl : present > Installing dbpedia_dav.vad > 00000 > > No errors detected > 00000 > > Installation of "DBpedia" is complete. > 00000 > > Now making a final checkpoint. > 00000 > > Final checkpoint is made. > 00000 > > SUCCESS >>>>> > ======================================= > Loading started. > ======================================= > Loading data/externallinks_en.nt (3653630 lines) 13:25:50 > Retryng from line 1235724 > Loading data/geo_en.nt (1110708 lines) 13:41:43 > Loading data/homepage_en.nt (65178 lines) 13:47:38 > Loading data/image_en.nt (2775360 lines) 13:49:16 > Loading data/infobox-mappingbased-loose.nt (8238555 lines) 14:11:35 > Loading data/infoboxproperties_en.nt (98244 lines) 15:07:53 > Loading data/infobox_en.nt (39855823 lines) 15:12:25 > Loading data/instancetype_en.nt (3790668 lines) 16:04:50 > Loading data/longabstract_en.nt (2788812 lines) 16:31:05 > Loading data/pagelinks_en.nt (77092350 lines) 19:11:57 > Retryng from line 16209818 > Loading data/persondata_en.nt (118002 lines) 14:44:55 > Loading data/redirect_en.nt (3494567 lines) 15:14:27 > Loading data/shortabstract_en.nt (2787499 lines) 16:36:46 > Loading data/skoscategories_en.nt (1844055 lines) 23:28:05 > Loading data/wikipage_en.nt (3201391 lines) 00:35:10 > Loading data/yagolink_en.nt (1889985 lines) 02:44:03 > Loading data/yago_en.nt (4645108 lines) 08:32:36 > Final checkpoint. > ======================================= > Check bad.nt file for skipped triples. > ======================================= > Running post-install scripts > PASSED: RDF_OBJ_FT_RULE_ADD : STATE=OK MESSAGE=OK > PASSED: VT_INC_INDEX_DB_DBA_RDF_OBJ : STATE=OK MESSAGE=OK > PASSED: RDF_QUAD_POGS : STATE=OK MESSAGE=OK > PASSED: RDF_QUAD_PSOG : STATE=OK MESSAGE=OK > PASSED: RDF_QUAD_SOPG : STATE=OK MESSAGE=OK > PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK > PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK > PASSED: DBpedia inference : STATE=OK MESSAGE=OK >>>>> > *** Error 23000: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR133: Can not set NULL to not nullable column 'DB.DBA.RDF_QUAD.O' > at line 36 of Top-Level: > ttlp_mt (file_to_string_output ('umbel_class_hierarchy_v071.n3'), '', > 'http://dbpedia.org/resource/classes/umbel#') > ***FAILED: Load umbel_class_hierarchy_v071 : STATE=23000 > MESSAGE=[OpenLink][Virtuoso ODBC Driver][Virtuoso Server]SR133: Can > not set NULL to not nullable column 'DB.DBA.RDF_QUAD.O' > PASSED: Load umbel_subject_concepts : STATE=OK MESSAGE=OK > PASSED: Load umbel_abstract_concepts : STATE=OK MESSAGE=OK > PASSED: Load umbel_external_ontologies_linkage : STATE=OK MESSAGE=OK > PASSED: UMBEL inference : STATE=OK MESSAGE=OK > PASSED: Load yago-class-hierarchy_en : STATE=OK MESSAGE=OK > PASSED: Yago inference : STATE=OK MESSAGE=OK > PASSED: Load opencyc_class_hierarchy : STATE=OK MESSAGE=OK >>>>> > *** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR337: Transaction aborted because the server is out of memory > at line 80 of Top-Level: > rdfs_rule_set ('http://dbpedia.org/resource/inference/rules/opencyc#', > 'http://dbpedia.org/resource/classes/opencyc#') >>>>> > *** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR337: Transaction aborted because the server is out of memory > at line 80 of Top-Level: > rdfs_rule_set ('http://dbpedia.org/resource/inference/rules/opencyc#', > 'http://dbpedia.org/resource/classes/opencyc#') > ***FAILED: Opencyc inference : STATE=40006 MESSAGE=[OpenLink][Virtuoso > ODBC Driver][Virtuoso Server]SR337: Transaction aborted because the > server is out of memory >>>>> > *** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SQ200: Out of memory > at line 84 of Top-Level: > checkpoint >>>>> > *** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR337: Transaction aborted because the server is out of memory > at line 84 of Top-Level: > checkpoint >>>>> > *** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SQ200: Out of memory > at line 85 of Top-Level: > DB..RDF_LOAD_RDFXML_MT (file_to_string_output > ('opencyc-2008-06-10-readable.owl'), > 'http://sw.opencyc.org/2008/06/10/concept/', > 'opencyc-2008-06-10-readable') >>>>> > *** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR337: Transaction aborted because the server is out of memory > at line 85 of Top-Level: > DB..RDF_LOAD_RDFXML_MT (file_to_string_output > ('opencyc-2008-06-10-readable.owl'), > 'http://sw.opencyc.org/2008/06/10/concept/', > 'opencyc-2008-06-10-readable') > ***FAILED: opencyc-2008-06-10-readable : STATE=40006 > MESSAGE=[OpenLink][Virtuoso ODBC Driver][Virtuoso Server]SR337: > Transaction aborted because the server is out of memory >>>>> > *** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SQ200: Out of memory > in lines 89-98 of Top-Level: > #line 89 "(console)" > ttlp ( > '@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > <http://dbpedia.org/property/birthcity> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/birthcountry> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/cityofbirth> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/countryofbirth> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/countyofbirth> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace>. > <http://dbpedia.org/property/cityofdeath> rdfs:subPropertyOf > <http://dbpedia.org/property/deathPlace> . > <http://dbpedia.org/property/countryofdeath> rdfs:subPropertyOf > <http://dbpedia.org/property/deathPlace> .', > '','http://dbpedia.org/schema/property_rules#') >>>>> > *** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR337: Transaction aborted because the server is out of memory > in lines 89-98 of Top-Level: > #line 89 "(console)" > ttlp ( > '@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > <http://dbpedia.org/property/birthcity> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/birthcountry> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/cityofbirth> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/countryofbirth> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace> . > <http://dbpedia.org/property/countyofbirth> rdfs:subPropertyOf > <http://dbpedia.org/property/birthPlace>. > <http://dbpedia.org/property/cityofdeath> rdfs:subPropertyOf > <http://dbpedia.org/property/deathPlace> . > <http://dbpedia.org/property/countryofdeath> rdfs:subPropertyOf > <http://dbpedia.org/property/deathPlace> .', > '','http://dbpedia.org/schema/property_rules#') >>>>> > *** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SQ200: Out of memory > at line 100 of Top-Level: > rdfs_rule_set ('http://dbpedia.org/schema/property_rules#', > 'http://dbpedia.org/schema/property_rules#') >>>>> > *** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR337: Transaction aborted because the server is out of memory > at line 100 of Top-Level: > rdfs_rule_set ('http://dbpedia.org/schema/property_rules#', > 'http://dbpedia.org/schema/property_rules#') > ***FAILED: DBpedia property inference : STATE=40006 > MESSAGE=[OpenLink][Virtuoso ODBC Driver][Virtuoso Server]SR337: > Transaction aborted because the server is out of memory >>>>> > *** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SQ200: Out of memory > at line 104 of Top-Level: > checkpoint >>>>> > *** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso > Server]SR337: Transaction aborted because the server is out of memory > at line 104 of Top-Level: > checkpoint > =============================== > Install finished. > Sat Sep 19 06:39:55 GMTDT 2009 > =============================== > The MAIL_SERVER environment variable is not set, e.g. export > MAIL_SERVER=mail.domain:25 > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJKt4fAAAoJEOqaT4w4/280u2MH/iwjAdQzUNchl1LjKh1N4fuS 87/SAHKminrlmvelrpbKoobIUt4rEWYfzOK1TugchVhqETWmwQ1XoPBfJ/3pVn0/ 7z9Sc8hjsu2LPmPIvDoOH3ogZa/GEL5ixhZW9jGMkPcsg3y9Zi5QaKXDp0w08yJ6 KsR9fmI6qo20by31+Oy5wttNAKaRxTQ5xw+MHdi3UBORc+SuCxaEGbY6fDvFSEew IeaxHjTGCdvH4sFC7WvWgWON/NcVVezY/QvHKrST9sVuBWxO2VSaY2/XpkhGN1u5 Dv3Fw5PVMqKnBGLmN/SzbX8n/uRIvnWoWtWhOjXBdolYfe39O4I+54OSPrxc+ac= =bIGX -----END PGP SIGNATURE-----
