Hugh Williams wrote: > Why were the 7.2.1 & 7.2.0 builds configured with different options, as they > not both being performed on the same OS, in which case I would expect the > configure options to the same ?
I tried the same configure options as well, and then I tried different options because I was grasping at straws. Here is a graph showing that 6.1.8 does not have the problem, that the changes to configure options doesn't matter for this issue. [cid:image003.png@01D0D8D6.3F850EB0] The basics for reproducing the problem are as follows: * Download ftp://ftp.nlm.nih.gov/online/mesh/mesh.nt and the vocabulary and void files from that location. * Load the vocabulary-0.9.ttl to graph http://id.nlm.nih.gov/mesh/vocab * Load the void-0.9.ttl to graph http://id.nlm.nih.gov/mesh/void * Load the mesh.nt file to graph http://id.nlm.nih.gov/mesh again and again (I used 15 seconds delay) with DB.DBA.ttlp(). From: Hugh Williams [mailto:hwilli...@openlinksw.com] Sent: Sunday, August 16, 2015 8:52 PM To: Davis, Daniel (NIH/NLM) [C] <daniel.da...@nih.gov> Cc: virtuoso-users@lists.sourceforge.net Subject: Re: [Virtuoso-users] Memory leaks with virtuoso 7.2.0 Hi Daniel, Why were the 7.2.1 & 7.2.0 builds configured with different options, as they not both being performed on the same OS, in which case I would expect the configure options to the same ? Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers On 13 Aug 2015, at 20:39, Davis, Daniel (NIH/NLM) [C] <daniel.da...@nih.gov<mailto:daniel.da...@nih.gov>> wrote: Hugh Williams wrote: Also, note the following 2 parameters: ThreadCleanupInterval, ResourcesCleanupInterval which can both be set to 1 in order to reduce memory leaking: http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ex_threadcleanupinterval although as this was not apparently occurring with 7.2.0 it may not help, but worth trying ... It does occur with 7.2.0 - I'd vainly hoped 7.2.1 would fix it. The XSLT process produces the file ftp://ftp.nlm.nih.gov/online/mesh/mesh.nt, which contains 11,917,672 triples. I'm also using the ontology ftp://ftp.nlm.nih.gov/online/mesh/vocabulary_0.9.ttl and the void file ftp://ftp.nlm.nih.gov/online/mesh/void_0.9.ttl. I cannot actually say it is a "memory leak" because I haven't pulled out those tools yet - VmRSS seems to grow linearly over time. VmSize does the same, only the sloop is not as steep. I've built Virtuoso 7.2.1 on CentOS 6.6 as follows: ./configure --prefix=<whatever> --with-readline Previously, with 7.2.0, my build was: export CFLAGS="-O2 -m64" ./configure --prefix=<whatever> --with-readline --with-iodbc=/usr --without-internal-zlib --enable-shared --disable-static I've also configured it with ThreadCleanupInterval = 1 and ResourcesCleanupInterval = 1. The reload I'm testing runs the following commands from an isql command-line: log_enable(2,1); sparql clear graph <http://id.nlm.nih.gov/mesh>; sparql clear graph <http://id.nlm.nih.gov/mesh/vocab>; sparql clear graph <http://id.nlm.nih.gov/mesh/void>; DB.DBA.ttlp(file_to_string_output('path-to-files/mesh.nt'), '', 'http://id.nlm.nih.gov/mesh', 0); DB.DBA.ttlp(file_to_string_output('path-to-files/vocabulary_0.9.ttl'), '', 'http://id.nlm.nih.gov/mesh/vocab', 0); rdfs_rule_set('http://id.nlm.nih.gov/mesh/vocab', 'http://id.nlm.nih.gov/mesh/vocab'); DB.DBA.ttlp(file_to_string_output('path-to-files/void_0.9.ttl'), '', 'http://id.nlm.nih.gov/mesh/void', 0); CHECKPOINT; I'm retrying with a shorter script, which only loads 100,000 triples: log_enable(2,1); sparql clear graph <http://id.nlm.nih.gov/meshtest>; DB.DBA.ttlp(file_to_string_output('path-to-files/small-mesh.nt', '', 'http://id.nlm.nih.gov/meshtest', 0); If that proves stable, I'll do the following: - stop clearing and reloading the vocabulary and void graphs and redefining the ruleset - let you guys know it worked. If that is not stable, I'll do one or several of the following: - try virtuoso-opensource-6 - open/comment on an issue on github.com/openlinksw/virtuoso-opensource/issues<http://github.com/openlinksw/virtuoso-opensource/issues> - install valgrind and valgrind-devel and build with malloc debug - evaluate switching to the directory scan option. Dan Davis, Systems/Applications Architect (Contractor), Office of Computer and Communications Systems, National Library of Medicine, NIH ------------------------------------------------------------------------------ _______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net<mailto:Virtuoso-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
------------------------------------------------------------------------------
_______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
