Hi Pantelis, I have seen data loads progressively get slower to the point of hanging , when loading large datasets and they is not sufficient memory available, but never the number of triples decrease ( roughly half in your case below) ???
How much memory is available on your system and what does the “status();”
command show at the point when the triples load rate get slow or start
“decreasing” ?
Have you performance tuned you database in particular assuming you are running
on Linux setup the swappiness to minimise swapping which tends to kill load
rates as detailed at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning
You can also use the Virtuoso LDMeter script to monitor the data load rates and
log to table as detailed at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideLDMeterUtility
As you indicated having compiled from source I would recommend updating to the
latest git stable/7 3216 branch as there are many fixes since 3214 from almost
a year ago …
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 18 Apr 2016, at 07:59, Pantelis Natsiavas <natsia...@gmail.com> wrote:
>
> Hello everybody.
>
> I tried to import a graph contained in the file "clinicaltrials.nq.gz". The
> file is rather big as it has a size of about 2GBs. I followed the process
> described in
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader
> <http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader>.
>
>
> Everything was going Ok for one day or so. I checked the loading process at
> some point and the new graph contained 63649311 triplets. One day after, I
> checked again and the graph contained 31133122 triplets (less than one day
> before)! I checked the table DB.DBA.LOAD_LIST and it showed no error on the
> importing procedure that it was still going (ll_state was 1).
>
> I considered the above described situation as invalid and checked the VM that
> Virtuoso was running. The disk is full with a
> /usr/local/var/lib/virtuoso/db/virtuoso.db of size 50522488832. When I
> removed the newly installed graph, the file of the size has not reduced at
> all!!!
>
> I have two questions:
>
> 1. Since I deleted a huge amount of triples, I would expect the database's
> file size to be reduced. Am I right? How could I free the respective disk
> size?
> 2. How could it be explained that while importing a triples file, the triples
> number is decreasing? As the DB.DBA.LOAD_LIST provides no information, is
> there a log file that I could check to see the error?
>
> I am using a Version: 07.20.3214, Build: Oct 14 2015 (I built it from source
> files).
>
> Kind regards,
> Pantelis Natsiavas
> ------------------------------------------------------------------------------
> Find and fix application performance issues faster with Applications Manager
> Applications Manager provides deep performance insights into multiple tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z_______________________________________________
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
