Hi Pekka,

The fastest method of loading large datasets into Virtuoso is using the RDF 
Bulk loader as detailed at:

        
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader

Having said that the getBulkUpdateHandler() should improve performance over 
single inserts, but you need to ensure you are using the latest version 
available from the git develop/6 branch as there have been recent fixes for 
using the jena bulk update method ...

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.      //              http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

On 27 Jun 2013, at 11:19, Pekka Aarnio <[email protected]> wrote:

> Hello
> I have coded a java program for mapping and loading relational data over 
> network into Virtuoso (open source edition) as rdf/owl graphs.
> This program uses first d2rq-tool api for relational-to-rdf mapping and 
> then loads the generated list of triples into Virtuoso using Virtuoso 
> Jena provider api function: 
> virtGraph.getBulkUpdateHandler().add(triplelist).
> When I tested the performance of this program, I noticed that the 
> execution time of rdf-loading was one hundred times longer than by 
> loading the rdf-file locally at server side using isql-tool and 
> Virtuosos TTLP_MT function.
> Programatical loading of 1.5 million triples took about 10000 seconds 
> whereas by using isql it took only 100s. If I try to load bigger rdf 
> graphs, I get Virtuoso Communications Link Failure (timeout).
> 
> 1)I understand that the performance of Jena provider must be lower in 
> general, but is this factor 100 what could be expected?
> 2)Is this getBulkUpdateHandler().add() function the best way to load big 
> rdf graphs, or is there some other options?
> 3)Configuration tricks to increase performance?
> 4)Another problem I faced with when trying to use d2rq api and virtuoso 
> jena provider in the same java program is that they seem to require 
> different versions of jena packages. So I can't use sparql queries in 
> the same program where I do graph loading using d2rq? Is there some 
> other good option for mapping and loading external relational data into 
> Virtuoso rdf graphs?
> 
> Virtuoso Server: Ubuntu 12.04, Intel Q9400 2,66MHz, 5GB.
> Java client: Windows 7, Mem: 10GB
> 
> Regards
> Pekka Aarnio
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
> 
> Build for Windows Store.
> 
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Virtuoso-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Attachment: smime.p7s
Description: S/MIME cryptographic signature

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Virtuoso-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Reply via email to