Hello Hugh,
I must thank you for convincing me to use the bulk loader because it
does a far better job than me in loading data! I haven't had a problem
so far.
I highly recommend the bulk loader to anyone that wants to load triples
from files.
Thanks,
Frans
On 2014-04-18 19:14, Hugh Williams wrote:
Hi Frans,
You definitely should use the rdf_loader_run() buik loader function
for loading multiple files. You did not provide the isql "status();"
and "top" command outputs when the ttlp function was apparently
hanging so we can see the state of the server at that point ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 18 Apr 2014, at 15:55, Frans Knibbe | Geodan
<frans.kni...@geodan.nl <mailto:frans.kni...@geodan.nl>> wrote:
Hi Hugh,
I am calling TTLP_MT() on the isql command line at the moment. I keep
the file size under 1 Gb and limit the number of lines (or triples)
to 5 million. My system has a modest memory of 4Gb.
I have just had a hang situation again, this time with a really small
file: 44 Mb and 300,000 lines, so I had a chance to look into some
details. The isql session hangs, but I could open another session and
execute status(). It showed that the TTLP_MT() statement was still
active. At that moment virtuoso-t did not feature in the list made by
the 'top' command. I could also count the number of triples imported,
that being some 80 or 90 percent of the file contents. I restarted
Virtuoso and reran the statement. That time it did quickly succeed. I
was happy to notice that despite me not having removed the partial
result of the first attempt TTLP_MT() did not create any duplicate
triples: the triple count was equal to the line count.
I have not tried the bulk loader, although I did notice its existence
when looking for help on the web. I assumed that when TTLP_MT() hangs
when I call it from isql it would also hang when I use
rdf_loader_run(). But maybe I should just give it a try.
Regards,
Frans
On 2014-04-18 12:43, Hugh Williams wrote:
Hi Frans,
How are you calling the TTLP_MT() function to load multiple files
and when you says these file are small what is there typical size or
number of lines/triples ? For loading of large number of files I
would expect you to be using the rdf_loader_run() bulk loader
function which calls ttlp() under the covers for loading as detailed at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader
Using this method you can then use the LDMeter functions to monitor
the load rates as detailed at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideLDMeterUtility
Also, at the time the TTLP load appear to hang I presume you can
still access the server in which case what does the "status();"
command reports as to the status of the server at that point ? How
much memory is on the system and how much does the "top" command
report as being used by Virtuoso ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 18 Apr 2014, at 10:12, Frans Knibbe | Geodan
<frans.kni...@geodan.nl <mailto:frans.kni...@geodan.nl>> wrote:
Hello,
I am using the function TTLP_MT to load ntriple files in Virtuoso
7.1. Even though I take care to have reasonably small files
(splitting the files by line numbers if necessary) TTLP_MT still
frequently hangs. I have to restart Virtuoso, remove the partial
load results and start over. This is quite bothersome, because I
have a lot of data to load. I would like to be able to load the
files from a script without constantly having to check if Virtuoso
hangs and making repairs.
Are there any tips on getting TTLP_MT to work without hanging? I
think the memory configuration is OK, available system memory is
used without disk swapping.
Regards,
Frans
------------------------------------------------------------------------
Frans Knibbe
Geodan
President Kennedylaan 1
1079 MB Amsterdam (NL)
T +31 (0)20 - 5711 347
E frans.kni...@geodan.nl
www.geodan.nl <http://www.geodan.nl/> | disclaimer
<http://www.geodan.nl/disclaimer>
------------------------------------------------------------------------
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases
and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
------------------------------------------------------------------------
Frans Knibbe
Geodan
President Kennedylaan 1
1079 MB Amsterdam (NL)
T +31 (0)20 - 5711 347
E frans.kni...@geodan.nl
www.geodan.nl <http://www.geodan.nl/> | disclaimer
<http://www.geodan.nl/disclaimer>
------------------------------------------------------------------------
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
------------------------------------------------------------------------
Frans Knibbe
Geodan
President Kennedylaan 1
1079 MB Amsterdam (NL)
T +31 (0)20 - 5711 347
E frans.kni...@geodan.nl
www.geodan.nl <http://www.geodan.nl> | disclaimer
<http://www.geodan.nl/disclaimer>
------------------------------------------------------------------------
------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
