Hi Frans,
You definitely should use the rdf_loader_run() buik loader function for loading
multiple files. You did not provide the isql "status();" and "top" command
outputs when the ttlp function was apparently hanging so we can see the state
of the server at that point ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 18 Apr 2014, at 15:55, Frans Knibbe | Geodan <frans.kni...@geodan.nl> wrote:
> Hi Hugh,
>
> I am calling TTLP_MT() on the isql command line at the moment. I keep the
> file size under 1 Gb and limit the number of lines (or triples) to 5 million.
> My system has a modest memory of 4Gb.
>
> I have just had a hang situation again, this time with a really small file:
> 44 Mb and 300,000 lines, so I had a chance to look into some details. The
> isql session hangs, but I could open another session and execute status(). It
> showed that the TTLP_MT() statement was still active. At that moment
> virtuoso-t did not feature in the list made by the 'top' command. I could
> also count the number of triples imported, that being some 80 or 90 percent
> of the file contents. I restarted Virtuoso and reran the statement. That time
> it did quickly succeed. I was happy to notice that despite me not having
> removed the partial result of the first attempt TTLP_MT() did not create any
> duplicate triples: the triple count was equal to the line count.
>
> I have not tried the bulk loader, although I did notice its existence when
> looking for help on the web. I assumed that when TTLP_MT() hangs when I call
> it from isql it would also hang when I use rdf_loader_run(). But maybe I
> should just give it a try.
>
> Regards,
> Frans
>
>
> On 2014-04-18 12:43, Hugh Williams wrote:
>> Hi Frans,
>>
>> How are you calling the TTLP_MT() function to load multiple files and when
>> you says these file are small what is there typical size or number of
>> lines/triples ? For loading of large number of files I would expect you to
>> be using the rdf_loader_run() bulk loader function which calls ttlp() under
>> the covers for loading as detailed at:
>>
>> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader
>>
>> Using this method you can then use the LDMeter functions to monitor the load
>> rates as detailed at:
>>
>>
>> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideLDMeterUtility
>>
>> Also, at the time the TTLP load appear to hang I presume you can still
>> access the server in which case what does the "status();" command reports as
>> to the status of the server at that point ? How much memory is on the
>> system and how much does the "top" command report as being used by Virtuoso ?
>>
>> Best Regards
>> Hugh Williams
>> Professional Services
>> OpenLink Software, Inc. // http://www.openlinksw.com/
>> Weblog -- http://www.openlinksw.com/blogs/
>> LinkedIn -- http://www.linkedin.com/company/openlink-software/
>> Twitter -- http://twitter.com/OpenLink
>> Google+ -- http://plus.google.com/100570109519069333827/
>> Facebook -- http://www.facebook.com/OpenLinkSoftware
>> Universal Data Access, Integration, and Management Technology Providers
>>
>> On 18 Apr 2014, at 10:12, Frans Knibbe | Geodan <frans.kni...@geodan.nl>
>> wrote:
>>
>>> Hello,
>>>
>>> I am using the function TTLP_MT to load ntriple files in Virtuoso 7.1. Even
>>> though I take care to have reasonably small files (splitting the files by
>>> line numbers if necessary) TTLP_MT still frequently hangs. I have to
>>> restart Virtuoso, remove the partial load results and start over. This is
>>> quite bothersome, because I have a lot of data to load. I would like to be
>>> able to load the files from a script without constantly having to check if
>>> Virtuoso hangs and making repairs.
>>>
>>> Are there any tips on getting TTLP_MT to work without hanging? I think the
>>> memory configuration is OK, available system memory is used without disk
>>> swapping.
>>>
>>> Regards,
>>> Frans
>>>
>>>
>>> Frans Knibbe
>>> Geodan
>>> President Kennedylaan 1
>>> 1079 MB Amsterdam (NL)
>>>
>>> T +31 (0)20 - 5711 347
>>> E frans.kni...@geodan.nl
>>> www.geodan.nl | disclaimer
>>> ------------------------------------------------------------------------------
>>> Learn Graph Databases - Download FREE O'Reilly Book
>>> "Graph Databases" is the definitive new guide to graph databases and their
>>> applications. Written by three acclaimed leaders in the field,
>>> this first edition is now available. Download your free book today!
>>> http://p.sf.net/sfu/NeoTech_______________________________________________
>>> Virtuoso-users mailing list
>>> Virtuoso-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>>
>
>
> Frans Knibbe
> Geodan
> President Kennedylaan 1
> 1079 MB Amsterdam (NL)
>
> T +31 (0)20 - 5711 347
> E frans.kni...@geodan.nl
> www.geodan.nl | disclaimer
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech_______________________________________________
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
