Hi Sumit,
Question 1: When you start a new/empty Virtuoso 7 database the initialisation
process automatically creates the RDF_QUAD table and indexes in column-wise
mode as this is considered to be the best default for RDF work loads, mainly
due to the 3 times better data compression achieved in this mode resulting the
a third the amount of memory and disk for hosting a given dataset.
Question 2: Yes, if you want to convert an existing database typically one
upgrade from v6 or if it just happened to be a v7 row-wise database if you
didn't have the column-wsie feature enabled then the command in that document
for converting to column-wise can be used.
If all your data is in RDF then yes the RDF Bulk Loader ie rdf_loader_run()
method with multiple processes running based on the number of cores available
for parallel loading of multiple datasets at the same time is the most
efficient and performant method of loading.
The RDF_QUAD table is the only one that needs to be changed/configured for
maximum performance as indicated in the RDF performance tuning guide at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning
I hope this answers your questions ...
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 22 May 2014, at 18:57, "Purohit, Sumit" <sumit.puro...@pnnl.gov> wrote:
> Thanks Kingsley and entire community for this meaning full discussion,
> After this discussion, I have 2 follow-up question.
>
> 1. How do I create a new empty database in my existing virtuoso and start
> loading RDF data into that by default. I think this way I will be using
> column-wise structure.
>
> 2. If I want to convert existing tables to column store format. Using
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksRowToColStoreConversion
>
> All my data is RDF data loaded using rdf_loader_run().
> What tables other than "RDF_QUAD" should be changed in order to maximize
> performance.
>
>
>
> Thanks
> sumit
>
> -----Original Message-----
> From: Kingsley Idehen [mailto:kide...@openlinksw.com]
> Sent: Thursday, May 22, 2014 7:24 AM
> To: virtuoso-users@lists.sourceforge.net
> Subject: Re: [Virtuoso-users] Using column store structure for exiting data
>
> On 5/22/14 9:50 AM, Kingsley Idehen wrote:
>>> Is it normal that re-loading the data takes significantly more time
>>> than dumping it?
>>>
>>> Regards,
>>> Bart
>>
>> Yes, for the following reasons:
>>
>> 1. Exporting is just about writing to files 2. Importing is about
>> indexing and writing to internal identifiers (named graph IRIs).
>>
>> You can speed up you loads by running multiple loaders in parallel.
>> Even better if you are using the cluster edition as the same source
>> files could be copied to multiple physical cluster nodes where each is
>> running multiple loaders.
>>
>>
>> Links:
>>
>> [1] http://bit.ly/11pPq6F -- old bulking load report re., Virtuoso 7
>> and LOD Cloud Cache.
>
> To be precise, I meant to say:
>
> 1. Exporting is just about writing to files 2. Importing is about writing to
> *internal documents* (data partitions or sources) denoted by named graph IRIs
> and indexing.
>
> --
>
> Regards,
>
> Kingsley Idehen
> Founder & CEO
> OpenLink Software
> Company Web: http://www.openlinksw.com
> Personal Weblog: http://www.openlinksw.com/blog/~kidehen
> Twitter Profile: https://twitter.com/kidehen
> Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
> LinkedIn Profile: http://www.linkedin.com/in/kidehen
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing platform available
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
