Thanks for the heads up, Dan. Will go and check the archives. I think I should get how to decide between tdb and TDB2 in the archives itself.
On Tue, 12 Nov, 2019, 8:59 PM Dan Pritts, <[email protected]> wrote: > Look through the list archives for posts from Andy describing the > differences between tdb1 and tdb2. they have different optimizations; I > don't recall the differences. > > thanks > danno > > Dan Pritts > ICPSR Computing and Network Services > > On 12 Nov 2019, at 7:29, Amandeep Srivastava wrote: > > > Hi, > > > > I'm trying to create a TDB database from Wikidata's official RDF dump > > to > > read the data using Fuseki service. I need to make a few queries for > > my > > personal project, running which the online service times out. > > > > I have a 12 core machine with 36 GB memory. > > > > Can you please advise on the best way for creating the database? Since > > the > > dump is huge, I cannot try all the approaches. Besides, I'm not sure > > if the > > tdbloader function works in a similar way on data of different sizes. > > > > Questions: > > > > 1. Which one would be better to use - tdb.tdbloader2 (TDB1) or > > tdb2.tdbloader (TDB2) for creating the database and why? Any specific > > configurations that I should be aware of? > > > > 2. I'm running a job currently using tdb.tdbloader2 but it is using > > just a > > single core. Also, it's loading speed is decreasing slowly. It started > > at > > an avg of 120k tuples and is currently at 80k tuples. Can you advise > > how > > can I utilize all the cores of my machine and maintain the loading > > speed at > > the same time? > > > > Regards, > > Aman >
