So it would be feasible to model the normalized rdf schema in MonetDB and hook up as an sdb backend ?
sent from my htc, forgive typos please ! On Sep 2, 2011 10:12 AM, "Paolo Castagna" <[email protected]> wrote: > > > Andy Seaborne wrote: >> >> It seems to use a non-normalized table design (as did the CStore paper) >> and rely on indexing. It would be interesting to see how that compares >> with a normalized design, which is what most RDF stores use. When >> normalized, the joins for patterns are on fixed size numbers, not string >> comparisons, and don't use secondary indexes (and at least one uses >> bitmap indexes). >> >> SDB is built around a normalized design, and portable across various >> different SQL DBs. The main triple table (or quad table) is 3 columns >> of 4 or 8 bytes (depending on hash vs index allocation of internal ids). >> That means the main table is a thin, long table. >> >> If you want a deep integration of SPARQL and MonetDB, the first thing to >> consider, if pure speed is the objective, is what schema design is best >> for MonetDB. >> >> The reported tests are on a dataset of 50 million triples which is > ^ > not > >> particularly big - it means RAM caching is going to play a significant >> role nowadays (they only used a one-CPU, 4G machine). >> Andy >> >> >> On 01/09/11 19:49, nat lu wrote: >>> Speed I think ! At least in some use cases. (But Im no MonetDB expert) >>> Heres an interesting article that tries a few of them out with MonetDB >>> and CStore. >>> >>> http://oai.cwi.nl/oai/asset/13806/13806B.pdf >>> >>> >>> Whats missing I believe is a the SPARQL endpoint integration. >>> >>> On 01/09/11 08:36, Andy Seaborne wrote: >>>> Tobias, >>>> >>>> To turn the question round - what unique features of MonetDB could be >>>> exposed through SDB to yield a better RDf store? >>>> >>>> The current design covers the current set of databases supported but >>>> it's not fixed - maybe there is something especially useful in MonetDB >>>> and maybe it needs an extension to the design. The design is based on >>>> templating via all those classes (the instance for each database is a >>>> small class). The SQL generator usually needs some per-DB work because >>>> SQL databases aren't exactly very "standard". >>>> >>>> Andy >>>> >>>> On 25/08/11 21:28, Paolo Castagna wrote: >>>>> Hi Tobias, >>>>> first of all, welcome on jena-users mailing list. >>>>> >>>>> Tobias Willig wrote: >>>>>> Hi everyone, >>>>>> >>>>>> I like to add support for MonetDB in SDB and I have two questions >>>>>> concerning >>>>>> this project: >>>>>> >>>>>> 1. How much effort it takes to add support for a new database type? >>>>> >>>>> It requires some effort. You need to add new Java classes and the >>>>> necessary tests. >>>>> >>>>> Have you checked out the SDB sources yet? >>>>> >>>>> If not: >>>>> >>>>> svn co >>>>> https://svn.apache.org/repos/asf/incubator/jena/Jena2/SDB/trunk/ SDB >>>>> cd SDB >>>>> mvn test >>>>> >>>>> You can use Eclipse and search for "Derby" which is one of the DBMS >>>>> supported >>>>> by SDB. This way you'll find the list of Java classes in SDB to >>>>> support Derby. >>>>> Then, you can read and study those classes. While you do that, you'll >>>>> learn the >>>>> design of SDB and you will get an idea on what it is required to add >>>>> MonetDB. >>>>> >>>>>> 2. Are there predefined extension points that allow adding a new >>>>>> database >>>>>> type easily? >>>>> >>>>> Yes, there are. Look at the super classes and interfaces from the >>>>> list of >>>>> classes above (i.e. searching for "Derby"). >>>>> >>>>> There isn't an "how to add a new database to SDB" guide, however make >>>>> sure >>>>> you read the general SDB documentation (it does not hurt). >>>>> >>>>> Also, if you want to contribute an "how to add a new database to SDB" >>>>> you >>>>> are more than welcome to do so. >>>>> >>>>>> If so could you give me the name of some classes and config files, >>>>>> which >>>>>> are important to accomplish that task? >>>>> >>>>> You can start from: >>>>> >>>>> GenerateSQLDerby -- extends --> GenrateSQL >>>>> FormatterSimpleDerby -- extends --> FormatterSimple >>>>> StoreSimpleDerby -- extends --> StoreBase1 >>>>> FmtLayout2HashDerby -- extends --> FmtLayout2 >>>>> StoreTriplesNodesHashDerby -- extends --> StoreBaseHash >>>>> TupleLoaderHashDerby -- extends --> TupleLoaderHashBase >>>>> FmtLayout2IndexDerby -- extends --> FmtLayout2HashDerby >>>>> StoreTriplesNodesIndexDerby -- extends --> StoreBaseIndex >>>>> TupleLoaderIndexDerby -- extends --> TupleLoaderIndexBase >>>>> >>>>> Also look at: >>>>> >>>>> - JDBC.java >>>>> - DatabaseType.java >>>>> - StoreFactory.java >>>>> - SDB.java >>>>> >>>>> And the existing tests. >>>>> >>>>> May I ask you what motivates you in adding MonetDB? >>>>> >>>>> I've never used it myself and, indeed, I use TDB instead of SDB. >>>>> Transactions are coming, hopefully in the 0.9.0 release of TDB. >>>>> >>>>> Last but not least, it's not only about the code. You should be willing >>>>> to support the users of your code too. Once you add support for >>>>> MonetDB, >>>>> people will start using it and, as they use your code, they'll find >>>>> bugs >>>>> and they'll ask you for more features, eventually. You should be >>>>> willing >>>>> to put some effort in fixing the bugs, at least... and you can always >>>>> say >>>>> "no" politely to new features. Until, someone else, who really needs >>>>> the >>>>> new feature and he/she is willing to put some effort, will take over >>>>> and >>>>> push the software a step further. >>>>> >>>>> Once you start, you might have more specific questions on SDB design. >>>>> You can post your questions here or on [email protected] >>>>> mailing list. The more you demonstrate you put some effort the more >>>>> likely you'll receive helpful answers back from the SDB developers. >>>>> If you don't put enough effort and expect others will do it for you, >>>>> I am afraid, you risk to have not much back. >>>>> >>>>> To conclude, it you are motivated and you think you'll have fun doing >>>>> it (and you need it for your work): go ahead, it's not a terribly huge >>>>> task and it could be a nice contribution (in particular for all the >>>>> MonetDB users out there). >>>>> >>>>> Paolo >>>>> >>>>>> >>>>>> Thanks in advance! >>>>>> >>>>>> Best Regards >>>>>> Tobias Willig >>>>>> >>>
