David Jordan wrote: > In terms of requirements, we will have several relatively large biomedical > ontologies that will not be changed at all between releases of our software, > it is basically reference data. The application user will not be changing > this data. There will be other, mostly smaller models that will have user's > data that will be changing daily and it must support concurrent updates. They > will also be making associations and inferencing relative to the large > read-only reference ontologies, but this can be stored in the users dataset. > So it seems like it would make sense to have the large reference ontologies > use TDB and the user data use SDB.
David, what "relatively large" corresponds in triples/quads? Paolo > > We would also have the reference model pre-inferenced (if that is the right > term to use), so that there is not a HUGE wait time when the model is first > read. > > -----Original Message----- > From: Andy Seaborne [mailto:[email protected]] > Sent: Friday, September 02, 2011 5:29 AM > To: [email protected] > Subject: Re: non-SQL databases and combining SDB and TDB models > > > > On 01/09/11 13:58, David Jordan wrote: >> I have a related question. I assume SDB is based on SQL as the >> database access language. TDB has its own triple store representation, >> which I understand is much faster than SDB/RDBMS, but has some >> limitations from an update concurrency standpoint. > > At the moment yes, soon, no. Transactions (full blown disk-level paranoia > about crashes and recovery) are coming. > > Current, TDB requires the application to do multiple-reader or single writer > locking. > >> Do both >> SDB and TDB build on top of a common core interface? If one had a >> non-SQL DBMS that potentially offered high performance, if this common >> core interface exists, would it be an appropriate interface to build >> an implementation to a non-SQL database? Or would making lots of >> alterations to TDB be necessary? > > SDB and TDB offer the Jena APIs. The only difefrence is how you get hold of > an initial dataset or model. After that, there is no difference. > >> One other question. If an application has a set of ontologies it uses, >> some of which are large and never change, some of which do require >> concurrent updates, can one have an application that opens a model in >> TDB, then opens a model from SDB, then combine these two models >> together? Would this allow one to take advantage of the TDB >> high-performance for the read-only ontologies, while still getting >> update capabilities for the ontologies in SDB? > > Yes - that's possible but depending on how you want to combine models. > > A dataset can have models from multiple storage subsystems. Or maybe your > usage can work with data in different datasets. > > Do you have specific requirements you are trying to meet? > > Andy >> -----Original Message----- From: Andy Seaborne >> [mailto:[email protected]] Sent: Thursday, September 01, >> 2011 3:37 AM To: [email protected] Subject: Re: Adding >> support for MonetDB >> >> Tobias, >> >> To turn the question round - what unique features of MonetDB could be >> exposed through SDB to yield a better RDf store? >> >> The current design covers the current set of databases supported but >> it's not fixed - maybe there is something especially useful in MonetDB >> and maybe it needs an extension to the design. The design is based on >> templating via all those classes (the instance for each database is a >> small class). The SQL generator usually needs some per-DB work >> because SQL databases aren't exactly very "standard". >> >> Andy >> >> On 25/08/11 21:28, Paolo Castagna wrote: >>> Hi Tobias, first of all, welcome on jena-users mailing list. >>> >>> Tobias Willig wrote: >>>> Hi everyone, >>>> >>>> I like to add support for MonetDB in SDB and I have two questions >>>> concerning this project: >>>> >>>> 1. How much effort it takes to add support for a new database type? >>> It requires some effort. You need to add new Java classes and the >>> necessary tests. >>> >>> Have you checked out the SDB sources yet? >>> >>> If not: >>> >>> svn co >>> https://svn.apache.org/repos/asf/incubator/jena/Jena2/SDB/trunk/ >>> SDB cd SDB mvn test >>> >>> You can use Eclipse and search for "Derby" which is one of the DBMS >>> supported by SDB. This way you'll find the list of Java classes in >>> SDB to support Derby. Then, you can read and study those classes. >>> While you do that, you'll learn the design of SDB and you will get an >>> idea on what it is required to add MonetDB. >>> >>>> 2. Are there predefined extension points that allow adding a new >>>> database type easily? >>> Yes, there are. Look at the super classes and interfaces from the >>> list of classes above (i.e. searching for "Derby"). >>> >>> There isn't an "how to add a new database to SDB" guide, however make >>> sure you read the general SDB documentation (it does not hurt). >>> >>> Also, if you want to contribute an "how to add a new database to SDB" >>> you are more than welcome to do so. >>> >>>> If so could you give me the name of some classes and config files, >>>> which are important to accomplish that task? >>> You can start from: >>> >>> GenerateSQLDerby -- extends --> GenrateSQL FormatterSimpleDerby >>> -- extends --> FormatterSimple StoreSimpleDerby -- extends --> >>> StoreBase1 FmtLayout2HashDerby -- extends --> FmtLayout2 >>> StoreTriplesNodesHashDerby -- extends --> StoreBaseHash >>> TupleLoaderHashDerby -- extends --> TupleLoaderHashBase >>> FmtLayout2IndexDerby -- extends --> FmtLayout2HashDerby >>> StoreTriplesNodesIndexDerby -- extends --> StoreBaseIndex >>> TupleLoaderIndexDerby -- extends --> TupleLoaderIndexBase >>> >>> Also look at: >>> >>> - JDBC.java - DatabaseType.java - StoreFactory.java - SDB.java >>> >>> And the existing tests. >>> >>> May I ask you what motivates you in adding MonetDB? >>> >>> I've never used it myself and, indeed, I use TDB instead of SDB. >>> Transactions are coming, hopefully in the 0.9.0 release of TDB. >>> >>> Last but not least, it's not only about the code. You should be >>> willing to support the users of your code too. Once you add support >>> for MonetDB, people will start using it and, as they use your code, >>> they'll find bugs and they'll ask you for more features, eventually. >>> You should be willing to put some effort in fixing the bugs, at >>> least... and you can always say "no" politely to new features. Until, >>> someone else, who really needs the new feature and he/she is willing >>> to put some effort, will take over and push the software a step >>> further. >>> >>> Once you start, you might have more specific questions on SDB design. >>> You can post your questions here or on [email protected] >>> mailing list. The more you demonstrate you put some effort the more >>> likely you'll receive helpful answers back from the SDB developers. >>> If you don't put enough effort and expect others will do it for you, >>> I am afraid, you risk to have not much back. >>> >>> To conclude, it you are motivated and you think you'll have fun doing >>> it (and you need it for your work): go ahead, it's not a terribly >>> huge task and it could be a nice contribution (in particular for all >>> the MonetDB users out there). >>> >>> Paolo >>> >>>> Thanks in advance! >>>> >>>> Best Regards Tobias Willig >>>> >> > >
