Compared to TDB1:
* No size limits on transactions
(my test is 100m BSBM triple upload into a live
Fuseki while still responding to queries)
* Transactional only (and currently no "autocommit").
* Better transaction control
* No queue of delayed updates
* No backlog problems.
* "Writer pays" - readers don't
* Space reclamation
* Binary node table
* Graphs can be passed across transactions
* Datatypes of numerics preserved; xsd:doubles supported.
The encoding of values into NodeId is different.
Work outstanding:
Before a release that is labelled as anything other than experimental,
the on-disk format needs stabilizing:
* Some kind of hook for layout format evolution
Future possibilities: compressed storage, other B+Trees.
* Canonicalize language tags.
Canonicalise all numbers (and time-fractional seconds)
even if going into the full node table.
* Code to look at an existing database and determine it if it is TDB1 or
TDB2 to stop accidentally messing up databases.
Other work:
* Graph.begin() - promotable read transactions
(this is common to TIM and TBD1)
* Autocommit, maybe. There are downsides to offering this.
* Better prefix management.
* StoreParams - configurable caches
Access to control configuration got lost from TDB1
* Transaction integration with jena-text
TDB2 uses a two level directory structure:
The top level "container" holds different generations of the database, a
new generation is created when the DB is compacted. It also holds a
backups area and potentially can be used to hold lucene indexes for
better usability of tdb2 + jena-text.
The inner level is a "storage" - this is a directory with the database
files in it (indexes, node table) like TDB1. The file are different
though. The indexes and node data datastructures are transactional in
their own right. There is no replay log. The journal is used to
temporarily record the new state settings of the datastructures - a few
hundred bytes.
BTrees are copy-on-write-transaction append datastructures - like TIM,
they retain their previous state. The nodes for the node table are
managed as a transactional append only file. Because they are all
append-only file, writing happens as the transaction goes along, with
some administration work at the point of commit.
That, and the binary node files, speeds loading into an existing
database up. There is no special bulk loader at the moment (there is a
placeholder command). In fact, TDB2 loading directly is about the same
as the TDB1 bulk loader.
A storage area database keeps all commits. This could be exploited to
have temporal features. Old states of the storage are never lost and, if
you track the set of index roots, the previous states are accessible.
But it means that database storage grows in size. This is addressed by
compaction which is copying the latest version into a new storage
directory and switching over to using the new copy. The new copy does
not have the old databases state so it is smaller. Compaction happens on
a live database. It currently blocks writers but readers can start and
continue while it's happening.
Andy
