You're on the right track. But let me sketch how Falcon handled
transactions as food for thought.
Falcon kept a linked list of transactions in transaction start order (batch
inserts of "unassigned" transaction objects that could be grabbed with a
compare and swap). When a transaction was started, it was assigned a
sequential transaction event number. When it was committed, it was assign
another sequential transaction number, essentially a commit sequence. When
a transaction needed to compute the relative transaction state of a record,
it used the record's transaction id as an index into a transaction hash
table (maintained with interlocked instructions), then compared the its
starting event to the other transaction commit event.
But if you were to take this further and assign committed transaction group
ids, where transactions of the same group id were known not to have any
intervening active transaction, then two consecutive record versions with
the same transaction transaction group could have the second an subsequent
versions garbage collected.
This dispenses with having to snapshot the transaction inventory vector,
significantly reducing transaction start overhead. It does incur a memory
cost for transaction objects between the oldest active and the current,
though possibly a separate data structure to determine transaction group
would help. And, of course, it doesn't work at all in classic, which is a
major concern.
But, all in all, intermediate version garbage collection is the holy grail
of Firebird.
On Wednesday, July 2, 2014, Nikolay Samofatov <
nikolay.samofa...@red-soft.biz> wrote:
> All,
>
> I have completed my investigation, and it seems possible to implement
> intermediate GC in a way that makes everything faster, not slower.
>
> Here is what I am going to do:
>
> 1) Move TPC into shared memory. For each transaction store its Commit # in
> memory.
> Transactions committed before the database started would get commit #1.
> Further commits would be numbered 2, 3, ...
>
> Snapshots (both request-level and transaction-level) will become
> identified by a
> single number, that is Commit #.
>
> In VIO_chase_record_version, I would look up Commit # for a transaction
> that committed
> the version and if this number <= Commit # for a snapshot, version would
> be used.
>
> I plan to organize TPC as paged array of transaction states in shared
> memory, and use
> lockless reads algorithm. So TPC will be really fast.
>
> 2) Implement another shared memory structure for snapshot accounting.
> Basically, store Commit # for each snapshot (request-level and
> transaction-level).
> Lock manager can actually be used for this purpose, is this operation is
> not
> as performance sensitive as point 1 above.
>
> 3) Implement intermediate GC algorithm as I explained earlier.
> Checking that two versions records being chased belong to the same snapshot
> becomes really simple and cheap.
>
> Since expected changes are relatively large, I do not plan to produce
> patches against 2.5
> code base for this logic, only against Firebird 3.0 codebase.
>
> With these changes, I expect to see no scenarios when request-level
> snapshots would impose
> significant performance impact (long-running transactions would not
> inhibit GC anymore),
> and it would be possible to use them by default without breaking backwards
> compatibility.
>
>
> On 06/25/2014 02:27 PM, James Starkey wrote:
>
> It's doable -- I implemented it in NuoDB, so consider that as an existence
> proof.
>
> On Wednesday, June 25, 2014, Nikolay Samofatov <
> nikolay.samofa...@red-soft.biz
> <javascript:_e(%7B%7D,'cvml','nikolay.samofa...@red-soft.biz');>> wrote:
>
>> All,
>>
>> On 06/25/2014 09:41 AM, James Starkey wrote:
>> > I suggest you investigate garbage collecting intermediate unreachable
>> record versions, which would
>> > completely take the sting out of long running transactions. It's not
>> difficult for superserver,
>> > but would be problematic for classic. But I don't think that in itself
>> is a problem.
>>
>> That is very reasonable suggestion. I will investigate.
>>
>> Nikolay
>>
>
> Nikolay
>
>
--
Jim Starkey
------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
