> AFAIK, concurrent GC in pre-committed transactions without transaction lock
> was unsafe, and created
> permanently broken databases.
> I believe corruption has become especially apparent after commit intended to
> fix CORE-3515. I don't
> know who fixed it, but Red Soft builds include a patch for this problem for
> about a year. I was
> surprised to learn last week than vanilla Firebird still has the problem.
Do you speak about CORE-3921 ? If yes, it was fixed more than half a year
ago.
...
>>> We tried to be extra careful not to hurt performance while establishing
>>> these snapshots.
>> > We do it much more efficiently than done for isc_tpb_concurrency.
>>
>> AFAIU, you maintain list of active transactions at top-level request.
>> You create this list
>> using new Lock Manager call which iterates thru LCK_tra locks and calls
>> callback which fills
>> list (actually BePlusTree) of active transactions.
>> Later reader consult this list (i.e. search BePlusTree) to check
>> transaction state.
>
> Yes, the data structure used is a bit more involved, but in general you are
> correct.
>
> BePlusTree based SparseBitmap implementation is quite quick
Let me disagree here ;)
> and is currently used in the most performance-sensitive parts of the code
> (data index retrieval by
> index, for example).
Sorry, but i don't understand what is "data index retrieval by index"...
>>
>> Is it really faster than copy part of (already cached) TIP contents into
>> transaction and then
>> test a bit in a plain vector ? Also, it allows to reuse existing code.
>
> No, this actually would have horrible performance characteristics. Take
> BroadView, for example.
> They have many millions transactions processed daily, and TIP size of
> 1500-4000 pages is typical.
> Imagine how costly is to copy such a thing?
Do you speak about active part of the TIP (between OIT and Next) or about
whole TIP ? I can easily
imagine TIP of 4000 pages. It is not a problem as we have no need to copy all
that pages, only starting
from OIT value. But if BroadView permanently have stuck OIT with difference of
few millions... it is not
good for them. Hope it is not so.
I guess that memove of 4-8-16KB is much faster than 100-1000 calls of callback
with search in
BePlusTree... Also, note, you pay additional cost of the search in BePlusTree
every time you test
transaction state, i.e. for every record version accessed by RC transaction.
> Actually, my code would probably improve performance of isc_tpb_concurrency
> transactions if used
> there, but I didn't try it.
Interesting to compare ;)
>> Next, i see that you trying to adjust transaction lock data to change
>> garbage collection
>> threshold. At first look it is correct and highly necessary, but i think it
>> can't work as
>> it can't change values already cached by concurrent transactions. I.e. you
>> can't make OST
>> less than it was known to someone (except of yourself). Therefore i consider
>> that new
>> transaction mode should set its lock data not to the self number but to the
>> OAT value found
>> at transaction start (i.e. same as snapshot transaction).
>
> No, I think you should go through the logic again. I might be in error, but I
> analyzed engine's GC
> watermarking algorithms for a few days with pen and paper, so this is
> unlikely. Consider this logic:
>
> 1. There is no need to hold GC watermark (lck_data) past transaction number
> if there are no request
> snapshots.
> 2. Cached values you are concerned about cannot go down because we started a
> request snapshot,
> because at the time we create a snapshot oldest active transaction is indeed
> ACTIVE (and thus
> holding cached GC watermarks because of its own existence). Possible
> concurrency during setting up
> of a snapshot is handled by the loop in TRA_setup_request_snapshot and is
> explained in a comment.
>
> I tried to be extra-careful and tested this logic. Please point me to an
> error.
So far i should agree with you here, but i still have a doubts (damn
paranoia :))
>>> This mimics Oracle's behavior in that regard but Oracle doesn't always
>>> implement nesting correctly, we do.
>> Could you explain, please ?
>
> Well, in READ COMMITTED mode Oracle Database always creates snapshot at the
> start of each statement.
> They call it *statement-level read consistency*//. This is why I named
> setting ReadConsistency.
> They also say that their cursors are "always stable".
>
> So yes, let's call new behavior "statement-level read consistency" to avoid
> confusion.
Yes, please, - avoid using "cursor stability" for this issue ! Its already
made a lot of confusion even in this
thread.
To make a brief resume - i can't point to an error in your logic (it
doesn't mean it have no error), but i have
doubt about performance characteristics of BePlusTree-based solution. As for
choice config setting\TPB
flag - it is discussed in this thread and i not going to speak about it here.
Regards,
Vlad
------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
