Richard Yen wrote:
On Nov 18, 2008, at 2:22 PM, Christopher Browne wrote:
Haven't had the Round Tuits to get to it; if you could provide the
beginnings of it, that would make it easier to find the (hopefully
fewer, if effort is shared!) hours of implementation effort.
Well, I've put some effort into getting finishTableAfterCopy and
ANALYZE into a separate thread. The problem that we run into is that
the entire copy_set is under one transaction. This means three things:
1. Entries in sl_table are not visible to other connections to the
database, so tab_id doesn't exist when you try to run
finishTableAfterCopy
2. If you provide full name of a table, reindex and analyze don't do
anything because you're doing this on stale data.
3. If you put each iteration of setAddTable, prepareTableForCopy, and
COPY under remote_worker.c:2906 in its own transaction, if anything
fails, we can't roll back, so the old data is all gone. This implies
that the reindex and analyze need either to happen *in* the
transaction or *after* everything is done.
1 and 2 are expected, but I wonder if you guys have anything to say
about 3. I'm probably going to be speaking some serious DBA-heresy
here, but the way I see it, if you're going to turn a node into a
subscriber, you don't care about whether the data gets lost anyways.
Therefore, we *do* put setAddTable, prepareTableForCopy and COPY in
one transaction, with the reindex and analyze in a separate
transaction, even if anything fails, we'll still be okay. Do you have
anything to say to this?
Are there any other suggestions?
#3 comes as no great surprise.
If you try to "multithread" on the subscriber, then it comes as no
surprise that this needs to represent several transactions on the
subscriber, and that failures could potentially take place in *any* of
those transactions.
It is a given that, by separating *any* of the subscriber work into
another transaction, you have multiple transactions. Once that is
accepted, then, yes, some "heresy" becomes implicit :-).
Yes, indeed, if [SetAddTable, prepareTable, COPY] and [reindex, analyze]
are in separate transactions, that has to be considered OK.
Failure of *any* transaction breaks the subscription attempt; that is
actually no worse than what we have today, because what we have, today,
is one giant transaction for the subscription, where if the slightest
bit of it falls over, the whole transaction rolls back. So while the
change means that there are more transactions (by count) that need to
succeed in order for the whole to succeed, that isn't introducing any
new real *work*.
Seems fine to me!
--
let name="cbbrowne" and tld="ca.afilias.info" in String.concat "@" [name;tld];;
<http://dba2.int.libertyrms.com/>
Christopher Browne
(416) 673-4124 (land)
_______________________________________________
Slony1-general mailing list
[email protected]
http://lists.slony.info/mailman/listinfo/slony1-general
