>>>>> "gp" == Greg Palmer <gregorylpal...@netscape.net> writes:
gp> Relying on the equivalent of crashing the database to perform
gp> backups isn't how professionals get the job done.
well, nevertheless, it is, and should be, supported by SQLite2.
gp> Let's take a simple case of a transaction which consists of
gp> three database updates within a transaction. One of those
gp> writes succeeds, you take a snapshot and then the two other
gp> writes succeed. Everyone concerned with the transaction
gp> believes it succeeded but your snapshot does not show that.
I'm glad you have some rigid procedures that work well for you, but it
sounds like you do not understand how DBMS's actually deal with their
backing store.
You could close the gap by reviewing the glossary entry for ACID.
It's irrelevant whether the transaction spawns one write or
three---the lower parts of the DBMS make updates transactional. As
long as writes are not re-ordered or silently discarded, it's not a
hand-waving recovery-from-chaos process. It's certain. Sometimes
writes ARE lost or misordered, or there are bugs in the DBMS or bad
RAM or who knows what, so I'm not surprised your vendor has given you
hand-waving recovery tools along with a lot of scary disclaimers. Nor
am I surprised that they ask you to follow procedures that avoid
exposing their bugs. But it's just plain wrong that the only way to
achieve a correct backup is with the vendor's remedial freezing tools.
I don't understand why you are dwelling on ``everyone concerned
believes it succeeded but it's not in the backup.'' So what?
Obviously the backup has to stop including things at some point. As
long as the transaction is either in the backup or not in the backup,
the backup is FINE. It's a BACKUP. It has to stop somewhere.
You seem to be concerned that a careful forensic scientist could dig
into the depths of the backup and find some lingering evidence that a
transaction might have once been starting to come into existence. As
far as I'm concerned, that transaction is ``not in the backup'' and
thus fine.
You might also have a look at the, somewhat overcomplicated
w.r.t. database-running-snapshot backups, SQLite2 atomic commit URL
Toby posted:
http://sqlite.org/atomiccommit.html
Their experience points out, filesystems tend to do certain
somewhat-predictable but surprising things to the data inside files
when the cord is pulled, things which taking a snapshot won't do. so,
I was a little surprised to read about some of the crash behaviors
SQLite had to deal with, but, with slight reservation, I stand by my
statement that the database should recover swiftly and certainly when
the cord is pulled. But! it looks like recovering from a
``crash-consistent'' snapshot is actually MUCH easier than a pulled
cord, at least a pulled cord with some of the filesystems SQLite2 aims
to support.
gp> [snapshots] have no knowledge of whether or not one of three
gp> writes required for the database to be consistent have
gp> completed.
it depends on what you mean by consistent. In my language, the
database is always consistent, after each of those three writes. The
DBMS orders the writes carefully to ensure this. Especially in the
case of a lightweight DB like SQLite2 this is the main reason you use
the database in the first place.
gp> Data does not hit the disk instantly, it takes some finite
gp> amount of time in between when the write command is issued for
gp> it to arrive at the disk.
I'm not sure it's critical to my argument, but, snapshots in ZFS have
nothing to do with when data ``hits the disk''.
gp> ZFS promises on disk consistency but as we saw in the recent
gp> thread about "Unreliable for professional usage" it is
gp> possible to have issues. Likewise with database systems.
yes, finally we are in agreement! Here is where we disagree: you want
to add a bunch of ponderous cargo-cult procedures and dire warnings,
like some convoluted way to tell SMF to put SQLite2 into
remedial-backup mode before taking a ZFS snapshot to clone a system.
I want to fix the bugs in SQLite2, or in whatever is broken, so that
it does what it says on the tin.
The first step in doing that is to convince people like you that there
is *necessarily* a bug if the snapshot is not a working backup.
Nevermind the fact that your way simply isn't workable with hundreds
of these lightweight SQLite/db4/whatever databases all over the system
in nameservices and Samba and LDAP and Thunderbird and so on.
Workable or not, it's not _necessary_, and installing this confusing
and incorrect expectation that it's necessary blocks bugs from getting
fixed, and is thus harmful for reliability overall (see _unworkable_
one sentence ago).
HTH.
pgp9wrTDqgHf8.pgp
Description: PGP signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
