On 09/14/2010 04:16 PM, Jeroen van Meeuwen (Kolab Systems) wrote:
Indeed it [cyrus-imapd.cvt_cyrusdb_all] exports the databases on service
shutdown, and then imports them
back on service startup; noted that a condrestart happens on the RPM update,
this about covers every thinkable scenario (or at least those I've had in the
past 7 years).
This seems like it would considerably slow down startup/shutdown on
systems with large databases....
If there is no information in /etc/imapd.conf explicitly, the defaults are
being used -just like the service itself would. The defaults for RPM (for the
purpose of the script) is to use skiplist for all databases.
This is another question that I have: is the set/type of database files
determined at run time (i.e. in imapd.conf) or at compile time? In
particular, the set of databases I have on 2.1.16 only has a very small
intersection with the set described in the forwarded email below:
------------------------------------------------
www:cyrus# pwd
/var/lib/cyrus
www:cyrus# ls
db db.backup2 log msg quota tls_sessions.db
db.backup1 deliver.db mailboxes.db proc socket user
------------------------------------------------
Again, for me this doesn't matter so much, as I'm planning to set up a
new mail server and imapsync mail from the old server to the new one,
but if we're talking about a postinst script which smoothly upgrades
users from one version of cyrus to another, then the database stuff has
to be solidly nailed down. Right now I'm confused about what determines
the number, type, and names of db files in various versions of cyrus.
=========================================================================
-------- Original Message --------
Subject: Re: Wiki, information on database back ends.
Date: Tue, 14 Sep 2010 16:30:55 -0500
From: Dan White <[email protected]>
To: Patrick Goetz <[email protected]>
CC: [email protected]
On 14/09/10 14:24 -0500, Patrick Goetz wrote:
>There's been some discussion on the Debian cyrus list about how to
>automate upgrades from cyrus 2.n.k to cyrus 2.m.j. Jeroen van
>Meeuwen (on both lists) suggested that the cyrus RPM package features
>a utility called cyrus-imapd.cvt_cyrusdb_all which might be useful
>for this.
>
>I've been looking at this script, and it mostly appears to be using
>cvt_cyrusdb to convert particular db files to Cyrus skiplists and
>then back again to the original db backend format. I can't follow
>the script completely as it seems to rely on DB configuration details
>found in the imapd.conf file I don't have in my Debian 2.1.16 imap
>server, and it's also not clear how the script is run.
>
>This raises a number of questions, though:
>
>1.
>Cyrus skiplists? I thought all the DB files were in Berkeley DB
>format. I tried to find some documentation on skiplists, but only
>found an old message to the developer list from Bron Gondwana
>discussing skiplist bugs
>(http://markmail.org/message/zbaq765brbg2acfj).
The current imapd.conf hints at the available data types for each cyrus
database. Depending on the option, the following database types are
available (in 2.3.16):
annotation_db: skiplist
Allowed values: berkeley, berkeley-hash, skiplist
duplicate_db: berkeley-nosync
Allowed values: berkeley, berkeley-nosync, berkeley-hash,
berkeley-hash-nosync, skiplist, sql
mboxkey_db: skiplist
Allowed values: berkeley, skiplist
mboxlist_db: skiplist
Allowed values: flat, berkeley, berkeley-hash, skiplist
ptscache_db: berkeley
Allowed values: berkeley, berkeley-hash, skiplist
quota_db: quotalegacy
Allowed values: flat, berkeley, berkeley-hash, skiplist, sql, quotalegacy
seenstate_db: skiplist
Allowed values: flat, berkeley, berkeley-hash, skiplist
subscription_db: flat
Allowed values: flat, berkeley, berkeley-hash, skiplist
statuscache_db: berkeley-nosync
Allowed values: berkeley, berkeley-nosync, berkeley-hash,
berkeley-hash-nosync, skiplist
tlscache_db: berkeley-nosync
Allowed values: berkeley, berkeley-nosync, berkeley-hash,
berkeley-hash-nosync, skiplist, sql
userdeny_db: flat
Allowed values: flat, berkeley, berkeley-hash, skiplist, sql
http://www.cyrusimap.org/docs/cyrus-imapd/2.3.16/internal/database-formats.php
has some discussion of the databases.
I don't know why the (rpm) upgrade script chose skiplist as an
intermediate format, rather than flat. Perhaps skiplist hasn't changed
much between major versions, and offers a speed benefit.
The script seems to use the file command to determine the current
database types, since they can be changed by an admin.
>2.
>The Redhat cvt_cyrusdb_all script seems to assume a specific set of
>database files. Is the set of cyrus imap DB files fixed, and if so
>what are they? Is there any documentation on what each database file
>contains? This would be very useful to people trying to convert older
>cyrus IMAP installations to new ones.
As far as skiplist, that seems to be a proprietary format.
As far as I can tell, all databases use the same underlying structure
(except for quotalegacy), which is defined in the database-formats above.
flat: ASCII text file
berkeley: berkeley db (with safer values).
berkeley-nosync: berkeley db with the DB_TXN_NOSYNC flag set.
berkeley-hash: berkeley db with the DB_HASH flag set.
berkeley-hash-nosync: berkeley db with both DB_TXN_NOSYNC and DB_HASH
flags?
sql: A mysql, pgsql, or sqlite store.
quotalegacy: See database-formats
skiplist: ?
With cyrus version 2.3.13 and above, the cyr_dbtool can be used to
insert and remove records manually for each database.
--
Dan White
_______________________________________________
Pkg-Cyrus-imapd-Debian-devel mailing list
[email protected]
http://lists.alioth.debian.org/mailman/listinfo/pkg-cyrus-imapd-debian-devel
