Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-29 Thread Andrew Morgan
On Wed, 29 Sep 2010, Tomasz Chmielewski wrote:

> Hmm - I added this to imapd.conf:
>
> annotation_db: skiplist
> duplicate_db: skiplist
> mboxlist_db: skiplist
> ptscache_db: skiplist
> quota_db: skiplist
> seenstate_db: skiplist
> tlscache_db: skiplist
>
>
> When starting cyrus, I have this:
>
> Sep 29 02:53:48 omega cyrus/master[1089]: process started
> Sep 29 02:53:48 omega cyrus/ctl_cyrusdb[1090]: recovering cyrus databases
> Sep 29 02:53:48 omega cyrus/ctl_cyrusdb[1090]: done recovering cyrus databases
> Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: DBERROR db4: Program version 
> 4.2 doesn't match environment version
> Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: DBERROR: dbenv->open 
> '/shared/var/lib/cyrus/db' failed: Invalid argument
> Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: DBERROR: init() on berkeley
> Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: duplicate_prune: pruning back 3 
> days
> Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: duplicate_prune: purged 0 out 
> of 0 entries
> Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: expunged 0 out of 0 messages 
> from 0 mailboxes
> Sep 29 02:53:49 omega cyrus/tls_prune[1092]: tls_prune: purged 0 out of 0 
> entries
> Sep 29 02:53:49 omega cyrus/master[1089]: ready for work
> Sep 29 02:53:49 omega cyrus/ctl_cyrusdb[1093]: checkpointing cyrus databases
> Sep 29 02:53:49 omega cyrus/ctl_cyrusdb[1093]: done checkpointing cyrus 
> databases
>
>
> # file /shared/var/lib/cyrus/db/*
> /shared/var/lib/cyrus/db/__db.001:   data
> /shared/var/lib/cyrus/db/__db.002:   data
> /shared/var/lib/cyrus/db/__db.003:   data
> /shared/var/lib/cyrus/db/__db.004:   data
> /shared/var/lib/cyrus/db/__db.005:   data
> /shared/var/lib/cyrus/db/log.01: Berkeley DB (Log, version 8, native 
> byte-order)
> /shared/var/lib/cyrus/db/skipstamp:  data
>
>
> The error and "Berkeley DB" log file is there even if I empty this 
> directory, and start Cyrus.
>
> Did I miss some value in imapd.conf?

Cyrus is always linked with Berkeley DB, so it always tries to init the 
Berkeley DB environment.  Even with all your backends set to "skiplist", 
you'll still see the Berkeley DB log files in {configdir}/db/.  You can 
safely ignore them.

I'm not sure why you still get Berkeley DB errors when starting Cyrus.  I 
have converted everything to skiplist, and I do not get those errors.

Andy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Tomasz Chmielewski
On 28.09.2010 12:55, Bron Gondwana wrote:
> On Tue, Sep 28, 2010 at 12:13:14PM +0200, Tomasz Chmielewski wrote:
>> On 28.09.2010 11:55, Michael Menge wrote:
>>
>>> Replication was introduced in 2.3.x. There are other features in 2.3.x
>>> I don't want to live with out (e.g. delayed expunge). There was a
>>> diskussion on the lists about that Debian wants to upgrade cyrus.
>>> The main problem is the upgrade path (update of BDB Databases).
>>
>> Assuming I start with empty mail pool (no accounts) - how can I trigger
>> the creation of Cyrus databases (in skiplist format - I assume adding
>> relevant skiplist info to the config file is not enough)?
> 
> All databases will create automatically upon use.  Just set the type
> in the config file.

Hmm - I added this to imapd.conf:

annotation_db: skiplist
duplicate_db: skiplist
mboxlist_db: skiplist
ptscache_db: skiplist
quota_db: skiplist
seenstate_db: skiplist
tlscache_db: skiplist


When starting cyrus, I have this:

Sep 29 02:53:48 omega cyrus/master[1089]: process started
Sep 29 02:53:48 omega cyrus/ctl_cyrusdb[1090]: recovering cyrus databases
Sep 29 02:53:48 omega cyrus/ctl_cyrusdb[1090]: done recovering cyrus databases
Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: DBERROR db4: Program version 4.2 
doesn't match environment version
Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: DBERROR: dbenv->open 
'/shared/var/lib/cyrus/db' failed: Invalid argument
Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: DBERROR: init() on berkeley
Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: duplicate_prune: pruning back 3 
days
Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: duplicate_prune: purged 0 out of 
0 entries
Sep 29 02:53:49 omega cyrus/cyr_expire[1091]: expunged 0 out of 0 messages from 
0 mailboxes
Sep 29 02:53:49 omega cyrus/tls_prune[1092]: tls_prune: purged 0 out of 0 
entries
Sep 29 02:53:49 omega cyrus/master[1089]: ready for work
Sep 29 02:53:49 omega cyrus/ctl_cyrusdb[1093]: checkpointing cyrus databases
Sep 29 02:53:49 omega cyrus/ctl_cyrusdb[1093]: done checkpointing cyrus 
databases


# file /shared/var/lib/cyrus/db/*
/shared/var/lib/cyrus/db/__db.001:   data
/shared/var/lib/cyrus/db/__db.002:   data
/shared/var/lib/cyrus/db/__db.003:   data
/shared/var/lib/cyrus/db/__db.004:   data
/shared/var/lib/cyrus/db/__db.005:   data
/shared/var/lib/cyrus/db/log.01: Berkeley DB (Log, version 8, native 
byte-order)
/shared/var/lib/cyrus/db/skipstamp:  data


The error and "Berkeley DB" log file is there even if I empty this directory, 
and start Cyrus.

Did I miss some value in imapd.conf?



-- 
Tomasz Chmielewski
http://wpkg.org

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Andre Felipe Machado
Hello
AFAIK, cyrus needs posix file locks and mmap support.
GlusterFS needs FUSE and it only supports writable mmap files after kernel
2.6.26 or so.
Therefore, you need recent kernel and recent fuse.
Further, you need to extremely fine tune your configuration, as the most robust
clustered filesystems suffer under load over small files.
Its their achiles heel...
And cyrus uses small files and "hot spot files".
We are evaluating clustered fs like GlusterFS, GFS, OCFS2 (and other
shared/mirrored alternatives) since 2007 and they are "not there" yet for such
heavy load profile (small files).
GlusterFS is the most elegant, flexible and promising of them.
But clustered filesystems worth their performance penalty if you need
active-active servers.
For such active-active, you may consider using Dovecot, that was designed having
taking into account clustered filesystems and shared storage and multiple
servers. It has four file locking methods to choose for best suitability for a
given storage method and even sql backends for mailer internal db (not for
messages) .
But dovecot does not support shared folders across multiple backends yet as
cyrus. And *this* is a killer feature for us.
If you wants active-passive configuration, it is best to stay away from any
clustered filesystem, to not pay the heavy performance cost for small files (and
another layer of bugs) without REALLY needing the active-active fs sharing.
Keep it simple.
Maybe you even do not need "real time" up to the microsecond
replication/mirroring  or sharing.
This allows even more simple and or reliable or recoverable or less resource
hungry solutions as more "sync delay" is accepted.
Low level (byte or even file) solutions will replicate crashes like bdb
corruptions and will slow down your app.
Byte, block, file replications need REALY FAST and EXTREMELY LOW LATENCY
networks, also. Notably for small files.
Answer yourself: What you desire? what you actually need?
Maybe you consider worthwhile to read some articles to bring some light to the
subject. 
Also, remember that glusterfs evolved since the written article and newer
versions use somewhat different confs and tuning, that depends of YOUR
infrastructure.
You will need some translation service to articles on brazilian portuguese. Look
for "Translate this page." link near bottom of each page.
Good luck.
Andre Felipe Machado



[0] http://www.techforce.com.br/news/linux_blog/glusterfs_tuning_small_files
[1 ]
http://www.techforce.com.br/news/linux_blog/lvm_raid_xfs_ext3_tuning_for_small_files_parallel_i_o_on_debian
[2]
http://www.techforce.com.br/news/linux_blog/storage_space_for_debian_on_ibm_ds_8300
[3]
http://www.techforce.com.br/news/linux_blog/how_to_configure_multipath_debian_centos_for_ibm_ds8300
[4] http://www.techforce.com.br/news/linux_blog/postgresql_ha_p1_5_com_glusterfs
[5] http://www.techforce.com.br/news/linux_blog/postgresql_ha_p1_com_glusterfs
[6]
http://www.techforce.com.br/news/media/multimedia/video_1_da_palestra_postgresql_em_alta_disponibilidade_parte_1_usando_sistema_de_arquivos_distribuido_glusterfs
[7] 
http://www.techforce.com.br/news/linux_blog/red_hat_cluster_suite_debian_etch
[8]
http://www.techforce.com.br/news/linux_blog/virtualizacao_e_servico_de_arquivos_em_cluster_ha_com_debian_etch_parte_1
[9]
http://www.techforce.com.br/news/linux_blog/virtualizacao_e_servico_de_arquivos_em_cluster_ha_com_debian_etch_parte_2
[10]
http://www.techforce.com.br/news/linux_blog/virtualizacao_e_servico_de_arquivos_em_cluster_ha_com_debian_etch_parte_3
[11] 
http://www.techforce.com.br/news/linux_blog/postgresql_ha_p1_5_com_glusterfs
[12] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595370



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Michael Menge

Quoting Tomasz Chmielewski :


On 28.09.2010 15:01, John Madden wrote:

Still, we need to have Cyrus database, mail storage accessible for
both servers. I though using glusterfs for it would be a good idea
(assuming Cyrus only runs on one of the servers at a given time).


IMO, don't use glusterfs for this. I found it to not even be sufficient
for a PHP session store; it'll certainly fall over with IMAP loads.


Any other suggestions? There is an alternatives like Ceph[1], but it is
just too new (and potentially can have some edge cases).

DRBD + GFS/OCFS2 just seem too complex for such setup.

Other than that, I use glusterfs in several setups, and I don't have any
dramatic performance problems with it (still slower than bare metal of
course) - will depend on workload and expected performance of course.


[1] http://ceph.newdream.net/about/


Most Cluster-/Sharedfilesystems are good with few big files. But  
because of the metadatahandling these FS all lose performance if you  
have many small files, and cyrus has many files.





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen

smime.p7s
Description: S/MIME Signatur

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread John Madden
> Any other suggestions? There is an alternatives like Ceph[1], but it is
> just too new (and potentially can have some edge cases).
>
> DRBD + GFS/OCFS2 just seem too complex for such setup.

If you're doing failover, you don't need a cluster filesystem.  You can 
use just plain DRDB+ext4 if you don't have real shared storage.

John





-- 
John Madden
Sr UNIX Systems Engineer
Ivy Tech Community College of Indiana
jmad...@ivytech.edu

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Tomasz Chmielewski
On 28.09.2010 15:01, John Madden wrote:
>> Still, we need to have Cyrus database, mail storage accessible for
>> both servers. I though using glusterfs for it would be a good idea
>> (assuming Cyrus only runs on one of the servers at a given time).
>
> IMO, don't use glusterfs for this. I found it to not even be sufficient
> for a PHP session store; it'll certainly fall over with IMAP loads.

Any other suggestions? There is an alternatives like Ceph[1], but it is 
just too new (and potentially can have some edge cases).

DRBD + GFS/OCFS2 just seem too complex for such setup.

Other than that, I use glusterfs in several setups, and I don't have any 
dramatic performance problems with it (still slower than bare metal of 
course) - will depend on workload and expected performance of course.


[1] http://ceph.newdream.net/about/


-- 
Tomasz Chmielewski
http://wpkg.org

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread John Madden
> Still, we need to have Cyrus database, mail storage accessible for
> both servers. I though using glusterfs for it would be a good idea
> (assuming Cyrus only runs on one of the servers at a given time).

IMO, don't use glusterfs for this.  I found it to not even be sufficient 
for a PHP session store; it'll certainly fall over with IMAP loads.

John





-- 
John Madden
Sr UNIX Systems Engineer
Ivy Tech Community College of Indiana
jmad...@ivytech.edu

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Bron Gondwana
On Tue, Sep 28, 2010 at 12:13:14PM +0200, Tomasz Chmielewski wrote:
> On 28.09.2010 11:55, Michael Menge wrote:
> 
> > Replication was introduced in 2.3.x. There are other features in 2.3.x
> > I don't want to live with out (e.g. delayed expunge). There was a
> > diskussion on the lists about that Debian wants to upgrade cyrus.
> > The main problem is the upgrade path (update of BDB Databases).
> 
> Assuming I start with empty mail pool (no accounts) - how can I trigger 
> the creation of Cyrus databases (in skiplist format - I assume adding 
> relevant skiplist info to the config file is not enough)?

All databases will create automatically upon use.  Just set the type
in the config file.

Bron.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Tomasz Chmielewski
On 28.09.2010 11:55, Michael Menge wrote:

> Replication was introduced in 2.3.x. There are other features in 2.3.x
> I don't want to live with out (e.g. delayed expunge). There was a
> diskussion on the lists about that Debian wants to upgrade cyrus.
> The main problem is the upgrade path (update of BDB Databases).

Assuming I start with empty mail pool (no accounts) - how can I trigger 
the creation of Cyrus databases (in skiplist format - I assume adding 
relevant skiplist info to the config file is not enough)?


-- 
Tomasz Chmielewski
http://wpkg.org

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Michael Menge

Quoting Tomasz Chmielewski :


On 28.09.2010 10:56, Michael Menge wrote:


Cyrus depends on locks and mmap, so your fs must support them.
I had written a summery of the diskussions about Cyrus and HA in the
old wiki. But the wiki was replaced by the new wiki. I will have a look
if I have a copy.


I would be grateful.


I didn't find the Wiki Text, but the thread that was the base of this
Wiki-Text

http://www.irbs.net/internet/info-cyrus/0611/0279.html




If you plan to run in active-passive mode, did you considre Cyrus
replication? You will need twice the disk space, but you remove a single
point of failure (glustefs)


Glusterfs is there to avoid SPOF - as the filesystem sits on two
servers. So assuming I won't do rm -rf /gluster-filesystem, it should be
quite safe. And it too needs twice the disk space, since it's replicated
with glusterfs on both servers.


So there is no differens in diskspace. If glustefs keeps two copies of
each file or if you have two Cyrus-Servers. But with Cyrus Replication
you don't have the problem with mmap and locking. It may help not to use
BDB for the databases. But i don't know how good skiplist is in 2.2.13.
Many skiplist bugs have been fixed in 2.3.x




However, I'm of course open to better alternatives.

I'm running Debian Lenny, which ships with Cyrus 2.2.13 - not sure if
Cyrus replication is possible there? I'd like to stick with distro
packages, but if a newer Cyrus version provides features which let you
do HA without too much hackarounds, I'll consider upgrading.



Replication was introduced in 2.3.x. There are other features in 2.3.x
I don't want to live with out (e.g. delayed expunge). There was a
diskussion on the lists about that Debian wants to upgrade cyrus.
The main problem is the upgrade path (update of BDB Databases).





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen

smime.p7s
Description: S/MIME Signatur

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Tomasz Chmielewski
On 28.09.2010 10:56, Michael Menge wrote:

> Cyrus depends on locks and mmap, so your fs must support them.
> I had written a summery of the diskussions about Cyrus and HA in the
> old wiki. But the wiki was replaced by the new wiki. I will have a look
> if I have a copy.

I would be grateful.


> If you plan to run in active-passive mode, did you considre Cyrus
> replication? You will need twice the disk space, but you remove a single
> point of failure (glustefs)

Glusterfs is there to avoid SPOF - as the filesystem sits on two 
servers. So assuming I won't do rm -rf /gluster-filesystem, it should be 
quite safe. And it too needs twice the disk space, since it's replicated 
with glusterfs on both servers.


However, I'm of course open to better alternatives.

I'm running Debian Lenny, which ships with Cyrus 2.2.13 - not sure if 
Cyrus replication is possible there? I'd like to stick with distro 
packages, but if a newer Cyrus version provides features which let you 
do HA without too much hackarounds, I'll consider upgrading.


-- 
Tomasz Chmielewski
http://wpkg.org


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Michael Menge

Quoting Tomasz Chmielewski :


How do you manage your Cyrus installations highly-available?

I though a minimal example could be like below:


   internet
  |
server1 - server2


There would be Heartbeat/Pacemaker running on both servers. Its role  
would be:


- assign "Cyrus IP" to a given server,
- start Cyrus where "Cyrus IP" is up.


Still, we need to have Cyrus database, mail storage accessible for  
both servers. I though using glusterfs for it would be a good idea  
(assuming Cyrus only runs on one of the servers at a given time).


However, something doesn't work with it very well when Cyrus data is  
on a glusterfs mount point (if I move it to a local disk, everything  
works well):




Cyrus depends on locks and mmap, so your fs must support them.
I had written a summery of the diskussions about Cyrus and HA in the
old wiki. But the wiki was replaced by the new wiki. I will have a look
if I have a copy.

If you plan to run in active-passive mode, did you considre Cyrus
replication? You will need twice the disk space, but you remove a single
point of failure (glustefs)


Regards

 Michael Mege


M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen

smime.p7s
Description: S/MIME Signatur

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Tomasz Chmielewski
On 28.09.2010 09:13, Pascal Gienger wrote:
>
> Le 28 sept. 2010 à 08:50, Tomasz Chmielewski a écrit :
>> Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: Program version 
>> 4.2 doesn't match environment version
>
> Are you sure on each node the _SAME_ Cyrus version linked to the _SAME_ bdb 
> libs is running?

100% sure. If I copy everything off glusterfs to a local filesystem, 
Cyrus doesn't report any errors.


> And - just a little side note - you can dump bdb in favor to skiplist... I 
> bet you'll have much less problems in your cluster environment setup.

Yep, I found more or less it could be some mmap problem with BDB.

Is there a way to convert the existing BDB databases to skiplist? Or, 
initialize "empty" skiplist databases for Cyrus?


-- 
Tomasz Chmielewski
http://wpkg.org

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Sebastian Hagedorn
--On 28. September 2010 08:50:00 +0200 Tomasz Chmielewski  
wrote:



How do you manage your Cyrus installations highly-available?


Check the archives. There have been many discussions regarding this.


I though a minimal example could be like below:


   internet
  |
server1 - server2


There would be Heartbeat/Pacemaker running on both servers. Its role
would be:

- assign "Cyrus IP" to a given server,
- start Cyrus where "Cyrus IP" is up.


Still, we need to have Cyrus database, mail storage accessible for both
servers. I though using glusterfs for it would be a good idea (assuming
Cyrus only runs on one of the servers at a given time).


We use a similar setup with standard ext3 file systems that are mounted and 
unmounted as needed; in our case that's done by the RHEL 3 Cluster Suite. 
That's been working great for almost 6 years now.

--
.:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:.
.:.Regionales Rechenzentrum (RRZK).:.
.:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:.

p7sisIZ9P1qN8.p7s
Description: S/MIME cryptographic signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: high-availability Cyrus (i.e. glusterfs)?

2010-09-28 Thread Pascal Gienger

Le 28 sept. 2010 à 08:50, Tomasz Chmielewski a écrit :
> Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: Program version 
> 4.2 doesn't match environment version

Are you sure on each node the _SAME_ Cyrus version linked to the _SAME_ bdb 
libs is running?

And - just a little side note - you can dump bdb in favor to skiplist... I bet 
you'll have much less problems in your cluster environment setup.

Pascal

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


high-availability Cyrus (i.e. glusterfs)?

2010-09-27 Thread Tomasz Chmielewski
How do you manage your Cyrus installations highly-available?

I though a minimal example could be like below:


   internet
  |
server1 - server2


There would be Heartbeat/Pacemaker running on both servers. Its role would be:

- assign "Cyrus IP" to a given server,
- start Cyrus where "Cyrus IP" is up.


Still, we need to have Cyrus database, mail storage accessible for both 
servers. I though using glusterfs for it would be a good idea (assuming Cyrus 
only runs on one of the servers at a given time).

However, something doesn't work with it very well when Cyrus data is on a 
glusterfs mount point (if I move it to a local disk, everything works well):

Sep 28 01:10:10 omega cyrus/master[21504]: ready for work
Sep 28 01:10:10 omega cyrus/master[21728]: about to exec /usr/sbin/ctl_cyrusdb
Sep 28 01:10:10 omega cyrus/master[21729]: about to exec 
/usr/lib/cyrus/bin/notifyd
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: Program version 
4.2 doesn't match environment version
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: dbenv->open 
'/shared/var/lib/cyrus/db' failed: Invalid argument
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: init() on berkeley
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: checkpointing cyrus databases
Sep 28 01:10:10 omega cyrus/notify[21729]: DBERROR db4: Program version 4.2 
doesn't match environment version
Sep 28 01:10:10 omega cyrus/notify[21729]: DBERROR: dbenv->open 
'/shared/var/lib/cyrus/db' failed: Invalid argument
Sep 28 01:10:10 omega cyrus/notify[21729]: DBERROR: init() on berkeley
Sep 28 01:10:10 omega cyrus/notify[21729]: executed
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: archiving database file: 
/shared/var/lib/cyrus/annotations.db
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: txn_checkpoint 
interface requires an environment configured for the transaction subsystem
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: couldn't checkpoint: 
Invalid argument
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: sync 
/shared/var/lib/cyrus/db: cyrusdb error
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: 
DB_ENV->log_archive interface requires an environment configured for the 
logging subsystem
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: error listing log 
files: Invalid argument
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: archive 
/shared/var/lib/cyrus/db: cyrusdb error
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: archiving database file: 
/shared/var/lib/cyrus/mailboxes.db
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: txn_checkpoint 
interface requires an environment configured for the transaction subsystem
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: couldn't checkpoint: 
Invalid argument
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: sync 
/shared/var/lib/cyrus/db: cyrusdb error
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: 
DB_ENV->log_archive interface requires an environment configured for the 
logging subsystem
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: error listing log 
files: Invalid argument
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: archive 
/shared/var/lib/cyrus/db: cyrusdb error
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: txn_checkpoint 
interface requires an environment configured for the transaction subsystem
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: couldn't checkpoint: 
Invalid argument
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: sync 
/shared/var/lib/cyrus/db: cyrusdb error
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR db4: 
DB_ENV->log_archive interface requires an environment configured for the 
logging subsystem
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: error listing log 
files: Invalid argument
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: DBERROR: archive 
/shared/var/lib/cyrus/db: cyrusdb error
Sep 28 01:10:10 omega cyrus/ctl_cyrusdb[21728]: done checkpointing cyrus 
databases



-- 
Tomasz Chmielewski
http://wpkg.org

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/