cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M
Hi,
I run into a problem on an old clearos server, where the cyrus shutdown
always failed at step exporting databases.
As I checked the situation using ps ax on an other console, I found
that, it was exporting deliver.db.skiplist file, which failed after a
lng time (some minutes).
I checked that file on the filesystem, I saw the file size is 2048MB,
which seems a limit for me and I suspect the problem should be that,
the 32 bit cyrus cannot write more data to that file and caused the
problem.
As I read the db_export.log, that confirmed my theory, file size limit
exceeded.
This kind of issue causes long hangup at system shutdown and I guess
causes other potential problems, as this file could be truncated and
useless.
My question, do you have any idea/hint, what to do to get it work?
For example, changing export.db format to something else? (How?)
Thank you!
I copied some important information below, please check it.
db_export.log:
cvt_cyrusdb_all version: 1.2.1db_checkpoint: open: No such file or
directory/usr/lib/cyrus-imapd/cvt_cyrusdb_all: line 199: 9150 File
size limit exceeded$cvt_cyrusdb "$target" "$old_db"
"${target}.skiplist" skiplistERROR: unable to convert
/var/lib/imap/deliver.db from berkeley to skiplistremoved
`/var/lib/imap/db/log.01'removed
`/var/lib/imap/db/__db.001'removed `/var/lib/imap/db/__db.002'removed
`/var/lib/imap/db/__db.003'removed `/var/lib/imap/db/__db.004'removed
`/var/lib/imap/db/__db.005'removed `/var/lib/imap/db/__db.006'removed
`/var/lib/imap/db/__db.007'removed `/var/lib/imap/db/__db.008'removed
`/var/lib/imap/db/__db.009'removed `/var/lib/imap/db/__db.010'removed
`/var/lib/imap/db/__db.011'removed `/var/lib/imap/db/__db.012'
The system details:
* Clearos 5.2 32 bit
* Cyrus is 2.3.11
* kernel 2.6.18-308.1.1.v5
* filesystem is ext3 (no problem to write 200-300GB into a single
file)
* size of email folder (/var/spool/imap/*): 1007 GB
* size of /var/lib/imap/deliver.db 48MB
Other interesting config files:
db.cfg.cache content
annotation_db=skiplistduplicate_db=berkeleymboxkey_db=skiplistmboxlist_
db=skiplistptscache_db=berkeleyquota_db=quotalegacyseenstate_db=skiplis
tsieve_version=2.2.3subscription_db=flattlscache_db=berkeley
/var/lib/imap/db/DB_CONFIG
set_cachesize 0 8388608 8set_lg_regionmax 5242880set_lg_bsize 8097152
Thank you!
István (Steve)
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M
> Hi, > > I run into a problem on an old clearos server, where the cyrus shutdown > always failed at step exporting databases. > As I checked the situation using ps ax on an other console, I found > that, it was exporting deliver.db.skiplist file, which failed after a > lng time (some minutes). > I checked that file on the filesystem, I saw the file size is 2048MB, > which seems a limit for me and I suspect the problem should be that, > the 32 bit cyrus cannot write more data to that file and caused the > problem. > As I read the db_export.log, that confirmed my theory, file size limit > exceeded. Hi, The question is why is the deliver db > 2GB in skiplist format? Is it normal or do you have a corrupt BDB db or does your db pruning not work for deliverdb. I think that should be something like 'delprune cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf. I think the easiest way would be to make sure you have pruning configured correctly, then change config of deliver db to skiplist, and start without a db so a new, empty deliver db is created. Then have an eye on the db file to see if it grows again to almost 2GB. If it doesn't grow so much, you should be fine. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M
2020. 06. 20, szombat keltezéssel 21.31-kor Simon Matter ezt írta:
>
> Hi,
>
> The question is why is the deliver db > 2GB in skiplist format? Is it
> normal or do you have a corrupt BDB db or does your db pruning not work
> for deliverdb. I think that should be something like 'delprune
> cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf.
>
> I think the easiest way would be to make sure you have pruning configured
> correctly, then change config of deliver db to skiplist, and start without
> a db so a new, empty deliver db is created.
>
> Then have an eye on the db file to see if it grows again to almost 2GB. If
> it doesn't grow so much, you should be fine.
>
> Regards,
> Simon
>
>
Hi,
Thank you for your answer!
I have this in my config at EVENTS section:
EVENTS {
# this is required
checkpointcmd="ctl_cyrusdb -c" period=30
# this is only necessary if using duplicate delivery suppression
delprune cmd="ctl_deliver -E 3" period=1440
# this is only necessary if caching TLS sessions
tlsprune cmd="tls_prune" period=1440
}
Regarding to users, we do not see any anomalies on daily use or restarting the
server sometimes, only this exporting failed.
I issued the mentioned command on command line (cyr_expire -D 7 -E 3 -X 7) it
finished in around 1-2 seconds. (At this moment there are no online users as
this is weekend).
Additional information I forgot to mention:
there are a lot of shared email folder of the catchall account to share common
mailboxes.
I got this error after some lines, when I run /usr/lib/cyrus-imapd/ctl_deliver
-d
fatal error: Internal error: assertion failed: duplicate.c: 344: (datalen ==
sizeof(time_t)) || (datalen == sizeof(time_t) + sizeof(unsigned long))
Probably deliver.db itself is corrupted?
Cheers,
István
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M
2020. 06. 20, szombat keltezéssel 21.31-kor Simon Matter ezt írta: > Hi, > > The question is why is the deliver db > 2GB in skiplist format? Is it > normal or do you have a corrupt BDB db or does your db pruning not work > for deliverdb. I think that should be something like 'delprune > cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf. > > I think the easiest way would be to make sure you have pruning configured > correctly, then change config of deliver db to skiplist, and start without > a db so a new, empty deliver db is created. > > Then have an eye on the db file to see if it grows again to almost 2GB. If > it doesn't grow so much, you should be fine. > > Regards, > Simon Hi, Something definitely not seems fine: -bash-3.2$ /usr/lib/cyrus-imapd/cyr_expire -E 3 -D 7 -X 7 -v expunged 0 out of 0 messages from 0 mailboxes The deliver.db still about 48MB. Tomorrow I will continue. Thanks, István Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
