Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M
> 2020. 06. 20, szombat keltezéssel 21.31-kor Simon Matter ezt írta: >> Hi, >> >> The question is why is the deliver db > 2GB in skiplist format? Is it >> normal or do you have a corrupt BDB db or does your db pruning not work >> for deliverdb. I think that should be something like 'delprune >> cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf. >> >> I think the easiest way would be to make sure you have pruning >> configured >> correctly, then change config of deliver db to skiplist, and start >> without >> a db so a new, empty deliver db is created. >> >> Then have an eye on the db file to see if it grows again to almost 2GB. >> If >> it doesn't grow so much, you should be fine. >> >> Regards, >> Simon > > Hi, > > Something definitely not seems fine: > > -bash-3.2$ /usr/lib/cyrus-imapd/cyr_expire -E 3 -D 7 -X 7 -v Please make sure the options here are also valid for your cyrus version. However, I also guess your deliver.db is corrupted somehow. From my own experience skiplist dbs are easier to handle than bdb and using skiplist only has not shown any issues. Regards, Simon > > expunged 0 out of 0 messages from 0 mailboxes > > The deliver.db still about 48MB. > > Tomorrow I will continue. > > Thanks, > István > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M
> Hi, > > I run into a problem on an old clearos server, where the cyrus shutdown > always failed at step exporting databases. > As I checked the situation using ps ax on an other console, I found > that, it was exporting deliver.db.skiplist file, which failed after a > lng time (some minutes). > I checked that file on the filesystem, I saw the file size is 2048MB, > which seems a limit for me and I suspect the problem should be that, > the 32 bit cyrus cannot write more data to that file and caused the > problem. > As I read the db_export.log, that confirmed my theory, file size limit > exceeded. Hi, The question is why is the deliver db > 2GB in skiplist format? Is it normal or do you have a corrupt BDB db or does your db pruning not work for deliverdb. I think that should be something like 'delprune cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf. I think the easiest way would be to make sure you have pruning configured correctly, then change config of deliver db to skiplist, and start without a db so a new, empty deliver db is created. Then have an eye on the db file to see if it grows again to almost 2GB. If it doesn't grow so much, you should be fine. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: imap clients say i have 4K messages but spool has 12894 files
> On Tue, 2020-05-26 at 08:47 -0500, Nic Bernstein wrote: >> >> |expunge_mode:| delayed >> >> The mode in which messages (and their corresponding cache >> entries) are expunged. “semidelayed” mode is the old behavior >> in >> which the message files are purged at the time of the >> EXPUNGE, >> but index and cache records are retained to facilitate >> QRESYNC. >> In “delayed” mode, which is the default since Cyrus 2.5.0, > > So this doesn't apply to my 2.4.17 then does it? How did you install your 2.4.17 server? Because you don't run the latest of the 2.4 series I guess you have installed it from packages? If so it's possible that you distribution uses different configuration than the default, at build time. You should check this. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: [Help] Cyrus 2.4.17 segfault
> Hello, > > we are experiencing a strange problem on cyrus-imapd 2.4.17. > > An user user/a has full ACL to another mailbox user/b. When the user/a > SELECT a folder on user/b where he has access the imap process crashes. > > $ telnet cyrus.example.com 143 > Trying 10.10.10.10 ... > Connected to cyrus.example.com. > Escape character is '^]'. > * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN SASL-IR] > cyrus.example.com Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6 server ready > a authenticate plain dsfrrFwaWNlQGNzaS5pdABveGN5cnVzAENdfsdfMx > a OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxte QUOTA > MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN > MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SORT SORT=MODSEQ > SORT=DISPLAY THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE > LIST-EXTENDED WITHIN QRESYNC SCAN XLIST URLAUTH URLAUTH=BINARY > X-NETSCAPE LOGINDISABLED COMPRESS=DEFLATE IDLE] Success (no protection) > SESSIONID= > a SELECT "user/b/SOME THINGS/WATER" > Connection closed by foreign host. > > > Could you suggest me some way to resolve this problem? I don't know but if it's a bug then it's probably solved in current version 2.4.20. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: script to check quota failed
> Hi, > > seems that there is a problem with your cyrus quota tree > (/var/lib/cyrus/quota) or at least some of its files. In the case of RHEL6 this will be /var/lib/imap/quota/. I suggest to look at the quota files there, they have names like user. and are text files with two lines. Should be easy to find the broken file. Regards, Simon > > Maybe strace command may help you find which files are corrupt: > su - cyrus -c "strace /usr/lib/cyrus-imapd/quota user" > > Anthony Prades > > On 3/18/19 2:32 PM, Stephane Branchoux wrote: >> Hello, >> >> We run cyrus-imapd-2.3.16-6 on a Red Hat Enterprise Linux Server >> release 6.10. >> >> 2 days ago, a power crash occured in our Datacenter. >> >> After reboot of cyrus imap, all seems ok except our script to check >> quota : >> >> su - cyrus -c "/usr/lib/cyrus-imapd/quota user" >> failed building quota list for 'user': System I/O error: %m >> >> I tried : >> >> su - cyrus -c "/usr/lib/cyrus-imapd/quota -f" >> failed building quota list for '*': System I/O error: %m >> >> Any idea ? >> >> Many thanks in advance >> >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Which imap command to rename a root mailbox while maintaining its partition
> Hi, > > seems that there is a problem with your cyrus quota tree > (/var/lib/cyrus/quota) or at least some of its files. In the case of RHEL6 this will be /var/lib/imap/quota/. I suggest to look at the quota files there, they have names like user. and are text files with two lines. Should be easy to find the broken file. Regards, Simon > > Maybe strace command may help you find which files are corrupt: > su - cyrus -c "strace /usr/lib/cyrus-imapd/quota user" > > Anthony Prades > > > On 3/18/19 10:13 AM, Marco wrote: >> Il 18/03/2019 08:25, Thomas Cataldo ha scritto: >>> Hi, >>> >>> On cyrus 2.4.x we used to run >>> A1 RENAME mbox1 mbox2 our_partition >>> >>> When we run that on cyrus 3.0.8 we get : >>> A1 NO Cross-server or cross-partition move w/rename not supported >>> >>> Is there a way to do this rename with one command with cyrus3. >>> If we run A1 RENAME mbox1 mbox2 >>> the mailbox is renamed as expected but it moves from our_partition to >>> default which is not what we want. >>> >>> Any suggestion ? >> >> Hello, no suggestions, I verified too. :( But if you have a mailbox >> with: >> >> folder1 --> part1 >> folder2 --> part2 >> >> then you can do: >> >> A1 RENAME folder1 folder2/folder1 >> >> and folder1 is now on part2. So, under some circumstances, a cross >> partition rename seems to be still supported... >> >> Bye >> Marco >> >> Ps: on the above example unixhierarchysep: 1 >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Mailbox hierarchy determination?
> On 3/14/19 9:46 AM, Simon Matter wrote: >> I guess you're missing the fact that these options have possibly changed >> between releases. Another thing is that distribution packages can also >> alter the defaults and if they don't do it correct, they may "forget" to >> also change the docs accordingly. >> > > Thank you for your help with this. I think what you're saying is that > for my configuration I should have > >fulldirhash: 1 >hashimapspool: 1 > > > I'm not 100% certain how the defaults are set for my installation -- is > there any way to check this? Our RPMs create two files at build time: /usr/share/cyrus-imapd/rpm/imapd.conf.dist This is what the source distribution uses. /usr/share/cyrus-imapd/rpm/imapd.conf.default This is what the patched build uses. Unfortunately I don't know of a way to get the same info at run time. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Mailbox hierarchy determination?
> On 3/14/19 3:50 AM, Simon Matter wrote: >> >> Dirhashing is controlled by "fulldirhash" and "hashimapspool". >> > > > Right. This is what it says in imapd.conf: > > > fulldirhash: 0 >If enabled, uses an improved directory hashing scheme which hashes on > the entire username instead of using just the first letter as the hash. > This changes hash algorithm used for quota and user directories and if > hashimapspool is enabled, the entire mail spool. > > Note that this option CANNOT be changed on a live system. The server > must be quiesced and then the directories moved with the rehash utility. > > hashimapspool: 0 > If enabled, the partitions will also be hashed, in addition to the > hashing done on configuration directories. This is recommended if one > partition has a very bushy mailbox tree. > > > I don't have these options enabled, and my interpretation of this > description is that enabling them would facilitate my configuration, not > the other way around. This is what confused me in the first place. > > What am I missing? I guess you're missing the fact that these options have possibly changed between releases. Another thing is that distribution packages can also alter the defaults and if they don't do it correct, they may "forget" to also change the docs accordingly. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Mailbox hierarchy determination?
> The first couple of times I set up cyrus-imapd I used Debian packages
> which separated mailboxes in the default partition into folders based on
> the starting letter of the mailbox; i.e. something like:
>
>/var/spool/cyrus/mail/{[A-Z,a-z,0-9]}/user
>
> Since I have fewer than 50 users and since I found myself frequently
> "going to the files" so to speak to definitely determine the presence or
> absence of some message a user was concerned about, this hierarchy was
> something of a annoyance, so during the last major upgrade I switched
> this to:
>
> /srv/imap/{user1,user2,...,userN}
>
> i.e. flattening the folder structure and moving the default partition to
> a large non-system disk unaffected by /var volatility.
>
> Now for the life of me I can't figure out what setting in imapd.conf (if
> any) facilitated this structure (not the location of the default
> partition, but rather not using the [A-Z,a-z,0-9] subfolders.
>
> Last time I left the sieve folders divided by letter; in the next
> iteration I'd like to flatten that folder structure to.
>
> Looking at the Arch package post_install script, it pre-creates a ton of
> folders:
>
> --
> # all kinds of directories needed for the IMAP spool
> for subdir in imap/{,db,log,msg,proc,socket,sieve}
> spool/imap/{,stage.}; do
> if [ ! -d /var/${subdir} ]; then
> mkdir -m 0750 -p "/var/${subdir}"
> fi
> done
> for subdir in imap/{user,quota,sieve} spool/imap; do
> for i in a b c d e f g h i j k l m n o p q r s t v u w x y z \
> A B C D E F G H I J K L M N O P Q R S T U V W X Y Z \
> 0 1 2 3 4 5 6 7 8 9; do
> if [ ! -d /var/${subdir}/${i} ]; then
> mkdir -m 0750 -p "/var/${subdir}/${i}"
> fi
> done
> --
>
> I'm wondering how much or if any of this is really necessary. Will cyrus
> autocreate the necessary folders if missing?
Dirhashing is controlled by "fulldirhash" and "hashimapspool".
IIRC creating the dirs like above is not needed, they are created on demand.
Regards,
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus-imapd 2.4.17: processes stick on mailbox locking resulting in total mailsystem failure
> Hi Ellie > > Thanks a lot, I will try to build and test 2.4.20 Maybe try this: http://www.invoca.ch/pub/packages/cyrus-imapd/RPMS/ils-7/SRPMS/cyrus-imapd-2.4.20-2.el7.src.rpm Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: sieve runtime error
> Thanks, this got me looking into sendmail a little closer. I've never used
> the program and didn't realize a dummy sendmail binary was installed on my
> system. Actually installed sendmail and it works now, messages are being
> filtered.
You don't have to install sendmail, also postfix has a compatible sendmail
binary. Only the sendmail config in imapd.conf may not point to it.
Regards,
Simon
>
> On Tue, Dec 18, 2018 at 4:32 AM Simon Matter
> wrote:
>
>> > I'm trying to setup sieve and getting the following error in my logs:
>> >
>> > Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
>> > jschaef...@harmonywave.net id
>> > :
>> > Reject: Sendmail process terminated normally, exit status 255
>> >
>> >
>> > I'm following the documentation here:
>> >
>> https://www.cyrusimap.org/imap/reference/admin/sieve.html?highlight=sieve#testing-the-sieve-server
>> >
>> > I'm trying to get sieve working on my IMAP server. Using Ubuntu 16.04
>> with
>> > the cyrus-imapd 2.4.18-3 package. I'm using the test sieve script
>> shown
>> in
>> > the documentation to reject everything from my personal email:
>> >
>> > require ["reject","fileinto"];
>> > if address :is :all "From" "jschaeffer0...@gmail.com"
>> > {
>> > reject "testing";
>> > }
>> >
>> > I then connect using sieveshell, upload the file, and activate it:
>> >
>> > root@bllmail01:~# sieveshell -u jschaef...@harmonywave.net -a
>> > jschaef...@harmonywave.net mail.harmonywave.cloud
>> > connecting to mail.harmonywave.cloud
>> > Please enter your password:
>> >> put /tmp/testing.sieve testing
>> >> activate testing
>> >> list
>> > testing <- active script
>> >> quit
>> >
>> > However when I send a test email from my personal account to the email
>> > I have setup on the IMAP server it always comes through and I get this
>> > in mail.log:
>> >
>> > Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
>> > jschaef...@harmonywave.net id
>> > :
>> > Reject: Sendmail process terminated normally, exit status 255
>>
>> I think sieve tries to send mail using the configured sendmail binary
>> and
>> that doesn't work for some reason. You may check the sendmail config in
>> your imapd.conf and also consult the mail logs to learn more.
>>
>> Regards,
>> Simon
>>
>>
>
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: sieve runtime error
> I'm trying to setup sieve and getting the following error in my logs:
>
> Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
> jschaef...@harmonywave.net id
> :
> Reject: Sendmail process terminated normally, exit status 255
>
>
> I'm following the documentation here:
> https://www.cyrusimap.org/imap/reference/admin/sieve.html?highlight=sieve#testing-the-sieve-server
>
> I'm trying to get sieve working on my IMAP server. Using Ubuntu 16.04 with
> the cyrus-imapd 2.4.18-3 package. I'm using the test sieve script shown in
> the documentation to reject everything from my personal email:
>
> require ["reject","fileinto"];
> if address :is :all "From" "jschaeffer0...@gmail.com"
> {
> reject "testing";
> }
>
> I then connect using sieveshell, upload the file, and activate it:
>
> root@bllmail01:~# sieveshell -u jschaef...@harmonywave.net -a
> jschaef...@harmonywave.net mail.harmonywave.cloud
> connecting to mail.harmonywave.cloud
> Please enter your password:
>> put /tmp/testing.sieve testing
>> activate testing
>> list
> testing <- active script
>> quit
>
> However when I send a test email from my personal account to the email
> I have setup on the IMAP server it always comes through and I get this
> in mail.log:
>
> Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
> jschaef...@harmonywave.net id
> :
> Reject: Sendmail process terminated normally, exit status 255
I think sieve tries to send mail using the configured sendmail binary and
that doesn't work for some reason. You may check the sendmail config in
your imapd.conf and also consult the mail logs to learn more.
Regards,
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: suddenly 'User unknown'?
> Dan > > I have, and have always had, an empty /var/log/imapd.log so I'm not > going to make progress until I fix that. > > In n /etc/rsyslog.conf > > # cyrus imapd > #local6.* /var/log/imapd.log - tried this > first. > local6.debug /var/log/imapd.log > auth.debug /var/log/auth.log > > and in /etc/imapd.conf > > syslog_prefix: cyrus > syslog_facility: LOCAL6 > > If I remove the file /etc/imapd.log then > > # /etc/init.d/rsyslog restart > > # logger local6.debug 'test log message' > > # cat /var/log/imapd.log > > Nov 29 15:06:42 dell2600-1 brad: test log message > > Obviously syslog is working local6. But still no messages from cyrus! > Therefor I'm now stuck with this secondary problem. > > I have followed the cyrus instructions as best I can, but no go. I say > again this has all worked for years, albeit with an always empty imapd.log > > There must be some missing cyrus syslog configuration. From README.RPM in the cyrus-imapd package: 12) Check your syslog configuration. This RPM uses the mail facility to log messages. On busy sites you may want to limit the mail facility to the info priority with something like 'mail.info/var/log/maillog' in /etc/syslog.conf. So, I guess your cyrus-imapd messages go to /var/log/maillog then, not? Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Thunderbird and "Login to server failed".
> Hi, > > I'm using cyrus-imapd (now 2.4.x) and Thunderbird. The problem is, when > I click fast on the e-mails, I often get the "Login to server foo.bar > failed" (when I click like one -email per 3 seconds everything is fine). > My password is okay, so I just press "Cancel". I live with this problem > for a really long time, and I can say the this is specific to the TB: it > doesn't happen in Outlook. I've read a lot about this, and, besides the > mailbox being improperly configured and failing authrization 100% of the > time (which is not my case), there's an urban legend that TB uses by > default 5 concurrent connections to the IMAP server (and this part is > true), while cyrus permits less. But the thing is, even with 1 connecton > configured in TB (in advanced settings), I still get this error when > clicking messages fast. No errors are logged on the cyrus side, just the > normal log stating that I was allowed to examine my mailbox. > > Is there any way to get rid of these boring messages ? > > Cyrus ML may not be the correct place to ask for this, but BT/Mozilla > community just gives me futile answers like "are you sure you type your > passwords right" and so on. Interesting, I've never seen this problem over the years. Are there firewall rules active on the cyrus-imapd host? Probably limiting connection rate? Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Migrate users
Only one issue that I am having with the new 2.4.17 install. I use unixhierarchysep: 1 and have a couple of users with a "." in their name, e.g. deb.tony. The cyrus folder on the partition for them is deb^tony, although they appear in cyradm etc as deb.tony. They auth OK to the system through Horde, which authenticates them to LDAP and IMAP. Log entry showing the Horde server logging them in: imap[28530]: login: emp06.simonandkate.lan [192.168.1.230] deb.tony PLAIN+TLS User logged in SESSIONID= Then I get a log entry with them as deb^tony: imap[28530]: USAGE deb^tony user: 0.006688 sys: 0.004995 But nslcd triggers errors every 10 to 15 minutes: Jul 26 13:11:53 emp07 nslcd[922]: [c5eb19] request denied by validnames option Jul 26 13:11:53 emp07 nslcd[922]: [fb6a0e] request denied by validnames option IMAP is the only thing I know on the system that uses "deb^tony", so I was wondering why I'm getting the errors? The user can logon ok (always through Horde's IMAP connection), use, send emails ok... I changed the validnames regex in nslcd to accept a ^ but I assume that's just hiding the problem, not fixing it. There is nothing in my LDAP logs that indicates any failure. Any thoughts? Simon Anyone got any ideas on this, or can I safely ignore with my 'hack/fix'? Thanks -- Simon Wilson M: 0400 12 11 16 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: 2.4.12 vs 3.0.2 different protocol behaviour
> I looked deeper into the problem. > Looks like the command: > . LIST "" "Other Users" > responds abnormally, until I run : > . LIST "" "Other Users/%" > that actually list the shared folders. > Here's a trace of the imap protocol: > * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE STARTTLS AUTH=PLAIN SASL-IR] > www Cyrus IMAP 3.0.2 server ready > . login gabriele.bul...@sonicle.com Nrdstg88! I hope the login credentials are only for your test here? Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Migrate users
I've now completed my cutover to the new server, and have answered some of my own questions Following on from my question on enabling squatter on my new install... I have upgraded from Cyrus 2.3.7 on a CentOS 5 server to 2.4.17. on a new CentOS 7 server. I've not yet migrated the users (only 6 x users). They are both VMs on the same host, and at migration point I can simply bring up the new server with the drive from the old Cyrus VM that has the Cyrus partition on it, so new Cyrus will be able to 'see' the mailboxes without having to do rsync or anything like that. I've tested the new Cyrus server and it all appears to be functioning - listening on correct ports / sockets, delivering mail etc. So my questions: 1. Is 2.4.17 compatible with the mailboxes transferred from the old Cyrus 2.3.7 server? I rsync'ed the entire partition structure across from the 2.3.7 server to the 2.4.17 server, along with the /var/lib/imap folder, and started cyrus-imapd - to see if it would work. The service started, and immediately started running through all of the mailboxes, updating indexes, e.g.: squatter[10495]: Index upgrade: user.simon.Saved Emails (9 -> 12) I was then able to log in to IMAP and everything was there, so I'm assuming we're all good. 2. Assuming it is? Once the new Cyrus can see the mailboxes, will a reconstruct be needed to have new Cyrus able to see the full mailbox structure? If so with what flags to rebuild out all sub-mailboxes? Will it retain 'seen' / replied flags and ACLs? I ran a basic reconstruct, but it did not appear to have needed it. All flags and ACLs appear to be fine. 3. Do I need to do anything with the contents of /var/lib/imap/ on the old server for retention on the new server? I rsync'ed it across and started the new server with the old mailbox databases, and it appears to be OK. 4. Will I need to rebuild quotas once new Cyrus can see the mailboxes? I had to rebuild a couple of the quotas that were appearing wrong. 5. What is the best way to migrate sieve scripts? These are NOT on the drive to be moved to the new server, so will need to be migrated manually from /var/lib/imap/sieve etc... As a test I did a manual copy to the new server of a sieve script, set permissions and soft links, and it appears to work - is that the best way? Sieve scripts came over with the /var/lib/imap folder, and apart from it now listening on 4190 instead of 2000 (which had me for a few minutes) all is not working fine with sieve. Thanks in anticipation of assistance :) Simon. -- Simon Wilson Only one issue that I am having with the new 2.4.17 install. I use unixhierarchysep: 1 and have a couple of users with a "." in their name, e.g. deb.tony. The cyrus folder on the partition for them is deb^tony, although they appear in cyradm etc as deb.tony. They auth OK to the system through Horde, which authenticates them to LDAP and IMAP. Log entry showing the Horde server logging them in: imap[28530]: login: emp06.simonandkate.lan [192.168.1.230] deb.tony PLAIN+TLS User logged in SESSIONID= Then I get a log entry with them as deb^tony: imap[28530]: USAGE deb^tony user: 0.006688 sys: 0.004995 But nslcd triggers errors every 10 to 15 minutes: Jul 26 13:11:53 emp07 nslcd[922]: [c5eb19] request denied by validnames option Jul 26 13:11:53 emp07 nslcd[922]: [fb6a0e] request denied by validnames option IMAP is the only thing I know on the system that uses "deb^tony", so I was wondering why I'm getting the errors? The user can logon ok (always through Horde's IMAP connection), use, send emails ok... I changed the validnames regex in nslcd to accept a ^ but I assume that's just hiding the problem, not fixing it. There is nothing in my LDAP logs that indicates any failure. Any thoughts? Simon -- Simon Wilson M: 0400 12 11 16 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Update from cyrus-imapd 2.5 to 3.0
> In my "dhill" mailbox, I no longer see my folders and if I create a new > folder in that mailbox, let's say "Sent", it'll create it this way: > > [root@zappa Sent]# ls -atlr > total 20 > drwx--. 3 cyrus mail 4096 Jun 20 04:44 .. > -rw---. 1 cyrus mail 166 Jun 20 04:44 cyrus.header > -rw---. 1 cyrus mail 336 Jun 20 04:45 cyrus.annotations > drwx--. 2 cyrus mail 4096 Jun 20 04:45 . > -rw---. 1 cyrus mail 128 Jun 27 10:57 cyrus.index > [root@zappa Sent]# pwd > /var/spool/imap/s/Sent > > Instead of > /var/spool/imap/d/user/dhill/Sent ... You should check the 'fulldirhash' and 'hashimapspool' settings in imapd.conf. They influence it. Regards, Simon > Dave > > > > On 2017-06-27 12:20 PM, Jason L Tibbitts III wrote: >>>>>>> "DH" == David Hill writes: >> DH> Also, if a folder is created, it is created in >> DH> /var/spool/imap/[first_letter_of_folder]/Junk instead of >> DH> /var/spool/imap/d/user/dhill/Junk ... >> >> In your bug you indicated that things were being stored under >> /var/spool/[first letter]/ in the previous version. Here you say they >> were being stored under /var/spool/imap/first letter]/. Perhaps that's >> why my responses to you in the bugzilla ticket are so confused. >> >> Maybe it would be better if you could provide more explicit information >> in some way. Like, a directory listing or something. >> >> - J< > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy
> On Wed, May 24, 2017, at 04:17 PM, Don Lewis wrote: >> This first version of the patch worked for me. A problem with >> >> the second version of the patch is that mlookup() gets called in >> >> a bunch of different places and the server variable would need >> >> to be initialized in all of them. In verify_user(), mlookup is called >> >> several times, so server would have to be set to NULL before >> >> each call. >> >> > > Oh, good catch, I didn't look for other occurrences. > > Attached is an updated version of my patch that initialises these > variables correctly. Thanks, I can confirm that the updated patch fixes the issue for me. Kind regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy
> On Sat, May 20, 2017, at 05:59 PM, Simon Matter wrote:
>> > This is the point where the wrong path is taken.
>> >
>> >
>> > r = mlookup(namebuf, &server, NULL, NULL);
>> > if (!r && server) {
>> > /* remote mailbox */
>> > proxy_adddest(&dlist, rcpt, n, server, authuser);
>> > status[n] = nosieve;
>> > }
>> >
>> > mlookup did return something that is not 0, and server has not been
>> set
>> > in mlookup. In 2.4.17 there has was "if (server) *server = NULL;"
>>
>> Hi Michael,
>>
>> Thanks a lot for pointing me into the right direction. Indeed attached
>> patch makes it work for me.
>>
>> I don't know if this is the proper fix for the issue. Hopefully someone
>> who knows better than me can comment.
>>
>> Regards,
>> Simon
>
> Hi,
>
> Thanks Simon and Michael for the sleuthing!
>
> I've attached what I think is a better patch -- it makes sure the server
> variable is correctly initialised to NULL, and mlookup() keeps the
> idiomatic implementation whereby it only touches the output variable if
> it succeeds.
>
> Are you able to confirm if this patch also fixes the issue for you? I
> don't have a 2.4 environment to test this on
Hi Ellie,
Thanks for looking into it.
Unfortunately with your new version of the patch I only get this:
lmtpunix[23046]: FATAL: Internal error: assertion failed: append.c: 368:
stage != NULL && stage->parts[0] != '\0'
Regards,
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy
>
> Quoting Simon Matter :
>
>> Hi,
>>
>> I'm resending this with new subject and one again attached the straces
>> of
>> a delivery with 2.4.18 and 2.4.19.
>>
>>>>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All
>> seems well with just some small changes to the build.
>>>>>
>>>>> However, I just can't get it to work: lmtpd segfaults on mail
>>>>> delivery.
>>>>>
>>>>> The logs show something like this:
>>>>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
>> 7f216292973b sp 7ffe5ebc1f58 error 4 in
>>>>> lmtpd[7f21628c1000+169000]
>>>>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to
>>>>> death
>>>>> by 11
>>>>>
>>>>> First I've rebuilt without some of the custom patches in the package
>>>>> like
>>>>> the autocreate patches. But without a change.
>>>>>
>>>>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and
>> once
>>>>> with 2.4.19 in the hope to find something. The traces are attached.
>>>>>
>>>>> I see that there were quite some change concerning lmtp, does anyone
>> of the developers have an idea where the problem in lmtpd changes
>> could be?
>>>>
>>>> It turns out that lmtpd wants to act as proxy, because only then it
>> creates the spoolfile in /tmp/. But why?
>>>> Regards,
>>>> Simon
>>>>
>>>
>>> Are you running a murder setup? Can you produce a coredump and brovide
>>> a
>> backtrace with symbols loaded?
>>
>> Hi,
>>
>> I did two things meanwhile:
>>
>> 1) Built a completely vanilla cyrus-imapd-2.4.19 without any patches to
>> make sure it's not anything wrong with additional patches. The crash was
>> the same.
>>
>> 2) Ran the normal build with an unstripped "lmtpd" binary and produced
>> core and backtrace. So below:
>>
>> Core was generated by `lmtpd'.
>> Program terminated with signal 11, Segmentation fault.
>> #0 strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 > out
>> of bounds>, len=1024) at xstrlcpy.c:65
>> 65 if ((dst[n] = src[n]) == '\0') break;
>> Missing separate debuginfos, use: debuginfo-install
>> cyrus-sasl-gssapi-2.1.23-15.el6_6.2.x86_64
>> cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
>> cyrus-sasl-md5-2.1.23-15.el6_6.2.x86_64
>> cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64 db4-4.7.25-22.el6.x86_64
>> glibc-2.12-1.209.el6_9.1.x86_64 keyutils-libs-1.4-5.el6.x86_64
>> krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-23.el6.x86_64
>> libselinux-2.0.94-7.el6.x86_64
>> nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
>> openssl-1.0.1e-57.el6.x86_64 pcre-7.8-7.el6.x86_64
>> tcp_wrappers-libs-7.6-58.el6.x86_64 zlib-1.2.3-29.el6.x86_64
>> (gdb) bt
>> #0 strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 > out
>> of bounds>, len=1024) at xstrlcpy.c:65
>> #1 0x7ffaa5150c2e in proxy_adddest (dlist=0x7ffc5d73c288,
>> rcpt=0x7ffaa64d26d0 "si...@imap.bi.corp.invoca.ch", rcpt_num=0,
>> server=0x5d73c6c7 , authas=0x0) at
>> proxy.c:81
>
> here the bt is already in the wrong path and server has not been set
> wich leads to the SIGSEV
>
>> #2 0x7ffaa514fbae in deliver (msgdata=0x7ffaa64d0950, authuser=0x0,
>> authstate=0x7ffc5d73c280) at lmtpd.c:749
>
> This is the point where the wrong path is taken.
>
>
> r = mlookup(namebuf, &server, NULL, NULL);
> if (!r && server) {
> /* remote mailbox */
> proxy_adddest(&dlist, rcpt, n, server, authuser);
> status[n] = nosieve;
> }
>
> mlookup did return something that is not 0, and server has not been set
> in mlookup. In 2.4.17 there has was "if (server) *server = NULL;"
Hi Michael,
Thanks a lot for pointing me into the right direction. Indeed attached
patch makes it work for me.
I don't know if this is the proper fix for the issue. Hopefully someone
who knows better than me can comment.
Regards,
Simon--- cyrus-imapd-2.4.19/imap/lmtpd.c.orig 2017-05-15 07:43:30.0 +0200
+++ cyrus-imapd-2.4.19/imap/lmtpd.c 2017-05-20 08:36:47.629758869 +0200
@@ -392,6 +392,8 @@
char *c;
struct mboxlist_entry mbentry;
+if (server) *server = NULL;
+
/* do a local lookup and kick the slave if necessary */
r = mboxlist_lookup(name, &mbentry, tid);
if (r == IMAP_MAILBOX_NONEXISTENT && config_mupdate_server) {
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy
Hi,
I'm resending this with new subject and one again attached the straces of
a delivery with 2.4.18 and 2.4.19.
>>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All
seems well with just some small changes to the build.
>>>
>>> However, I just can't get it to work: lmtpd segfaults on mail delivery.
>>>
>>> The logs show something like this:
>>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
7f216292973b sp 7ffe5ebc1f58 error 4 in
>>> lmtpd[7f21628c1000+169000]
>>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to death
>>> by 11
>>>
>>> First I've rebuilt without some of the custom patches in the package like
>>> the autocreate patches. But without a change.
>>>
>>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and
once
>>> with 2.4.19 in the hope to find something. The traces are attached.
>>>
>>> I see that there were quite some change concerning lmtp, does anyone
of the developers have an idea where the problem in lmtpd changes
could be?
>>
>> It turns out that lmtpd wants to act as proxy, because only then it
creates the spoolfile in /tmp/. But why?
>> Regards,
>> Simon
>>
>
> Are you running a murder setup? Can you produce a coredump and brovide a
backtrace with symbols loaded?
Hi,
I did two things meanwhile:
1) Built a completely vanilla cyrus-imapd-2.4.19 without any patches to
make sure it's not anything wrong with additional patches. The crash was
the same.
2) Ran the normal build with an unstripped "lmtpd" binary and produced
core and backtrace. So below:
Core was generated by `lmtpd'.
Program terminated with signal 11, Segmentation fault.
#0 strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
65 if ((dst[n] = src[n]) == '\0') break;
Missing separate debuginfos, use: debuginfo-install
cyrus-sasl-gssapi-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-md5-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64 db4-4.7.25-22.el6.x86_64
glibc-2.12-1.209.el6_9.1.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-23.el6.x86_64
libselinux-2.0.94-7.el6.x86_64 nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
openssl-1.0.1e-57.el6.x86_64 pcre-7.8-7.el6.x86_64
tcp_wrappers-libs-7.6-58.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0 strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
#1 0x7ffaa5150c2e in proxy_adddest (dlist=0x7ffc5d73c288,
rcpt=0x7ffaa64d26d0 "si...@imap.bi.corp.invoca.ch", rcpt_num=0,
server=0x5d73c6c7 , authas=0x0) at
proxy.c:81
#2 0x7ffaa514fbae in deliver (msgdata=0x7ffaa64d0950, authuser=0x0,
authstate=0x7ffc5d73c280) at lmtpd.c:749
#3 0x7ffaa5152f41 in lmtpmode (func=0x7ffaa54b0020, pin=, pout=0x7ffaa64cf850, fd=) at
lmtpengine.c:1325
#4 0x7ffaa514f4ec in service_main (argc=1, argv=0x7ffaa64c9010,
envp=) at lmtpd.c:282
#5 0x7ffaa514dd95 in main (argc=, argv=, envp=0x7ffc5d740718) at service.c:585
The imapd.conf used for the test was:
configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN
tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
sieve_maxscriptsize: 128
allowplaintext: 1
delete_mode: delayed
expunge_mode: delayed
flushseenstate: 1
lmtp_strict_rfc2821: 0
mboxname_lockpath: {configdirectory}/volatile/lock
proc_path: {configdirectory}/volatile/proc
statuscache_db_path: {configdirectory}/volatile/statuscache.db
tlscache_db_path: {configdirectory}/volatile/tls_sessions.db
Any help would be much appreaciated!
Regards,
SimonProcess 2830 attached
accept(4, 0, NULL) = 11
fcntl(10, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
alarm(0)= 0
getpeername(11, {sa_family=AF_LOCAL, NULL}, [2]) = 0
write(3, "\2\0\0\0\16\v\0\0", 8)= 8
dup2(11, 0) = 0
dup2(11, 1) = 1
dup2(11, 2) = 2
close(11) = 0
write(3, "\3\0\0\0\16\v\0\0", 8)= 8
getpeername(0, {sa_family=AF_LOCAL, NULL}, [2]) = 0
open("/var/lib/imap/log/postman/lmtpunix-2830", O_WRONLY|O_CREAT|O_APPEND,
0644) = -1 ENOENT (No such file or directory)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
write(1, "220 mh1.bi.corp.testmh.ch Cyrus "..., 83) = 83
select(1, [0], NULL, NULL, {360, 0})= 1 (in [0], left {359, 97})
Re: Cyrus IMAP 2.4.19 released
> Hi,
>
> Quoting Simon Matter :
>
>>>> The Cyrus team is proud to announce the immediate availability of a
>>>> new
>>>> version of Cyrus IMAP: 2.4.19.
>>>
>>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems
>>> well with just some small changes to the build.
>>>
>>> However, I just can't get it to work: lmtpd segfaults on mail delivery.
>>>
>>> The logs show something like this:
>>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
>>> 7f216292973b sp 7ffe5ebc1f58 error 4 in
>>> lmtpd[7f21628c1000+169000]
>>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to
>>> death
>>> by 11
>>>
>>> First I've rebuilt without some of the custom patches in the package
>>> like
>>> the autocreate patches. But without a change.
>>>
>>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and
>>> once
>>> with 2.4.19 in the hope to find something. The traces are attached.
>>>
>>> I see that there were quite some change concerning lmtp, does anyone of
>>> the developers have an idea where the problem in lmtpd changes could
>>> be?
>>
>> It turns out that lmtpd wants to act as proxy, because only then it
>> creates the spoolfile in /tmp/. But why?
>> Regards,
>> Simon
>>
>
> Are you running a murder setup? Can you produce a coredump and brovide
> a backtrace with symbols loaded?
Hi,
I did two things meanwhile:
1) Built a completely vanilla cyrus-imapd-2.4.19 without any patches to
make sure it's not anything wrong with additional patches. The crash was
the same.
2) Ran the normal build with an unstripped "lmtpd" binary and produced
core and backtrace. So below:
Core was generated by `lmtpd'.
Program terminated with signal 11, Segmentation fault.
#0 strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
65 if ((dst[n] = src[n]) == '\0') break;
Missing separate debuginfos, use: debuginfo-install
cyrus-sasl-gssapi-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-md5-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64 db4-4.7.25-22.el6.x86_64
glibc-2.12-1.209.el6_9.1.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-23.el6.x86_64
libselinux-2.0.94-7.el6.x86_64 nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
openssl-1.0.1e-57.el6.x86_64 pcre-7.8-7.el6.x86_64
tcp_wrappers-libs-7.6-58.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0 strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
#1 0x7ffaa5150c2e in proxy_adddest (dlist=0x7ffc5d73c288,
rcpt=0x7ffaa64d26d0 "si...@imap.bi.corp.invoca.ch", rcpt_num=0,
server=0x5d73c6c7 , authas=0x0) at
proxy.c:81
#2 0x7ffaa514fbae in deliver (msgdata=0x7ffaa64d0950, authuser=0x0,
authstate=0x7ffc5d73c280) at lmtpd.c:749
#3 0x7ffaa5152f41 in lmtpmode (func=0x7ffaa54b0020, pin=, pout=0x7ffaa64cf850, fd=) at
lmtpengine.c:1325
#4 0x7ffaa514f4ec in service_main (argc=1, argv=0x7ffaa64c9010,
envp=) at lmtpd.c:282
#5 0x7ffaa514dd95 in main (argc=, argv=, envp=0x7ffc5d740718) at service.c:585
The imapd.conf used for the test was:
configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN
tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
sieve_maxscriptsize: 128
allowplaintext: 1
delete_mode: delayed
expunge_mode: delayed
flushseenstate: 1
lmtp_strict_rfc2821: 0
mboxname_lockpath: {configdirectory}/volatile/lock
proc_path: {configdirectory}/volatile/proc
statuscache_db_path: {configdirectory}/volatile/statuscache.db
tlscache_db_path: {configdirectory}/volatile/tls_sessions.db
Any help would be much appreaciated!
Regards,
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus IMAP 2.4.19 released
> Hi, > > Quoting Simon Matter : > >>>> The Cyrus team is proud to announce the immediate availability of a >>>> new >>>> version of Cyrus IMAP: 2.4.19. >>> >>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems >>> well with just some small changes to the build. >>> >>> However, I just can't get it to work: lmtpd segfaults on mail delivery. >>> >>> The logs show something like this: >>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip >>> 7f216292973b sp 7ffe5ebc1f58 error 4 in >>> lmtpd[7f21628c1000+169000] >>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to >>> death >>> by 11 >>> >>> First I've rebuilt without some of the custom patches in the package >>> like >>> the autocreate patches. But without a change. >>> >>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and >>> once >>> with 2.4.19 in the hope to find something. The traces are attached. >>> >>> I see that there were quite some change concerning lmtp, does anyone of >>> the developers have an idea where the problem in lmtpd changes could >>> be? >> >> It turns out that lmtpd wants to act as proxy, because only then it >> creates the spoolfile in /tmp/. But why? >> Regards, >> Simon >> > > Are you running a murder setup? Can you produce a coredump and brovide > a backtrace with symbols loaded? No murder involved. Unfortunately I'm unable to produce a coredump right now. It will take some time for me to do so... Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus IMAP 2.4.19 released
>> The Cyrus team is proud to announce the immediate availability of a new >> version of Cyrus IMAP: 2.4.19. > > I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems > well with just some small changes to the build. > > However, I just can't get it to work: lmtpd segfaults on mail delivery. > > The logs show something like this: > May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip > 7f216292973b sp 7ffe5ebc1f58 error 4 in lmtpd[7f21628c1000+169000] > May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to death > by 11 > > First I've rebuilt without some of the custom patches in the package like > the autocreate patches. But without a change. > > I've then straced an lmtpd during mail delivery, once with 2.4.18 and once > with 2.4.19 in the hope to find something. The traces are attached. > > I see that there were quite some change concerning lmtp, does anyone of > the developers have an idea where the problem in lmtpd changes could be? It turns out that lmtpd wants to act as proxy, because only then it creates the spoolfile in /tmp/. But why? Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus IMAP 2.4.19 released
> The Cyrus team is proud to announce the immediate availability of a new > version of Cyrus IMAP: 2.4.19. I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems well with just some small changes to the build. However, I just can't get it to work: lmtpd segfaults on mail delivery. The logs show something like this: May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip 7f216292973b sp 7ffe5ebc1f58 error 4 in lmtpd[7f21628c1000+169000] May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to death by 11 First I've rebuilt without some of the custom patches in the package like the autocreate patches. But without a change. I've then straced an lmtpd during mail delivery, once with 2.4.18 and once with 2.4.19 in the hope to find something. The traces are attached. I see that there were quite some change concerning lmtp, does anyone of the developers have an idea where the problem in lmtpd changes could be? Thanks, Simon strace-2.4.18.txt.gz Description: application/gzip strace-2.4.19.txt.gz Description: application/gzip Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Squatter after upgrade
- Message from Sebastian Hagedorn - Date: Fri, 28 Apr 2017 15:45:00 +0200 From: Sebastian Hagedorn Subject: Re: Squatter after upgrade To: si...@simonandkate.net Cc: "info-cyrus lists.andrew.cmu.edu" --On 28. April 2017 um 23:13:46 +1000 Simon Wilson wrote: I notice there is no SQUATTER entry in the new one - is it required, or is the functionality no longer needed? Should I put a SQUATTER entry in the new server's cyrus.conf? If you want to use the SQUAT index, you need to run squatter. That doesn't have to happen from cyrus.conf, though. We have a cronjob for that purpose that is a bit more involved. -- .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:. - End message from Sebastian Hagedorn - Thanks Sebastian. What advantages / disadvantages to running squatter from cron vs. from cyrus.conf? What does your cronjob look like? Simon. -- Simon Wilson M: 0400 12 11 16 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Squatter after upgrade
- Message from Anton -
Date: Fri, 28 Apr 2017 21:22:04 +0700
From: Anton
Subject: Re: Squatter after upgrade
To: si...@simonandkate.net
Cc: "info-cyrus lists.andrew.cmu.edu"
Hi Simon!
Just put your squatter at the same place in EVENT section in new
server. Actually I recommend you don't use stock cyrus version in
centos 7 due to IDLE bugs. Jump to 2.5.10 from kolab repo.
Thanks. I've added squatter and it seems to be running OK.
I've not been able to find any information about IDLE bugs in the
CentOS 7 packaged Cyrus. The whole intent of running CentOS is for
package stability - it would need to be a major issue to get me to
break out of that and sideload a newer Cyrus. Do you have more info on
the IDLE bugs you mention?
28 апр. 2017 г., в 8:13 ПП, Simon Wilson
написал(а):
Hi list,
I have just upgraded from Cyrus 2.3.7 on a CentOS 5 server to
2.4.17. on a new CentOS 7 server.
On the old one events was:
EVENTS {
# this is required
checkpointcmd="ctl_cyrusdb -c" period=30
# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprune cmd="cyr_expire -E 3" at=0400
# this is only necessary if caching TLS sessions
tlsprune cmd="tls_prune" at=0400
# running squatter
squatter cmd="/usr/lib/cyrus-imapd/squatter -r user" period=1440
}
On the new one it's:
EVENTS {
# this is required
checkpointcmd="ctl_cyrusdb -c" period=30
# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprune cmd="cyr_expire -E 3" at=0400
# this is only necessary if caching TLS sessions
tlsprune cmd="tls_prune" at=0400
}
I notice there is no SQUATTER entry in the new one - is it
required, or is the functionality no longer needed? Should I put a
SQUATTER entry in the new server's cyrus.conf?
Thanks
Simon.
--
Simon Wilson
M: 0400 12 11 16
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
- End message from Anton -
--
Simon Wilson
M: 0400 12 11 16
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Migrate users
Following on from my question on enabling squatter on my new install... I have upgraded from Cyrus 2.3.7 on a CentOS 5 server to 2.4.17. on a new CentOS 7 server. I've not yet migrated the users (only 6 x users). They are both VMs on the same host, and at migration point I can simply bring up the new server with the drive from the old Cyrus VM that has the Cyrus partition on it, so new Cyrus will be able to 'see' the mailboxes without having to do rsync or anything like that. I've tested the new Cyrus server and it all appears to be functioning - listening on correct ports / sockets, delivering mail etc. So my questions: 1. Is 2.4.17 compatible with the mailboxes transferred from the old Cyrus 2.3.7 server? 2. Assuming it is? Once the new Cyrus can see the mailboxes, will a reconstruct be needed to have new Cyrus able to see the full mailbox structure? If so with what flags to rebuild out all sub-mailboxes? Will it retain 'seen' / replied flags and ACLs? 3. Do I need to do anything with the contents of /var/lib/imap/ on the old server for retention on the new server? 4. Will I need to rebuild quotas once new Cyrus can see the mailboxes? 5. What is the best way to migrate sieve scripts? These are NOT on the drive to be moved to the new server, so will need to be migrated manually from /var/lib/imap/sieve etc... As a test I did a manual copy to the new server of a sieve script, set permissions and soft links, and it appears to work - is that the best way? Thanks in anticipation of assistance :) Simon. -- Simon Wilson M: 0400 12 11 16 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Squatter after upgrade
Hi list,
I have just upgraded from Cyrus 2.3.7 on a CentOS 5 server to 2.4.17.
on a new CentOS 7 server.
On the old one events was:
EVENTS {
# this is required
checkpointcmd="ctl_cyrusdb -c" period=30
# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprune cmd="cyr_expire -E 3" at=0400
# this is only necessary if caching TLS sessions
tlsprune cmd="tls_prune" at=0400
# running squatter
squatter cmd="/usr/lib/cyrus-imapd/squatter -r user" period=1440
}
On the new one it's:
EVENTS {
# this is required
checkpointcmd="ctl_cyrusdb -c" period=30
# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprune cmd="cyr_expire -E 3" at=0400
# this is only necessary if caching TLS sessions
tlsprune cmd="tls_prune" at=0400
}
I notice there is no SQUATTER entry in the new one - is it required,
or is the functionality no longer needed? Should I put a SQUATTER
entry in the new server's cyrus.conf?
Thanks
Simon.
--
Simon Wilson
M: 0400 12 11 16
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyr_expire suddenly segfaulting
> So everything is 2.4 defaults, which are:
>
> brong@wot:~/src/cyrus-imapd$ grep '_db"' lib/imapoptions
> { "annotation_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash",
> "skiplist")}
> { "duplicate_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync",
> "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql")}
> { "mboxkey_db", "skiplist", STRINGLIST("berkeley", "skiplist") }
> { "mboxlist_db", "skiplist", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist")}
> { "ptscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash",
> "skiplist")}
> { "quota_db", "quotalegacy", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist", "sql", "quotalegacy")}
> { "seenstate_db", "skiplist", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist")}
> { "subscription_db", "flat", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist")}
> { "statuscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync",
> "berkeley-hash", "berkeley-hash-nosync", "skiplist") }
> { "tlscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync",
> "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql")}
> { "userdeny_db", "flat", STRINGLIST("flat", "berkeley", "berkeley-hash",
> "skiplist", "sql")}
>
> So you shouldn't have any berkeley databases at all. It should be
> perfectly safe to shut down Cyrus, delete the $confdir/db folder, and
> restart Cyrus.
I suggest to make sure first that the ClearOS build really uses those
defaults - because they could change defaults in their builds.
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: What happened to normalizeuid?
> Hi, > > I considered opening a Github issue, but the question seems too trivial > for > that. > > I'm playing around with cyrus-imapd-3.0.0-rc1. When I ran "cyr_info > conf-lint" on the conf files from our 2.4.x production server, I got a few > items I knew how to deal with, but also this: > > normalizeuid: 1 > tls_require_cert: off > > tls_require_cert was off by default anyway, and all the tls options were > reworked a lot, so I figured that one could just be dropped. > > But in 2.4 normalizeuid 1 was off by default. I couldn't find the string > normalizeuid anywhere in the source or the release notes. I'd say that's > bad. If you get rid of an option, that should be documented. What is the > new default? Hi, We and others had this as a patch in our RPMs but I think it has never been part of vanilla cyrus-imapd. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: imap impersonate
> Hi, > is there any mechanism with Cyrus imap to impersonate another user? > I've seen other imap servers scenarios where one may use plain > authentication and sending user as mailboxuser plus a separator plus > adminuser and use only adminpassword, to get access to the mailboxuser as > is (dovecot, exchange). > Anything like this in Cyrus? > Gabriele Hi Gabriele, Check the "proxyservers" directive in imapd.conf. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Could not connect to socket /var/imap/socket/lmtp: Connection refused by localhost
> Hi. > > On 17.01.2017 19:09, Andy Dorman via Info-cyrus wrote: >> >> I am not an expert by any means and I hope someone corrects me if I >> make a bad suggestion...but I have two questions: >> >> 1. It sounds like you have a heavily used server, so why do you have >> Cyrus listening on both "localhost:lmtp" AND a unix socket >> "/var/imap/socket/lmtp"? >> >> From the log entry it looks like your MTA uses a unix socket. Unless >> you have something else (mail clients or other MTAs running on your >> Cyrus server?) that need to communicate via the localhost:lmtp port, >> you could comment out the unneeded lmtp service line and save those >> resources. > Well, on one hand you are right, seems like noone uses network lmtp > connections, but on the other hand how can the idle processes save > resources ? They only can save the memory, which doesn't seem to be the > problem. However, I will try you advice. >> >> 2. You say "increasing this value can make the situation even worse". >> Which value? There are 5 values on those two lines that you could >> increase. And by "even worse" do you mean even more refused >> connections? > The maxchild number. >> >> While I am not a Cyrus guru, I have seen my share of overloaded mail >> servers and if you are running into a disk IO limit, adding more >> processes fighting over a limited resource is very likely to make >> things worse. So you should also confirm a hardware limitation is not >> at play here. > Yup, this is exaclty what happens when increasing the maxchild number: > more messages start to bounce. And yes, the disks iops seems to be the > limiting factor. So, are there any other approaches besides scaling out > the disks iops ? I remember a situation more than a decade ago where we had to tune this. The problem with LMTP deliveries was that a lot of mails with hundreds of recipients on the same server were sent. Our Postfix MTA has sent every mail by a single LMTP transfer which resulted in high LMTP load and prevented effective usage of single instance store. The problem was solved by setting "local_destination_recipient_limit = X00" on the Postfix MTA (I don't remember the exact number). Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: FreeBSD 9.3 STABLE, cyrus-imapd 2.5.9, Listen queue overflow
Hi,
I'm not an expert in this but just a wild guess: Could it be that you have
an issue with /dev/random or /dev/urandom?
Maybe others on this list can tell more about it and you can search the
list archives to find some information on the topic.
Regards,
Simon
> Hello.
>
> I have strange problem with cyrus-imapd 2.5.9, after upgrade my server
> OS from FreeBSD 7 to FreeBSD 9.3 STABLE and cyrus-imapd from 2.3.x to
> 2.5.9. All packages reinstalled after upgrade OS.
> A problem that, after a 2 or 4 or 7 days (differently) of normal work of
> service cyrus-imapd, he freeze. Users can't connect to mailboxes.
> In the dmesg log i see it - sonewconn: pcb 0xfe0101da7c40: Listen
> queue overflow: 49 already in queue awaiting acceptance (12 occurrences).
> And netstat -LAan show me it - fe0081a747a0 tcp4 49/0/32
> *.993.
> Why queue overflow, i'm not understand. I think that a problem in the
> process idled, because after stoped cyrus-imapd process
> ("/usr/local/etc/rc.d/imapd stop"), several processes of idled are
> visible in the list of processes (ps -ax).
> I run command "kiilall -9 idled" and "/usr/local/etc/rc.d/imapd start" -
> service normal work.
>
> Sorry about my english, my english is hard.
> Maybe someone help me.
>
> i have provided below detailed information about system and from logs
> (service cyrus-imapd freeze).
>
> System information:
> # uname -a
> FreeBSD mail.spectrum.ru 9.3-STABLE FreeBSD 9.3-STABLE #0: Sat Jul 2
> 17:48:04 MSK 2016
> old...@mail.spectrum.ru:/usr/obj/usr/src/sys/SRVKERNEL amd64
>
> # pkg info | grep cyrus
> cyrus-imapd25-2.5.9The cyrus mail server, supporting POP3
> and IMAP4 protocols
> cyrus-sasl-2.1.26_12 RFC SASL (Simple Authentication and
> Security Layer)
> cyrus-sasl-saslauthd-2.1.26_3 SASL authentication server for cyrus-sasl2
>
>
> Conf files
> # cat /usr/local/etc/cyrus.conf
> START {
> idled cmd="idled"
> recover cmd="ctl_cyrusdb -r"
> # syncclient cmd="sync_client -r"
> }
>
> SERVICES {
> imapcmd="imapd" listen="imap" prefork=0
> imaps cmd="imapd -s" listen="imaps" prefork=0
> pop3cmd="pop3d" listen="pop3" prefork=0
> pop3s cmd="pop3d -s" listen="spop3" prefork=0
> sieve cmd="timsieved" listen="sieve" prefork=0
> lmtpunixcmd="lmtpd" listen="/data/imap/socket/lmtp"
> prefork=0
> smmap cmd="smmapd" listen="/data/imap/socket/smmap"
> prefork=1
> # syncclient cmd="sync_client -r" listen="csync" prefork=1
> }
>
> EVENTS {
> checkpoint cmd="ctl_cyrusdb -c" period=30
> delprunecmd="cyr_expire -v -E 3" at=0400
> tlsprunecmd="tls_prune" at=0400
> squattercmd="squatter -i" at=0100
> }
>
> # cat /usr/local/etc/imapd.conf
> admins: cyrus
> allowanonymouslogin: no
> allowplaintext: yes
> altnamespace: yes
> annotation_db: skiplist
> configdirectory: /data/imap
> defaultpartition: default
> duplicate_db: berkeley-nosync
> imapidresponse: yes
> imapmagicplus: 0
> lmtp_downcase_rcpt: 1
> lmtpsocket: /data/imap/socket/lmtp
> mboxlist_db: skiplist
> munge8bit: 0
> partition-default: /data/spool/imap
> postmaster: postmaster
> ptscache_db: berkeley
> rfc2046_strict: 0
> sasl_auxprop_plugin: sasldb
> sasl_log_level: 1
> sasl_mech_list: plain cram-md5 digest-md5 login
> sasl_pwcheck_method: auxprop
> seenstate_db: skiplist
> sendmail: /usr/sbin/sendmail
> sievedir: /data/imap/sieve
> subscription_db: flat
> # sync_authname: cyrus
> # sync_batch_size: 0
> # sync_host: support.spectrum.ru
> # sync_log: 1
> # sync_password: pDkazwok
> # sync_repeat_interval: 5
> # sync_shutdown_file: /data/imap/socket/syncshutdown
> tls_server_ca_file: /etc/ssl/imapserver.pem
> tls_server_cert: /etc/ssl/imapserver.pem
> tls_server_key: /etc/ssl/imapserver.pem
> tls_session_timeout: 1440
> unixhierarchysep: no
> virtdomains: userid
>
>
> Information from logs:
> # netstat -LAan
> Current listen queue sizes (qlen/incqlen/maxqlen)
> TcpcbProto Listen Local Address
> fe01257a33d0 tcp4 0/0/32 *.4190
> fe0125e45b70 tcp4 0/0/32 *.995
> fe00a4a38b70 tcp4 0/0/32 *.110
> fe0081a747a0 tcp4 49/0/32*.993
> fe00a434e000 tcp4 0/0/32
Re: stock centos 7 based Cyrus-imapd not working - troubleshooting suggestions?
> Trying to troubleshoot a centos 7/Cyrus-imap configuration. The process
> is not running correctly - the /var/log/maillog fills up with messages and
> once this grep string is executed this is what is seen:
>
> grep -v Fatal maillog | grep -v abnormally | grep -v "ptions not presen"
>
> Jun 18 12:23:42 mail master[20269]: setrlimit: Unable to set file
> descriptors limit to -1: Operation not permitted
> Jun 18 12:23:42 mail master[20269]: retrying with 4096 (current max)
> Jun 18 12:23:42 mail ctl_cyrusdb[20277]: recovering cyrus databases
> Jun 18 12:23:42 mail ctl_cyrusdb[20277]: done recovering cyrus databases
> Jun 18 12:23:42 mail master[20269]: unable to setsocketopt(IP_TOS):
> Operation not supported
> Jun 18 12:23:42 mail ctl_cyrusdb[20314]: checkpointing cyrus databases
> Jun 18 12:23:42 mail ctl_cyrusdb[20314]: done checkpointing cyrus
> databases
AFAIK those messages are normal and don't show any real problem. What
exactly does not work?
Regards,
Simon
>
>
>
> The firewall is off (to attempt to see if the setsocketopt would go away):
> # systemctl status firewalld
> firewalld.service - firewalld - dynamic firewall daemon
> Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled;
> vendor preset: enabled)
> Active: inactive (dead)
>
> And SELinux is set to not stop things.
> # getenforce
> Permissive
>
>
>
> strace /usr/lib/cyrus-imapd/cyrus-master isn't really being helpful either
>
> (lots snipped)
> access("/etc/system-fips", F_OK)= -1 ENOENT (No such file or
> directory)
> open("/etc/imapd.conf", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=692, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f259e1b8000
> read(3, "configdirectory: /var/lib/imap\np"..., 4096) = 692
> read(3, "", 4096) = 0
> close(3)= 0
> munmap(0x7f259e1b8000, 4096)= 0
> uname({sys="Linux", node="mail.example.com", ...}) = 0
> close(0)= 0
> open("/dev/null", O_RDWR) = 0
> close(1)= 0
> open("/dev/null", O_RDWR) = 1
> close(2)= 0
> open("/dev/null", O_RDWR) = 2
> close(3)= -1 EBADF (Bad file descriptor)
> dup(0) = 3
> close(4)= -1 EBADF (Bad file descriptor)
> dup(0) = 4
> setrlimit(RLIMIT_NOFILE, {rlim_cur=RLIM64_INFINITY,
> rlim_max=RLIM64_INFINITY}) = -1 EPERM (Operation not permitted)
> open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 5
> fstat(5, {st_mode=S_IFREG|0644, st_size=3559, ...}) = 0
> fstat(5, {st_mode=S_IFREG|0644, st_size=3559, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f259e1b8000
> read(5, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\6\0\0\0\6\0\0\0\0"...,
> 4096) = 3559
> lseek(5, -2272, SEEK_CUR) = 1287
> read(5, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\7\0\0\0\7\0\0\0\0"...,
> 4096) = 2272
> close(5)= 0
> munmap(0x7f259e1b8000, 4096)= 0
> (lots more snipped)
>
> Nothing in the strace output for setsocketopt.
>
>
> Suggestions on what to do to figure out why on
> # cat /etc/redhat-release
> CentOS Linux release 7.2.1511 (Core)
>
> the installed packages
>
> # rpm -q -a | grep cyrus
> cyrus-imapd-2.4.17-8.el7_1.x86_64
> cyrus-sasl-devel-2.1.26-20.el7_2.x86_64
> cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
> cyrus-imapd-utils-2.4.17-8.el7_1.x86_64
> cyrus-imapd-devel-2.4.17-8.el7_1.x86_64
> cyrus-sasl-plain-2.1.26-20.el7_2.x86_64
> cyrus-sasl-2.1.26-20.el7_2.x86_64
> cyrus-sasl-scram-2.1.26-20.el7_2.x86_64
> cyrus-sasl-md5-2.1.26-20.el7_2.x86_64
>
> are not working.
>
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: lmtpd triggering a delivery.db checkpointing (Cyrus 2.3.16)
> Hi, > > > Several times a month our server freezes up on deliveries and the system > load average shoots up into the hundreds. Things quickly return to normal > between one and two minutes later but this has always puzzled me. > > Today I was watching the system from up close when it happened. > > > May 17 10:59:14 lmtp[24980]: skiplist: checkpointed > /ssd/cyrs/imap/deliver.db (223062 records, 25295200 bytes) in 119 seconds > > > > I took a quick dive into the code but could not find where and when lmtpd > is supposed to trigger a delivery.db checkpointing action. Isn't it controlled by 'checkpointcmd="ctl_cyrusdb -c" period=30' in cyrus.conf? Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cannot connect with cyradm
> That did it, but why did I have to specify PLAIN? None of the docs mention > having to do that with a default install. I don't know, but you don't have a default install but a configuration tailored by fedora. Simon > > > On Fri, May 6, 2016 at 2:16 AM, Simon Matter > wrote: > >> > I am trying to set up a basic system with cyrus-imap and postfix on >> amazon >> > linux >> > >> > I can connect using imtest, but cannot connect with cyradm: >> > >> > Additionally, when I connect via a client with a user I know has mail, >> > it's >> > saying that no INBOX exists. Postfix's mail spool is /var/spool/mail, >> but >> > I >> > don't see anywhere to verify that imapd or lmtpd is looking there. >> > >> > Sorry if these are dumb questions. I'm completely new to cyrus-imap >> and >> > the >> > documentation at cyrusimap.org is extremely sketchy. >> > >> > Any advice appreciated. >> > >> > $ more /etc/imapd.conf >> > configdirectory: /var/lib/imap >> > partition-default: /var/spool/imap >> > admins: cyrus >> > sievedir: /var/lib/imap/sieve >> > sendmail: /usr/sbin/sendmail >> > hashimapspool: true >> > sasl_pwcheck_method: saslauthd >> > sasl_mech_list: PLAIN LOGIN >> > tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem >> > tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem >> > tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt >> > >> > $ grep MECH /etc/init.d/saslauthd >> > MECH=pam >> > >> > $ imtest -t "" -u cyrus -a cyrus localhost >> > S: * OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS LOGINDISABLED >> > COMPRESS=DEFLATE] ip-172-31-1-214 Cyrus IMAP >> > v2.3.16-Fedora-RPM-2.3.16-6.9.amzn1 server ready >> > C: S01 STARTTLS >> > S: S01 OK Begin TLS negotiation now >> > verify error:num=18:self signed certificate >> > TLS connection established: TLSv1 with cipher DHE-RSA-AES256-SHA >> (256/256 >> > bits) >> > C: C01 CAPABILITY >> > S: * CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID AUTH=PLAIN AUTH=LOGIN >> SASL-IR >> > COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE >> UIDPLUS >> > NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ >> > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE >> CONDSTORE >> > SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH >> > S: C01 OK Completed >> > Please enter your password: >> > C: A01 AUTHENTICATE PLAIN Y3lydXMAY3lydXMAaGVsbC1oYXRoLW5vLWZ1cnk= >> > S: A01 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED >> > COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE >> UIDPLUS >> > NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ >> > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE >> CONDSTORE >> > SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH] Success (tls >> > protection) >> > Authenticated. >> > Security strength factor: 256 >> > . LIST "" "*" >> > . OK Completed (0.000 secs 1 calls) >> > . LOGOUT >> > * BYE LOGOUT received >> > . OK Completed >> > Connection closed. >> > >> > $ cyradm --user cyrus --authz cyrus localhost >> > Login disabled. >> > cyradm: cannot authenticate to server with as cyrus >> > $ cyradm --user cyrus --authz cyrus --auth pam localhost >> > verify error:num=18:self signed certificate >> > cyradm: cannot authenticate to server with pam as cyrus >> > $ cyradm --user cyrus --authz cyrus --auth shadow localhost >> > verify error:num=18:self signed certificate >> > cyradm: cannot authenticate to server with shadow as cyrus >> >> What does it do if you run with "--auch PLAIN" instead? >> >> Regards, >> Simon >> >> > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cannot connect with cyradm
> I am trying to set up a basic system with cyrus-imap and postfix on amazon > linux > > I can connect using imtest, but cannot connect with cyradm: > > Additionally, when I connect via a client with a user I know has mail, > it's > saying that no INBOX exists. Postfix's mail spool is /var/spool/mail, but > I > don't see anywhere to verify that imapd or lmtpd is looking there. > > Sorry if these are dumb questions. I'm completely new to cyrus-imap and > the > documentation at cyrusimap.org is extremely sketchy. > > Any advice appreciated. > > $ more /etc/imapd.conf > configdirectory: /var/lib/imap > partition-default: /var/spool/imap > admins: cyrus > sievedir: /var/lib/imap/sieve > sendmail: /usr/sbin/sendmail > hashimapspool: true > sasl_pwcheck_method: saslauthd > sasl_mech_list: PLAIN LOGIN > tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem > tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem > tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt > > $ grep MECH /etc/init.d/saslauthd > MECH=pam > > $ imtest -t "" -u cyrus -a cyrus localhost > S: * OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS LOGINDISABLED > COMPRESS=DEFLATE] ip-172-31-1-214 Cyrus IMAP > v2.3.16-Fedora-RPM-2.3.16-6.9.amzn1 server ready > C: S01 STARTTLS > S: S01 OK Begin TLS negotiation now > verify error:num=18:self signed certificate > TLS connection established: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 > bits) > C: C01 CAPABILITY > S: * CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID AUTH=PLAIN AUTH=LOGIN SASL-IR > COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS > NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE > SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH > S: C01 OK Completed > Please enter your password: > C: A01 AUTHENTICATE PLAIN Y3lydXMAY3lydXMAaGVsbC1oYXRoLW5vLWZ1cnk= > S: A01 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED > COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS > NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE > SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH] Success (tls > protection) > Authenticated. > Security strength factor: 256 > . LIST "" "*" > . OK Completed (0.000 secs 1 calls) > . LOGOUT > * BYE LOGOUT received > . OK Completed > Connection closed. > > $ cyradm --user cyrus --authz cyrus localhost > Login disabled. > cyradm: cannot authenticate to server with as cyrus > $ cyradm --user cyrus --authz cyrus --auth pam localhost > verify error:num=18:self signed certificate > cyradm: cannot authenticate to server with pam as cyrus > $ cyradm --user cyrus --authz cyrus --auth shadow localhost > verify error:num=18:self signed certificate > cyradm: cannot authenticate to server with shadow as cyrus What does it do if you run with "--auch PLAIN" instead? Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: Disappearing Mailbox Content
> > > --On September 9, 2015 at 13:49:40 -0500 "Robert T. Covell" > wrote: > >> That is the problem. I cannot reproduce (reliably or at all). It might >> be months before we hear about it. It has been happening for about two >> years. Always chalked it up to user error. But I can't say that it is >> or isn't. >> > > Check the ACL on the mailbox and make sure no user has permission to > delete > the mailbox-- no "x" right. Loss of all content including the cyrus* files > is mailbox deletion. (This is the voice of experience talking) > > Too bad you didn't implement delayed delete. You can just rename the > deleted folder to its original name. But, since he's using our Invoca RPM, delayed delete should be used by default. So yes, deleted folders should be there, are they? Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Migrate from 2.2.13 to 2.4.17 disasters
> Am Montag, 24. August 2015, 07:44:42 schrieb bs...@vsvinc.com: >> From what I have read in the documentation, you must have each user's >> password. > This is incorrect for cyrus. > >> Is there something I'm missing in the docs? > It seem so. > > You can use a single cyrus admin user instead too which have access to any > account content usually - imapsync provides this authentication scheme (by > an > option "auth-user" or so) too... And also check the proxyservers option in imapd.conf which can be used to "...to proxy for other users..." Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus IMAP 3.0.0-beta1 released
> The Cyrus team is proud to announce the immediate availability of the > first beta from the Cyrus IMAP 3.0 series: 3.0.0-beta1. Hi, As I'm currently working on a 2.5.x release of our Invoca RPMs I'm wondering if it would make sense to just go with 3.0 beta? The work on 2.5.x will still take some more time because it's a big change with the new build system, new functionality and support for new systems with systemd. Thanks for any insights which help me to make a decision. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: RPM for Cyrus-IMAPd 2.5.x?
> RHEL6 doesn't use systemd. Or at least ours doesn't. I know that. But our packages build on all supported RHEL versions from a single source RPM, that means since RHEL7 it has to support both, sysv and systemd. > >> -Original Message- >> From: Michael Gratton [mailto:m...@vee.net] >> Sent: Tuesday, May 05, 2015 12:58 AM >> To: Simon Matter >> Cc: Rosenbaum, Larry M.; 'info-cyrus@lists.andrew.cmu.edu' >> Subject: Re: RPM for Cyrus-IMAPd 2.5.x? >> >> >> I hear systemd also eats babies. >> >> Anyone have a PPA with 2.5 for Ubuntu 14.04 LTS (or later)? >> >> -- >> ⨠Michael Gratton, Percept Wrangler. >> â <http://mjog.vee.net/> >> >> >> On Tue, 5 May, 2015 at 2:46 PM, Simon Matter >> wrote: >> >> Where can I get an RPM for Cyrus IMAPd 2.5.1 (or even 2.5.0)? We >> are >> >> running RHEL6. >> > >> > Doing RPMs for current Cyrus IMAPd is on my RPM TODO list. >> > Unfortunately >> > this list is quite long (~30) at the moment. The reason is simple and >> > is >> > called systemd. >> > >> > Regards, >> > Simon >> > >> > >> > Cyrus Home Page: http://www.cyrusimap.org/ >> > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> > To Unsubscribe: >> > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> > > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: RPM for Cyrus-IMAPd 2.5.x?
> Where can I get an RPM for Cyrus IMAPd 2.5.1 (or even 2.5.0)? We are > running RHEL6. Doing RPMs for current Cyrus IMAPd is on my RPM TODO list. Unfortunately this list is quite long (~30) at the moment. The reason is simple and is called systemd. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Problem with quota
> On 04/04/2015 09:02 AM, Patrick Boutilier wrote: >>> Quoting Simon Matter , Sat, 04 Apr 2015: >>> >>>> I guess that's because of single instance store. It's not a bug then >>>> but a >>>> feature if "duplicatesuppression: 1". Duplicate messages are >>>> hardlinked on >>>> disk, they don't consume space there, but are still calculated in >>>> quota >>>> usage. >> >> Isn't that singleinstancestore:1 ? duplicatesuppression is where lmtpd >> will suppress delivery of a message to a mailbox if a message with the >> same message-id (or resent-message-id) is recorded as having already >> been delivered to the mailbox. > > Yes Patrick, you're correct. However there is a connection, in that > singleinstancestore requires the duplicate DB in order to do its work, > so people often conflate the two. Thanks Nic, that's correct. I shouldn't post before having a useful coffeine level. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Problem with quota
> /Hi, > we > - stoped cyrus-imapd > - reconstructed a mailbox (/usr/lib/cyrus-imapd/reconstruct -r -f > user."account") > ///- started cyrus-imapd/ > - deleted file /var/lib/imap/quota/"a"/user."account"// > - set quota (sq user."account" 3000) > -//recalculated the quota (/usr/lib/cyrus-imapd/quota -f > user.///"account")/ > > The output of "cyrus quota" is differente to real space of mailbox on > disk. > > > lq///user."account"/ > STORAGE 9789922/3000 (32.63307%) > > du -s /var/spool/imap/"a"/user/"account"/ > 7308012var/spool/imap/"a"/user/"account"// > > There are a difference//greater than 2.0 GB (9.8 GB - 7.3 GB) > > //Why? Hi, I guess that's because of single instance store. It's not a bug then but a feature if "duplicatesuppression: 1". Duplicate messages are hardlinked on disk, they don't consume space there, but are still calculated in quota usage. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: deleting emails directly
On Wed, Mar 04, 2015 at 11:13:25AM +0100, Sven Schwedas wrote: > > I'm running a virus scan over the spool directory and wonder how to get > > those messages removed within which a virus has been found. The easiest > > way would be to let the virus scanner do this, and the virus scanner > > doesn't use IMAP. > > Wouldn't it be safer to integrate the virus scanner in your MTA's > processing pipeline, /before/ delivering it to clients that might > automatically download the viruses before your file-based virus scanner > has a chance to delete it? Doing both would be best, I believe. We scan the spool, and it finds a few items that have made it into the virus definitions between delivery and the scan. Simon. -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: saslauthd question
> On 12/11/2014 12:45 PM, Andrew Morgan wrote: >> I only have PAM files for "imap", "lmtp", and "sieve" >> although I have other service names for some of them. >> > > I don't understand why you have PAM files for lmtp and sieve, but most > particularly lmtp. lmtpd is just a local daemon that transfers stuff > from your smtp server to cyrus. Are you running cyrus and smtpd on > different servers? If so, what does the PAM lmtp configuration look like? If you do lmtp over the network, you may want to authenticate who can deliver mails. Otherwise you can just use "lmtpd -a" if the environment is considered secure. > > I don't know anything about sieve, but thought the filters where all > internal, too; hence not in need of authentication. Sieve rules have to be managed per user, and therefore you need authentication. The server itself doesn't need to auth anything to run the filters. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Strange load issue with 2.4.17
> --On 13. Oktober 2014 17:35:25 +0200 Simon Matter > wrote: > >>> Hi, >>> >>> for the last week we have seen strange load issues on our Cyrus server. >>> All >>> of a sudden the load increases to several thousands, user CPU goes down >>> to basically zero, system CPU spikes. In the past we've had trouble >>> with >>> poor I/O performance, but that went along with an increase in Wait I/O. >>> We don't >>> see that now. vmstat shows a massive increase in context switches. When >>> the >>> system reaches this state, all we can do is restart Cyrus or reboot the >>> machine if that doesn't work anymore. >>> >>> I'm attaching a Ganglia screenshot that shows the problem clearly. When >>> the >>> problem exists, there's not much we can do to analyze it. A colleague >>> suggested that what we see could be related to this bug: >>> >>> https://bugzilla.cyrusimap.org/show_bug.cgi?id=3744 >>> >>> It was reported for 2.4.16, and it sounds as if it has been fixed, but >>> is >>> that fix really part of 2.4.17? Any other ideas? >> >> Is this a physical host or running virtualized? > > It's virtualized, but it's been that way for more than a year. Is this by any chance running on KVM, maybe on an AMD cpu? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Strange load issue with 2.4.17
> Hi, > > for the last week we have seen strange load issues on our Cyrus server. > All > of a sudden the load increases to several thousands, user CPU goes down to > basically zero, system CPU spikes. In the past we've had trouble with poor > I/O performance, but that went along with an increase in Wait I/O. We > don't > see that now. vmstat shows a massive increase in context switches. When > the > system reaches this state, all we can do is restart Cyrus or reboot the > machine if that doesn't work anymore. > > I'm attaching a Ganglia screenshot that shows the problem clearly. When > the > problem exists, there's not much we can do to analyze it. A colleague > suggested that what we see could be related to this bug: > > https://bugzilla.cyrusimap.org/show_bug.cgi?id=3744 > > It was reported for 2.4.16, and it sounds as if it has been fixed, but is > that fix really part of 2.4.17? Any other ideas? Is this a physical host or running virtualized? Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Weird CMD and ENVIRON
> Hi! > > Since sometime in 2.4, we've seen odd behaviour in the CMD and ENVIRON of > imapd > processes. Like this: > > > --klip-- > [jmmpelto@pcn1 ~]$ ps -ef|grep imapd|head -n 1 > cyrus 300 15885 0 13:30 ?00:00:00 imapd: > webmail-3.mappi.helsinki.fi [128.214.20.217] ajokela > [jmmpelto@pcn1 ~]$ sudo cat /proc/300/cmdline > imapd: webmail-3.mappi.helsinki.fi [128.214.20.217] ajokela > Hi, I see the same behaviour. Could it be that it has something to do with setproctitle()? If I "strings /proc/20488/environ" on preforked processes the ENV looks good until a process actually gets used. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cvt_cyrusdb_all
> Simon, while I think you're the best contact for my question, I'm
> including the list to be helpful to others that may have a similar
> question.
>
> We run multiple instances of Cyrus on a server and I noticed that your
> cvt_cyrusdb_all script cites mult-instance support several places in the
> change log. However, there's no man page for the script and it's not
> immediately obvious to me by looking at the code how to appropriately
> pass an instance name to the script as an argument. As is, this script
> attempts to operate on /var/lib/imap, which is empty in my case. Can you
> please explain how to pass an instance name to your script?
Hi, the script takes the instance name in the var ${INSTANCE}. You should
find how to use it in
/usr/share/doc/cyrus-imapd-2.4.17/README.multi-instance. Please note that
the script should only be used by the init scripts from the RPM package.
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Problem creating / finding mailboxes
>> CentOS 6.5, cyrus-imapd 2.3.16-6 >> >> So, I gather you're supposed to create mailboxes called user. >> But I wind up with user^, and mailbox can't be found. >> >> [joliver@localhost ~]$ cyradm -u testuser1 localhost >> Password: >> localhost> cm user.testuser2 >> localhost> cm user.testuser3 >> localhost> cm user.textuser4 >> >> [joliver@localhost ~]$ sudo ls -l /var/vmail/u/ >> total 12 >> drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^testuser2 >> drwx--. 2 cyrus mail 4096 Sep 15 14:51 user^testuser3 >> drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^textuser4 >> >> Sep 15 15:05:18 localhost lmtpunix[15704]: verify_user(user.testuser2) > > I guess you should send mail to "testuser2", not "user.testuser2"? Sorry, I think that was wrong. However, you should check your "unixhierarchysep", "hashimapspool" and "fulldirhash" settings. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Problem creating / finding mailboxes
> CentOS 6.5, cyrus-imapd 2.3.16-6 > > So, I gather you're supposed to create mailboxes called user. > But I wind up with user^, and mailbox can't be found. > > [joliver@localhost ~]$ cyradm -u testuser1 localhost > Password: > localhost> cm user.testuser2 > localhost> cm user.testuser3 > localhost> cm user.textuser4 > > [joliver@localhost ~]$ sudo ls -l /var/vmail/u/ > total 12 > drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^testuser2 > drwx--. 2 cyrus mail 4096 Sep 15 14:51 user^testuser3 > drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^textuser4 > > Sep 15 15:05:18 localhost lmtpunix[15704]: verify_user(user.testuser2) I guess you should send mail to "testuser2", not "user.testuser2"? Simon > failed: Mailbox does not exist > Sep 15 15:05:18 localhost master[15929]: about to exec > /usr/lib/cyrus-imapd/lmtpd > Sep 15 15:05:18 localhost lmtpunix[15929]: executed > Sep 15 15:05:18 localhost postfix/lmtp[15928]: 4B2E61ED5: > to=, > relay=localhost.localdomain[/var/lib/imap/socket/lmtp], delay=32, > delays=32/0.01/0/0.02, dsn=5.1.1, status=bounced (host > localhost.localdomain[/var/lib/imap/socket/lmtp] said: 550-Mailbox > unknown. Either there is no mailbox associated with this 550-name or > you do not have authorization to see it. 550 5.1.1 User unknown (in > reply to RCPT TO command)) > > > What did I do wrong? > > -- > *** > * John Oliver http://www.john-oliver.net/ * > * * > *** > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus-imapd can't find my mail
> I'm pretty certain this is because it just doesn't know where to look, > but I'm not sure how to tell it :-( No, it will not work that way because Cyrus uses a different message store format. You may search the list archives for more on this topic. Regards, Simon > > I was using http://www.linuxmail.info/postfix-dovecot-ldap-centos-5/ as > a guide, but needed to switch to cyrus-imapd I think cyrus doesn't know > how to look in /var/vmail/ as the vmail user. > > -- > *** > * John Oliver http://www.john-oliver.net/ * > * * > *** > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backup rsync
> Am Freitag, den 29.08.2014, 07:56 -0700 schrieb David R Bosso: >> --On August 29, 2014 at 4:27:57 PM +0200 Marcus Schopen >> wrote: >> >> > Hi, >> > >> > I'm planing to use lvm snaps and rsync for a daily disaster recovery >> > backup on my master cyrus (2.4.12 Ubuntu 12.04 LTS): >> >> Have you tested lvm snaps for this purpose yet? When I tried to use lvm >> snapshots in this way a few years ago, it absolutely killed performance >> on >> the volume with the active snapshots. Hopefully things have improved, >> but >> I wouldn't bet on it. > > Yes, I've tested it and don't see any performance problems while the > snaphot is active. My problem is how to backup about 140 GB mailspool in > an acceptable time slot. What tools are recommended? > > If using rsync what options are best practice? I suggest -aH to preserve single instance storage in the backup. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: problems with deleted messages and reconstruct
Then I guess "unexpunge -l" should show them. Simon > > I didnt apply any patch. This is a vanilla cyrus imap clean install. > > Shouldnt reconstruct -O take care of non indexed files? Should I assume > that, if it didnt delete them, they are still indexed? How can I list all > expunged/non-deleted mails? > > Thank you, > Rodrigo > > On 19/06/2014, at 11:51, Simon Matter wrote: > >> Depending on how you installed the defaults could have been changed by a >> patch. >> >> Simon >> >>> >>> No, it does not. I do not specify it in imapd.conf, so I assume the >>> default behavior (not delayed). >>> >>> Rodrigo >>> >>> On 19/06/2014, at 10:47, Simon Matter wrote: >>> >>>>> >>>>> Hello all, Im having the following issue with cyrus imapd 2.4.13: a >>>>> user >>>>> deletes a large amount of mails but many of those mails remain in the >>>>> mailbox folder. I double checked and imapd reports no mails prior to, >>>>> say, >>>>> 2008. However, there are mail files in the mailbox older than 2008. I >>>>> tried to "reconstruct -r user.xpto", as well as reconstruct -r -O >>>>> user.xpto, but this doesnt seem to solve the problem. Also, I >>>>> didnt >>>>> find a solution/explanation in neither FAQ nor googling. >>>>> >>>>> Any clues on how to diagnose this problem? >>>> >>>> Does your server use "expunge_mode: delayed"? >>>> >>>> Simon >>>> >>> >>> Rodrigo Ventura >>> Institute for Systems and Robotics >>> Instituto Superior Técnico >>> Lisbon, Portugal >>> http://users.isr.ist.utl.pt/~yoda >>> >> >> > > Rodrigo Ventura > Institute for Systems and Robotics > Instituto Superior Técnico > Lisbon, Portugal > http://users.isr.ist.utl.pt/~yoda > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: problems with deleted messages and reconstruct
Depending on how you installed the defaults could have been changed by a patch. Simon > > No, it does not. I do not specify it in imapd.conf, so I assume the > default behavior (not delayed). > > Rodrigo > > On 19/06/2014, at 10:47, Simon Matter wrote: > >>> >>> Hello all, Im having the following issue with cyrus imapd 2.4.13: a >>> user >>> deletes a large amount of mails but many of those mails remain in the >>> mailbox folder. I double checked and imapd reports no mails prior to, >>> say, >>> 2008. However, there are mail files in the mailbox older than 2008. I >>> tried to "reconstruct -r user.xpto", as well as reconstruct -r -O >>> user.xpto, but this doesnt seem to solve the problem. Also, I didnt >>> find a solution/explanation in neither FAQ nor googling. >>> >>> Any clues on how to diagnose this problem? >> >> Does your server use "expunge_mode: delayed"? >> >> Simon >> > > Rodrigo Ventura > Institute for Systems and Robotics > Instituto Superior Técnico > Lisbon, Portugal > http://users.isr.ist.utl.pt/~yoda > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: problems with deleted messages and reconstruct
> > Hello all, Im having the following issue with cyrus imapd 2.4.13: a user > deletes a large amount of mails but many of those mails remain in the > mailbox folder. I double checked and imapd reports no mails prior to, say, > 2008. However, there are mail files in the mailbox older than 2008. I > tried to "reconstruct -r user.xpto", as well as reconstruct -r -O > user.xpto, but this doesnt seem to solve the problem. Also, I didnt > find a solution/explanation in neither FAQ nor googling. > > Any clues on how to diagnose this problem? Does your server use "expunge_mode: delayed"? Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus super user -- able to login in as any user with a superuser password
> hi all, > > i have a cyrus instance with an ldap backend -- works beautifully ... > > i was wondering if there is a way to be able to login as any user with a > `superuser` password without messing with the users `real` password. > > i am assuming this would have to involve some saslauthd magic ... Not really saslauthd magic, just check the "proxyservers" option in imapd.conf. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: undelete a single message, delete_mode + expunge_mode activated
On 10 Mar 2014, at 20:31, Marcus Schopen wrote:
>
> When deleting a single message from Inbox or a subfolder these message
> are not moved into a DELETED structure and keep staying at their
> originally place on filesystem. Those messages are definitely delete. I
> can't see them in my imap client anymore (tested with evolution and
> Thunderbird). How do I undelete these single messages?
Undeleting messages in a mailbox
The following examples assume you have an installation of cyrus where there are
binaries in /usr/lib/cyrus-imapd/ - if not, adjust path to suit.
List messages available to unexpunge:
# su cyrus -c "/usr/lib/cyrus-imapd/unexpunge -l user/si...@leaky.org"
Each message will give you something like the following:
UID: 11422
Size: 7786
Sent: Mon Mar 10 12:00:00 2014
Recv: Mon Mar 10 16:06:32 2014
Expg: Mon Mar 10 16:53:55 2014
From: john wade
To :
Cc :
Bcc :
Subj: {44}
re: sieveshell aborts with "expected rparen"
To unexpunge a single message:
# su cyrus -c "/usr/lib/cyrus-imapd/unexpunge -udv user/si...@leaky.org 11422"
restoring expunged messages in mailbox 'user/si...@leaky.org'
Unexpunged user/si...@leaky.org: 11422 => 11438
restored 1 expunged messages
To unexpunge all the messages and mark them as undeleted as well:
# su cyrus -c "/usr/lib/cyrus-imapd/unexpunge -adv user/si...@leaky.org"
NOTE: This isn't recursive
It will only restore the inbox. To find other folders run
# su cyrus -c "/usr/lib/cyrus-imapd/ctl_mboxlist -d" | grep leaky.org
leaky.org!user.simon0 default si...@leaky.org lrswipkxtecda
leaky.org!user.simon.Lists 0 default si...@leaky.org lrswipkxtecda
leaky.org!user.simon.Lists.cyrus0 default si...@leaky.org
lrswipkxtecda
leaky.org!user.simon.Deleted Messages 0 default si...@leaky.org
lrswipkxtecda
Run the unexpunge command for every folder that needs to have mail undeleted.
For folder names that have spaces ' ', the spaces need to be escaped with a
backslash
su cyrus -c "/usr/lib/cyrus-imapd/unexpunge -adv user/simon/Deleted\
messa...@leaky.org"
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Re[2]: Postfix with Cyrus Imap
> Ok, well thanks for the info. > There is only one problem with my Cyrus Imapd 2.4.16... I have not found > autocrate path for this version of Cyrus. Maybe the patches from here will work http://slackware.org.uk/slackbuilds.org/14.0/network/cyrus-imapd/patches/ Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: ctl_mboxlist dump/removal
> Hi,
>
> Quoting Simon Beale :
>
>> I'm trying to plan a promote/demote of a backend and a replica, to
>> effectively swap them over in 2.4.13 cluster.
>>
>> Given the hostnames are different, and we haven't got a magic dns record
>> to update, the list of mailboxes on the master needs to be changed. My
>> thinking of how to do this was to run this command on the master:
>>
>> ctl_mboxlist -d -x -p '${BACKEND}!default
>>
>> ... which will lose the current mailboxes from the master, then run
>>
>> ctl_mboxlist -m -a
>>
>> on the new machine to report and import the new destinations.
>>
>> However, running the dump command on the master gives:
>>
>> user.testmbox2 1 dc201kvmuser-12816!default testmbox2 lrswipkxtecda
>> fatal error: Internal error: assertion failed: cyrusdb_skiplist.c: 645:
>> db->lock_status == UNLOCKED
>>
>> It gives a similar response if I instead try running the same command
>> (after repairing the mailboxes list) on the backend.
>>
>> Am I doing something wrong with the use of ctl_mboxlist here?
>>
>> How do other people handle this switchover of backends/replicas for
>> disaster recovery purposes? In the absence of the '-x' options working,
>> I'm thinking it'll have to be a full master dump piped through sed and
>> reimported to replace references to the old server.
>>
>
> AFAIK "ctl_mboxlist -m -a" is sufficient. It will overide the old backend
Thanks, yes, in testing it certainly appears to do the right thing. I will
have to tidy up the mailboxes before running this in live, as I do appear
to have some mailbox inconsistencies between backend and replica.
For anyone following afterwards, one gotcha I feel worth pointing out is
that if you have a line in your cyrus.conf START section of:
mupdatepush cmd="ctl_mboxlist -m"
... make sure you change that temporarily on the replica to:
mupdatepush cmd="ctl_mboxlist -m -a"
... before you start cyrus, else you will likely immediately delete all
the mailboxes on the replica.
(Glad I found that one while still in test mode :)
I'm still curious as to why the "ctl_mboxlist -d -x" didn't work, but I do
at least have a way to do the migration now.
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
ctl_mboxlist dump/removal
I'm trying to plan a promote/demote of a backend and a replica, to
effectively swap them over in 2.4.13 cluster.
Given the hostnames are different, and we haven't got a magic dns record
to update, the list of mailboxes on the master needs to be changed. My
thinking of how to do this was to run this command on the master:
ctl_mboxlist -d -x -p '${BACKEND}!default
... which will lose the current mailboxes from the master, then run
ctl_mboxlist -m -a
on the new machine to report and import the new destinations.
However, running the dump command on the master gives:
user.testmbox2 1 dc201kvmuser-12816!default testmbox2 lrswipkxtecda
fatal error: Internal error: assertion failed: cyrusdb_skiplist.c: 645:
db->lock_status == UNLOCKED
It gives a similar response if I instead try running the same command
(after repairing the mailboxes list) on the backend.
Am I doing something wrong with the use of ctl_mboxlist here?
How do other people handle this switchover of backends/replicas for
disaster recovery purposes? In the absence of the '-x' options working,
I'm thinking it'll have to be a full master dump piped through sed and
reimported to replace references to the old server.
Thanks
Simon
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re:
Hi Stefan, On 24 Jul 2013, at 22:13, "Stefan Schlörholz" wrote: > Hello Simon, > >> Did you try running "reconstruct -r -f ..."? > > I did try to run "reconstruct -r user.paul". The -f switch is not > known/accepted by my cyradm. Use the binary not the cyradm command. Depending on the system, it could be somewhere like /usr/lib/cyrus-imapd/reconstruct It's in the same place as imapd, lmtpd, and cyrus-master live (at least on CentOS) so you might be able to locate it using something like this command: ps ax | grep cyrus-master 32268 ?Ss 0:07 /usr/lib/cyrus-imapd/cyrus-master -d The -f argument to reconstruct according to the man page: Examine the filesystem underneath mailbox, adding all directories with a cyrus.header found there as new mailboxes. Useful for restoring mailboxes from backups. Regards, Simon (a different one) -- Simon Amor si...@leaky.org Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: mailboxes.db discrepancies between mailbox and mupdate servers
On 9 Jul 2013, at 22:20, Shawn Winnington-Ball wrote:
>
>> You can force a backend to push all of its mailboxes to the mupdate master
>> by running "ctl_mboxlist -m" on the backend. If you're not 100% sure
>> whether you want to push every mailbox before you know what state things
>> are in, you can individually push mailboxes, again using mupdate protocol.
>> Log in to the backend and run
>>
>> $ mupdatetest your.mupdate.server.com.
>> 1 MUPDATEPUSH user.foo
>
> However, I tried running this command and got
>
> B01 MUPDATEPUSH user.foo
> B01 BAD "Unrecognized command"
>
> I can't find MUPDATEPUSH in RFC 3656 either.
>
According to my notes, MUPDATEPUSH is used through an admin user's IMAP
connection (via imtest) not through mupdatetest.
If the mailbox doesn't actually exist, you might need to use LOCALCREATE via
IMAP to the backend prior to MUPDATEPUSH.
LC1 LOCALCREATE user.foo
ACL0 SETACL user.foo foo lrswipkxtea
MP1 MUPDATEPUSH user.foo
I use virtual domains so the syntax of the SETACL command may be slightly off
('foo' is the authenticated user as far as I can remember).
If you've already told the mupdate server about the mailbox, you probably don't
need the last 2 commands, just the localcreate should be enough to let them
connect to the mailbox.
Simon
Disclaimer: I'm not a cyrus expert - I just break it a lot :)
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re:
Hi, Did you try running "reconstruct -r -f ..."? BTW, could you change your mailer so you don't send HTML only? Regards, Simon > Hello, > > > > I had a cyrus installation running that worked perfectly. After a > crash of a hard drive I had to reinstall the system. The hard drive > contained the cyrus installation. However, the cyrus > "partition-default" was on a different hard drive. > > > > I reinstalled cyrus with the "partition-default" as > before. The user mailbox can be seen in cyradmin [user.paul > (\HasChildren)]. However all subfolders exept user.paul.Drafts, > user.paul.Mülleimer, user.paul.Versendete_Nachrichten can NOT be > seen. The directories and the messages in the storage partion under > /var/spool/imap/user/paul/ are all there. > > > > With a squirrelmail I do see all messages in INBOX and I do see the > three subfolders. However, the subfolders are being shown to be empty. The > files in the corresponding directories exist. > > > > I did a reconstruct -r with the cyrus admin allready, however I still > would have the data as they were after the hard drive crash. > > > > Is there anybody who can help to get the old subfolders and messages > back? > > > > Best regards > > > > Stefan > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Squatter crash with statusdb
> Hi Andy, could you file a bug for this? Then it will not be forgotten...
Or, could you check this bug here
http://bugzilla.cyrusimap.org/show_bug.cgi?id=3757
The patch below was the fix, could you verify if it also fixes your issue?
Thanks,
Simon
>From 1661683d453ea444aae5832b4a2cb7fd54489672 Mon Sep 17 00:00:00 2001
From: Bron Gondwana
Date: Sun, 09 Dec 2012 19:42:17 +
Subject: Bug #3757 - don't segfault on mailbox close with no user
Broke squatter and possibly other users as well.
This is probably a 2.4 only fix - the code has changed
for statuscache a bit since.
---
diff --git a/imap/statuscache_db.c b/imap/statuscache_db.c
index fadc58d..41dfd2b 100644
--- a/imap/statuscache_db.c
+++ b/imap/statuscache_db.c
@@ -150,9 +150,11 @@ static char *statuscache_buildkey(const char
*mailboxname, const char *userid,
/* Build statuscache key */
len = strlcpy(key, mailboxname, sizeof(key));
+/* double % is a safe separator, it can't exist in a mailboxname */
key[len++] = '%';
key[len++] = '%';
-len += strlcpy(key + len, userid, sizeof(key) - len);
+if (userid)
+ len += strlcpy(key + len, userid, sizeof(key) - len);
*keylen = len;
@@ -410,11 +412,9 @@ int statuscache_invalidate(const char *mboxname,
struct statusdata *sdata)
drock.db = statuscachedb;
drock.tid = NULL;
-key = statuscache_buildkey(mboxname, "", &keylen);
+key = statuscache_buildkey(mboxname, /*userid*/NULL, &keylen);
-/* strip off the second NULL that buildkey added, so we match
- * the entires for all users */
-r = DB->foreach(drock.db, key, keylen - 1, NULL, delete_cb,
+r = DB->foreach(drock.db, key, keylen, NULL, delete_cb,
&drock, &drock.tid);
if (r != CYRUSDB_OK) {
syslog(LOG_ERR, "DBERROR: error invalidating: %s (%s)",
>
> Quoting Andy Fiddaman , Mon, 24 Jun 2013:
>
>> FWIW, this gets it working again:
>>
>> --- cyrus-imapd-2.4.17.dist/imap/statuscache_db.c 2013-06-24
>> 10:10:08.219203100 +
>> +++ cyrus-imapd-2.4.17/imap/statuscache_db.c2013-06-24
>> 10:10:20.537711377 +
>> @@ -152,7 +152,7 @@
>> len = strlcpy(key, mailboxname, sizeof(key));
>> key[len++] = '%';
>> key[len++] = '%';
>> -len += strlcpy(key + len, userid, sizeof(key) - len);
>> +len += strlcpy(key + len, userid ? userid : "cyrus", sizeof(key) -
>> len);
>>
>> *keylen = len;
>>
>>
>> On Mon, 24 Jun 2013, Andy Fiddaman wrote:
>>
>> ;
>> ; Hi,
>> ;
>> ; I've just upgraded my Cyrus installation to 2.4.17 and squatter is
>> ; crashing in statuscache_buildkey() because userid is NULL.
>> ;
>> ; I'm not sure what the best fix for this is. Should squatter even be
>> using
>> ; the statuscache or should it populate "cyrus" as the username when
>> ; initialising the index, or something else?
>> ;
>> ; Thanks,
>> ;
>> ; Andy
>> ;
>> ; Program received signal SIGSEGV, Segmentation fault.
>> ; [Switching to Thread 1 (LWP 1)]
>> ; 0xfd7ffe3dccb0 in .strlenalign16_loop () from /lib/64/libc.so.1
>> ; (gdb) where
>> ; #0 0xfd7ffe3dccb0 in .strlenalign16_loop () from
>> /lib/64/libc.so.1
>> ; #1 0xfd7ffe414149 in strlcpy () from /lib/64/libc.so.1
>> ; #2 0x004610ac in statuscache_buildkey (
>> ; mailboxname=0x5ab8b0 "example.net!user.silo", userid=0x0,
>> ; keylen=0xfd7fffdfe0cc) at statuscache_db.c:155
>> ; #3 0x0046169a in statuscache_update_txn (
>> ; mboxname=0x5ab8b0 "example.net!user.silo",
>> sdata=0xfd7fffdfe290,
>> ; tidptr=0xfd7fffdfe218) at statuscache_db.c:326
>> ; #4 0x004619ad in statuscache_invalidate (
>> ; mboxname=0x5ab8b0 "example.net!user.silo",
>> sdata=0xfd7fffdfe290)
>> ; at statuscache_db.c:425
>> ; #5 0x00434099 in mailbox_unlock_index (mailbox=0x5db998,
>> ; sdata=0xfd7fffdfe290) at mailbox.c:1637
>> ; #6 0x00422feb in index_unlock (state=0x5af6a0) at
>> index.c:1232
>> ; #7 0x00420c98 in index_open (name=0x5ac960
>> ; "example.net!user.silo",
>> ; init=0x0, stateptr=0xfd7fffdfec18) at index.c:246
>> ; #8 0x00420395 in index_me (name=0x5ac960
>> "example.net!user.silo",
>> ; matchlen=20, maycreate=0, rock=0xfd7fffdffcac) at
>> squatter.c:594
>> ; #9 0x004208f9 in main (argc=3, argv=0xfd7fffdffcf8)
>> ; at squatter.c:745
>> ;
>> ;
>> ; (gdb) frame 7
>> ;
Re: namesppace trouble
> Hello, > the cyrus namespace base is located in my default installation in > > /var/spool/imap/user/ > > I expected afterrunning the mkimap script to find the alphabetical > letters of the usernames in that directory: > > a > b > c > d > e > > and so on What's you 'hashimapspool' setting? Is it 0|false? Regards Simon > > and have every user mailbox created in the dorresponding alphabetical > subdirectory > > for example user foo would be in > > /var/spool/imap/user/f/foo > > but it is not like that > > when I Create user foo or user bar their mailbox are set up as > > /var/spool/imap/user/foo > /var/spool/imap/user/bar > > How can I change this behaviour and have the classical schema ? > > /var/spool/imap/user/a/ > /var/spool/imap/user/b/ > /var/spool/imap/user/c/ > /var/spool/imap/user/d/ > . > . > . > /var/spool/imap/user/z/ > > I am using cyrus-imapd-2.4.17 on FreeBSD 9.1 > > > thank you > > Rick > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Safe to delete lock files in /var/imap/lock ?
> I am moving my Cyrus server to new hardware using rsync. I noticed that > the /var/imap/lock has 2.7G worth of lock files in it. Can the files in > /var/imap/lock be deleted once Cyrus is shutdown? If so, I will not > bother rsyncing them to the new server at all. I'd say yes, you don't have to sync them. But, I'm wondering why you have 2.7G size in lock/? Since all the locks are 0 byte files I'm wondering where the size comes from? Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: small patch to disable openssl compression
> Hi - This patch disables openssl compression - not sure if this is a > security risk or not... but, I don't think I like the encryption library > performing compression anyway, it's complicated already. Maybe. Just FYI, with latest RHEL5+6 openssl now disables compression by default. I guess the end result for cyrus is the same like with your patch? Maybe other distributions do the same. Simon > > Chris > > > diff -rupN cyrus-imapd-2.4.17/imap/tls.c cyrus-imapd-2.4.17.f/imap/tls.c > --- cyrus-imapd-2.4.17/imap/tls.c 2012-12-01 19:57:54.0 + > +++ cyrus-imapd-2.4.17.f/imap/tls.c 2013-01-18 15:27:58.0 + > @@ -667,6 +667,11 @@ int tls_init_serverengine(const char > off |= SSL_OP_NO_SSLv3; > } > SSL_CTX_set_options(s_ctx, off); > + > +#ifdef SSL_OP_NO_COMPRESSION > +SSL_CTX_set_options(s_ctx, SSL_OP_NO_COMPRESSION); > +#endif > + > SSL_CTX_set_info_callback(s_ctx, (void (*)()) > apps_ssl_info_callback); > > /* Don't use an internal session cache */ > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Exchange to cyrus-imapd mail migration query
> On 02/21/13 15:59 +0530, jayesh shinde wrote: >>Hi all , >> >>I am migrating the Exchange 2003 server to Cyrus-imapd + postfix server >>Due to some reason , on Exchange server imap protocol is not open and >>only mapi protocol is available. >>Along with this end users are not allowing the share the passwords. >> >>In Exchange by giving the administrator access ( i.e ACL ) on >>particular mailbox , admin ID can see other account's emails. But >>its only opening on Mapi enable outlook. > > You might see if it's possible to enable IMAP on your exchange server. > IMAP > runs on port 993 on our's (imaps). > >>Is there any tool ( like imapsync ) by which I can connect to >>Exchange server over Mapi protocol >>and sync the emails in cyrus-impad's imap ( 143 ) enable server ? >> >>I want to know , how peoples are handing such situation and migrating >>the emails ? > > Try configuring an Outlook client simultaneously with both a MAPI > connection, and an IMAP connection, and then copy the messages between the > two. > > Another approach would be to export your messages, using Outlook, to a > .pst > file, and figure out a way to import those messages into cyrus, or back > into outlook after IMAP has been configured. If IMAP is enabled on the exchange side, then imapsync should also work. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: problems with quota files
> Hello I am running cyrus 2.4.17 > > I realized that quota shown for users with quota command is much less > than real filesystem usage > > an example: > > /usr/lib/cyrus-imapd/quota user.username > Quota % Used Used Root > 1500 34 5189501 user.username > > du -sh /var/spool/imap/m/user/username > 11G/var/spool/imap/m/user/username > > I am a bit worried about it. > > I am using ext4 > > the same problem was also on the previous cyrus version I was using > 2.3.16 because I have the same problems. > > I fixed the quota files with quota -f but it did not resolve my problem. > > any hints ? Maybe single instance store is causing it? Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: multiple statuscache db (only Invoca rpm?)
> Hello cyrus user,
>
> on my cyrus-imapd server I see this:
>
> man imapd.conf:
>
> statuscache: 0
> Enable/disable the imap status cache.
>
> statuscache_db: skiplist
> The cyrusdb backend to use for the imap status cache.
>
> Allowed values: berkeley, berkeley-nosync, berkeley-hash,
> berkeley-hash-nosync, skiplist
>
> statuscache_db_path: {configdirectory}/volatile/statuscache.db
> The absolute path to the statuscache db file.
>
>
> In my imapd.conf I set:
> statuscache: 1
>
> Every time I restart cyrus-imapd, a copy of
> /var/lib/imap/volatile/statuscache.db appears on /var/lib/imap path:
What kind of problem do you see?
The idea behind the volatile/ directory is that you can put it on tmpfs
for performance reason if you like and the files there are saved to
/var/lib/imap/ on shutdown and restored from there on startup.
And yes, that's Invoca rpm specific.
>
> #ls -l /var/lib/imap/
> total 68
> -rw--- 1 cyrus mail 852 Feb 8 09:15 annotations.db
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 backup
> drwx-- 2 cyrus mail 4096 Feb 8 08:52 db
> drwx-- 2 cyrus mail 4096 Feb 8 09:15 db.backup1
> drwx-- 2 cyrus mail 4096 Feb 8 09:10 db.backup2
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 log
> -rw--- 1 cyrus mail 144 Feb 8 09:15 mailboxes.db
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 meta
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 msg
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 ptclient
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 quota
> drwx-- 2 cyrus mail 4096 Feb 8 09:15 rpm
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 sieve
> drwxr-x--- 2 cyrus mail 4096 Feb 8 09:15 socket
> -rw--- 1 cyrus mail 144 Feb 8 09:10 statuscache.db
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 sync
> drwx-- 2 cyrus mail 4096 Jan 14 08:43 user
> drwx-T 2 cyrus mail 60 Feb 8 09:15 volatile
>
> I have to remove it by hand...
Why? Are there any problems?
Regards,
Simon
>
> I don't know if this happens only on Invoca rpm which I use:
>
> name : Cyrus IMAPD
> version: v2.4.17-Invoca-RPM-2.4.17-6.el6 d1df8aff 2012-12-01
> vendor : Project Cyrus
> support-url: http://www.cyrusimap.org
> os : Linux
> os-version : 2.6.32-279.el6.x86_64
> environment: Built w/Cyrus SASL 2.1.23
> Running w/Cyrus SASL 2.1.23
> Built w/OpenSSL 1.0.0-fips 29 Mar 2010
> Running w/OpenSSL 1.0.0-fips 29 Mar 2010
> Built w/zlib 1.2.3
> Running w/zlib 1.2.3
> CMU Sieve 2.4
> TCP Wrappers
> NET-SNMP
> mmap = shared
> lock = fcntl
> nonblock = fcntl
> idle = idled
>
> Regards
> Marco
>
>
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: How can this happen?
> On Fri, 25 Jan 2013 10:48:28 +0100 Simon Matter wrote: >> > On Thu, 24 Jan 2013 13:12:33 -0800 (PST) Andrew Morgan wrote: >> >> On Thu, 24 Jan 2013, Frank Elsner wrote: >> >> >> >> > >> >> > Hello, >> >> > >> >> > we have the strange situation with our murder environment that >> >> > >> >> > mailbackend has >> >> > >> >> > user.x.Sent2 default x lrswipkxtecda >> >> > >> >> > but the folder is non-existent in the filesystem. >> >> > >> >> > The mupdate server doesn't know this folder (not in mailboxes.db) >> >> > >> >> > How can this happen? >> >> >> >> Some bug in Cyrus? >> > >> > Don't know. >> > >> >> If you want to fix this, you can try creating the proper structure on >> >> the >> >> filesystem, run reconstruct to get Cyrus to sync back up with it, >> then >> >> delete the folder using cyradm or an IMAP client. >> > >> > I didn't ask for a fix. I ask for a possible reason. >> >> I suggest to at lest report the version of Cyrus used. > > Oh, shit. Shame on me. > 2.3.16 on RHEL. OK, the current version is 2.4.17. If it's a bug there's a good chance it has been fixed since 2.3.16 :) Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: How can this happen?
> On Thu, 24 Jan 2013 13:12:33 -0800 (PST) Andrew Morgan wrote: >> On Thu, 24 Jan 2013, Frank Elsner wrote: >> >> > >> > Hello, >> > >> > we have the strange situation with our murder environment that >> > >> > mailbackend has >> > >> > user.x.Sent2 default x lrswipkxtecda >> > >> > but the folder is non-existent in the filesystem. >> > >> > The mupdate server doesn't know this folder (not in mailboxes.db) >> > >> > How can this happen? >> >> Some bug in Cyrus? > > Don't know. > >> If you want to fix this, you can try creating the proper structure on >> the >> filesystem, run reconstruct to get Cyrus to sync back up with it, then >> delete the folder using cyradm or an IMAP client. > > I didn't ask for a fix. I ask for a possible reason. I suggest to at lest report the version of Cyrus used. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Load spikes when new email arrives
> >> In another email discussion on the Redhat mailing list, I've confirmed >> we have >> an issue with partition alignment. This is getting to be quite the mess >> out there. I saw one posting where it is speculated there are >> thousands of >> poorly set up disk partitions for their RAID stripe size. fdisk and >> OS installers were late getting updated for the new TB disks >> and SSD disks as well. Partition alignment might account >> for 5 to 30% of a performance hit. >> >> I've checked and my cyrus lmtpd process count >> never exceeds 11 under work load. >> await jumps up to 150-195 at worst. >> >> If I'm already at IO saturation, I can't see how a higher lmtpd limit >> would help. >> >> My goal is to keep the system load reasonable so it is responsive for >> mailbox access by the end users. Right now we get nagios alerts >> about 6 times a day for excessive load. If I can move the mail >> queue workload into a hill instead of a sharp peak on the cacti >> load graph, it would be good. There are minutes around the peaks >> where the queue is emptied and we have only 5 messages >> inbound per minute. >> >> In hind sight, I agree RAID 10 should have been implemented. >> At the time, four years ago, getting lots of space was the >> priority as space needs always grow. We've never seen load >> issues until this month, and it seems to coincide with a >> general increase of all email volume and traffic. Our primary >> MX is also getting hit more than normal. >> >> > > There are a couple suggestions I'd like to put forth. First, improper > partition alignment is generally masked by the controller cache. I > strongly encourage you to check that your RAID array is making use of > this cache by enabling the WriteBack caching option on this array, > especially if your PERC card has a BBU (I think this was optional on > perc 5). You can install the MegaCLI tool from LSI to verify this (can > also be checked from OpenManage or reboot into the controller BIOS). I strongly suggest to do that *ONLY* with proper BBU in place! > > MegaCLI Link: > http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=migr-5082327 > The relevant commands are as follows: > MegaCli -AdpBbuCmd -aALL > MegaCli -LDInfo -Lall -aALL > > Second, the PERC card does support RAID level migration, so if you want > to add a spindle or even change RAID levels, you can. This can be done > via either OpenManage (hit or miss) or the MegaCLI tool (daunting, but > there are cheat sheets). You could also add a separate array to act as a > dedicated mail spool. You can also replace the existing disks with > faster (and/or larger) disks for additional performance without ever > touching the software. > > > To directly answer your question of "If I can move the mail queue > workload into a hill instead of a sharp peak on the cacti load graph, it > would be good. ", then lowering the LMTP limit in cyrus (or the upstream > MX server) to turn the mail flow into a trickle, rather than a flood, > would do this. You can adjust the concurrency rate of LMTP deliveries in > postfix using lmtp_destination_concurrency_limit (default 20). The > cyrus method has already been mentioned. You may also look at other ways > to reduce IO wait, such as disk defragmentation or utilizing hard links > in cyrus (singleinstancestore: 1). Another thing is to check partitioning here. Using separate spindles for /var/lib/imap seems a good idea, RAID1 on two small but fast disks has always worked fine for me. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus-imap configuration question SOLVED
> Thanks All, > > Well yes, ahem.. obviously! Since I'm a complete beginner at this, who, > what > and how did the ownership of mailboxes.db get set wrongly in the first > place? > > The script /usr/lib/cyrus-imapd/mkimap was run as user cyrus, as per the > instructions. I think you should follow the instructions mentioned in the Fedora rpm docs and not the generic instructions of the cyrus docs. IMHO things like mkimap should not be needed with a packaged version of cyrus (but of course it should not break things). If it still fails after following the package docs you may report this to Fedora as the package could have a bug then. Regards, Simon > > Surely cyrus CANNOT create a root owned file ?? > > Observation: > > Just as a quick experiment I changed mailboxes.db permissions to 666 and > low > it works! But something knows and changed the ownership to cyrus:mail with > permissions 600 > > Something very fishy there. > > ~ o ~ > > While on the subject of incorrect permissions. The file > /var/lib/imap/db/skipstamp was also root:root and causing this from > systemctl > status cyrus-imapd after a SUCCESSFUL start. > > Jan 20 12:15:44 blanked.com ctl_cyrusdb[1652]: DBERROR: writing > /var/lib/imap/db/skipstamp: Permission denied > > I have know idea why these Permission denied messages are not in > /var/log/messages, perhaps it has something to do with SELinux !! > > Thanks for all the help, Charles Bradshaw > > On: Sun, 20 Jan 2013 07:20:18 -0400, Patrick wrote: >> On 01/20/2013 06:40 AM, Charles Bradshaw wrote: >> > Thanks Dan, Here are the details you asked for. >> > >> > My /etc/imapd.conf : >> > >> > configdirectory: /var/lib/imap >> > partition-default: /var/spool/imap >> > admins: cyrus >> > sievedir: /var/lib/imap/sieve >> > sendmail: /usr/sbin/sendmail >> > hashimapspool: true >> > sasl_pwcheck_method: auxprop >> > sasl_mech_list: PLAIN DIGEST-MD5 CRAM-MD5 >> > sasl_auxprop_plugin:sasldb >> > #allowplaintext: no >> > #defaultdomain: mail >> > tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem >> > tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem >> > tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt >> > # uncomment this if you're operating in a DSCP environment (RFC-4594) >> > # qosmarking: af13 >> > >> > I have played with allowplaintext and defaultdomain. >> > For the record the 3 sasl_... values are the same as the entries in >> > /etc/sasl2/Sendmail.conf >> > >> > Directory ownership: >> > >> > ls -ld /var/lib/imap >> > drwxr-x---. 18 cyrus mail 4096 Jan 19 19:37 lib/imap >> > >> > ls -l lib/imap >> > total 72 >> > drwx--. 2 cyrus mail 4096 Jan 20 03:35 backup >> > drwx--. 2 cyrus mail 4096 Jan 19 19:37 db >> > drwx--. 2 cyrus mail 4096 Jan 19 19:37 db.backup1 >> > drwx--. 2 cyrus mail 4096 Jan 19 16:45 db.backup2 >> > -rw---. 1 cyrus mail 144 Jan 19 12:20 deliver.db >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 log >> > -rw---. 1 root root 144 Jan 14 07:13 mailboxes.db >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 md5 >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 meta >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 msg >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 proc >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 ptclient >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 quota >> > drwx--. 2 cyrus mail 4096 Jan 19 15:29 rpm >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 sieve >> > drwxr-x---. 2 cyrus mail 4096 Jan 19 19:37 socket >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 sync >> > drwx--. 2 cyrus mail 4096 Mar 15 2012 user >> > >> > ls -ld /var/spool/imap >> > drwxr-x---. 4 cyrus mail 4096 Jan 19 11:39 /var/spool/imap >> > >> > ls -l /var/spool/imap >> > total 8 >> > drwxr-xr-x. 2 cyrus mail 4096 Jan 19 11:39 stage. >> > drwxr-xr-x. 2 cyrus mail 4096 Jan 19 11:39 sync. >> > >> > stage. and sync. are empty >> > >> > The only possibility I see above is mailboxes.db is root:root <<<< IS >> THIS THE >> > PROBLEM? >> >> mailboxes.db should be owned by cyrus user. >> >> > >> > I assume this was created during: >> > [root@dell2600 ~]# su cyrus >> > bash-4.2$ /usr/lib/cyrus-imapd/mkimap >> > >> > Charles Bradshaw >> > >> > >> > -- Origi
Re: Stripping of attachments using Horde 4/IMP 5.
>> - Message from awill...@whitemice.org - >> Date: Sun, 30 Dec 2012 18:56:36 -0500 >> From: Adam Tauno Williams >> Reply-To: awill...@whitemice.org >> Subject: Re: Stripping of attachments using Horde 4/IMP 5. >>To: info-cyrus@lists.andrew.cmu.edu >> >> >>> On Sun, 2012-12-30 at 12:23 +0530, an...@isac.gov.in wrote: >>>> Dear experts, >>>> I had extensive discussion on this issue in Horde/IMP mailing list. >>>> Later I reported the problem in RedHat Bugzilla. The details of the >>>> problem are also part of bugzilla. >>>> https://bugzilla.redhat.com/show_bug.cgi?id=885620 >>>> Though patch for cyrus-imapd was given to me for testing, I was >>>> suppose to reproduce the problem in another server. I am unable to >>>> reproduce the problem. Now, I have a feeling that, the problem may be >>>> due to /var/lib/imap residing in NetApp storage (though not NFS). >>>> I have a feeling, the problem may solve, if I make /var/lib/imap part >>>> of OS disk. Your opinion please... >>> >>> Are you using Berkley databases? If it does seg-fault what does the >>> backtrace look like? [I don't see a backtrace in the bug report, just a >>> core file; but the core isn't really useful unless one has the same >>> version of the software]. >>> >>> -- >>> Adam Tauno Williams GPG D95ED383 >>> Systems Administrator, Python Developer, LPI / NCLA >>> >>> >>> Cyrus Home Page: http://www.cyrusimap.org/ >>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>> To Unsubscribe: >>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>> >> >> The databases used are: >> >> -rw--- 1 cyrus mail 144 Dec 9 09:29 ./annotations.db >> ./annotations.db: Cyrus skiplist DB >> >> -rw--- 1 cyrus mail 144 Dec 30 11:30 ./db.backup1/annotations.db >> ./db.backup1/annotations.db: Cyrus skiplist DB >> >> -rw--- 1 cyrus mail 1882668 Dec 30 11:30 ./db.backup1/mailboxes.db >> ./db.backup1/mailboxes.db: Cyrus skiplist DB >> >> -rw--- 1 cyrus mail 144 Dec 30 11:00 ./db.backup2/annotations.db >> ./db.backup2/annotations.db: Cyrus skiplist DB >> >> -rw--- 1 cyrus mail 1882668 Dec 30 11:00 ./db.backup2/mailboxes.db >> ./db.backup2/mailboxes.db: Cyrus skiplist DB >> >> -rw--- 1 cyrus mail 18038784 Dec 30 11:30 ./deliver.db >> ./deliver.db: Berkeley DB (Btree, version 9, native byte-order) >> >> -rw--- 1 cyrus mail 1882668 Dec 29 14:33 ./mailboxes.db >> ./mailboxes.db: Cyrus skiplist DB >> >> >> I don't have backtrace? I am using RHEL rpms. > > Unfortunately we can also not see what has been changed in the patched > RPMs you got from RH. > >> >> But, is it OK to use NetApp Storage for /var/lib/imap file system? > > NetApp often sounds like NFS but you told us you are not using NFS but FC > attached disks? If so I don't know why it shouldn't work exactly as local > disks would. > > But when I hear Horde/IMP I remember a problem that some people hit after > upgrading Horde/IMP. I don't remember what it was but you should find it > in the archives. IIRC it has been fixed in the latest version of > cyrus-imapd. Sorry forgot to mention, if you make any tests on a test system make sure it's the same arch as the production system. I mean, don't test on a i386 box if production is running x86_64 or so. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Stripping of attachments using Horde 4/IMP 5.
> - Message from awill...@whitemice.org - > Date: Sun, 30 Dec 2012 18:56:36 -0500 > From: Adam Tauno Williams > Reply-To: awill...@whitemice.org > Subject: Re: Stripping of attachments using Horde 4/IMP 5. >To: info-cyrus@lists.andrew.cmu.edu > > >> On Sun, 2012-12-30 at 12:23 +0530, an...@isac.gov.in wrote: >>> Dear experts, >>> I had extensive discussion on this issue in Horde/IMP mailing list. >>> Later I reported the problem in RedHat Bugzilla. The details of the >>> problem are also part of bugzilla. >>> https://bugzilla.redhat.com/show_bug.cgi?id=885620 >>> Though patch for cyrus-imapd was given to me for testing, I was >>> suppose to reproduce the problem in another server. I am unable to >>> reproduce the problem. Now, I have a feeling that, the problem may be >>> due to /var/lib/imap residing in NetApp storage (though not NFS). >>> I have a feeling, the problem may solve, if I make /var/lib/imap part >>> of OS disk. Your opinion please... >> >> Are you using Berkley databases? If it does seg-fault what does the >> backtrace look like? [I don't see a backtrace in the bug report, just a >> core file; but the core isn't really useful unless one has the same >> version of the software]. >> >> -- >> Adam Tauno Williams GPG D95ED383 >> Systems Administrator, Python Developer, LPI / NCLA >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> > > The databases used are: > > -rw--- 1 cyrus mail 144 Dec 9 09:29 ./annotations.db > ./annotations.db: Cyrus skiplist DB > > -rw--- 1 cyrus mail 144 Dec 30 11:30 ./db.backup1/annotations.db > ./db.backup1/annotations.db: Cyrus skiplist DB > > -rw--- 1 cyrus mail 1882668 Dec 30 11:30 ./db.backup1/mailboxes.db > ./db.backup1/mailboxes.db: Cyrus skiplist DB > > -rw--- 1 cyrus mail 144 Dec 30 11:00 ./db.backup2/annotations.db > ./db.backup2/annotations.db: Cyrus skiplist DB > > -rw--- 1 cyrus mail 1882668 Dec 30 11:00 ./db.backup2/mailboxes.db > ./db.backup2/mailboxes.db: Cyrus skiplist DB > > -rw--- 1 cyrus mail 18038784 Dec 30 11:30 ./deliver.db > ./deliver.db: Berkeley DB (Btree, version 9, native byte-order) > > -rw--- 1 cyrus mail 1882668 Dec 29 14:33 ./mailboxes.db > ./mailboxes.db: Cyrus skiplist DB > > > I don't have backtrace? I am using RHEL rpms. Unfortunately we can also not see what has been changed in the patched RPMs you got from RH. > > But, is it OK to use NetApp Storage for /var/lib/imap file system? NetApp often sounds like NFS but you told us you are not using NFS but FC attached disks? If so I don't know why it shouldn't work exactly as local disks would. But when I hear Horde/IMP I remember a problem that some people hit after upgrading Horde/IMP. I don't remember what it was but you should find it in the archives. IIRC it has been fixed in the latest version of cyrus-imapd. Regards, Simon Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cannot delete mailbox in virtual domain
> hello here ! > > i have a problem with cyrus-imapd-2.4.17 and virtual domains: Thanks for letting us know that cyrus-imapd-2.4.17 exists :) Seems the announcement did not find its way to info-cyrus but I found this now http://asg.andrew.cmu.edu/archive/message.php?mailbox=archive.cyrus-announce&msg=211 Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Making sub-folder email reappear after recovery from backup
> On 11/2/2012 12:09 PM, Adam Tauno Williams wrote: >> On Fri, 2012-11-02 at 11:16 -0400, Gordon Marler wrote: >>> Can't find this mentioned in the docs or with any search I've done on >>> the mailing list, so here goes: >>> - Upgraded from 2.4.10 to 2.4.16 after losing my 2.4.10 system, but >>> having mail spool backed up >>> - After moving to the 2.4.16 system, able to see Inbox for accounts >>> - Had to make sub-folders reappear with command like (Thank you mailing >>> list for that tidbit!): >>> reconstruct -p default -rf user.gmarler. >>> - Now I can see the Inbox, Subfolders, Sub-subfolders, etc in >>> Thunderbird >>> BUT - Only Inbox has visible emails - All of the subfolders "appear" to >>> be empty, but there are definitely mails in them on the mail spool. >>> Looks like I forgot a step somewhere. Where have I gone wrong? >> reconstruct, with the -r [recursive] and -f [examine filesystem for >> mailboxes] options. > Ah, more involved than that - the crash on 2.4.10 seems to have left the > spool in an odd state, like it was completely reorganizing the spool and > got interrupted: > - $SPOOL is located in /opt/csw/var/cyrus/mail/ > - Original emails were left in $SPOOL/.u/g/user/gmarler/ > - A partial copy (subfolders all empty) was here: >$SPOOL/g/user/gmarler/ (I wasn't looking here) > - Another complete copy was here: >$SPOOL/user/gmarler/ (This is what I was looking at, but Cyrus wasn't) > > - Merged everything from $SPOOL/.u/g/user/gmarler/ and > $SPOOL/user/gmarler/ into $SPOOL/g/user/gmarler/ > - Ran reconstruct -p default -rf user.gmarler > - Started cyrus and started getting these errors: >DBERROR db4: PANIC: fatal region error detected; run recovery > - Killed off the master and cleaned up the DB4 *.db databases (except > for db/) > - Ran reconstruct -rf user.gmarler > - Restarted master > > Now everything from 1995 shows up. Thanks for that extra tip! I'm not sure but did you look at the "hashimapspool" and "fulldirhash" options (and defaults)? Can it be that the new server didn't have the same configs? Simon > > GM >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: auto create / subscribe folder patches
On 10/03/2012 01:32 PM, Andrew Morgan wrote: > If I remember right, the autocreate patches don't work in a Cyrus > Murder (cluster). Until the autocreate patches work with all the > supported ways of running Cyrus IMAP, I don't think they will be > included. > > I can understand your issue though. The bigger Cyrus sites already > use scripts to create new mailboxes when new users are created. The > really small sites probably don't mind creating mailboxes by hand. > The sites with more than 10 but fewer than 100 users probably are not > satisfied by either solution. > Understood. Hopefully something like that will be implemented that works with Murder. Perhaps the same code that cyradm uses to create mailboxes? We do have quite a few users. I'm pretty sure anyone managing users via LDAP would prefer to have mailboxes automatically created. I may look into an OpenLDAP overlay. Failing that, Dovecot. Anyway, thanks for explaining this. Cheers, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
auto create / subscribe folder patches
Hi all, I've been digging into the possibility of auto creation of INBOX and maybe other folders. All I find for Cyrus is patches. May I suggest putting these into the source tree? They're optional settings. So those who don't like them, don't need to use the feature. Not everyone complies their mail servers. I like to use packages for example. Yes, it will take a long time to make it downstream, but it's been discussed on this mailing list for over 10 years. I see this kind of reasoning: Date: Thu, 14 Nov 2002 11:16:41 -0800 (PST) From: Jonathan Marsden [...] The name of the option is potentially confusing, in that the desired mailbox is not 100% *automatically* created. Rather, it is only created when the new user logs in and issues a CREATE INBOX command. I initially thought the INBOX would be created at login. Right, it turns out because of the way CMU phased in Cyrus & phased out our legacy e-mail system (AMS) this was the desired behavior. Users ran a program, "convertmail", that created their inbox and uploaded their mail---and until they ran that, we didn't want INBOXs to be created. This probably isn't the desired behavior at almost any other site. Larry We are using IMAP more and more for various solutions all around the world. Cyrus is a very good IMAP server. This limitation is quite a pain. Here is the use case: You have set up a nice LDAP interface for some manager(s) / admin(s) to set up user accounts. In your company all employees have email addresses. So naturally you want each user account to have a corresponding email address. With Cyrus, you'd have to either use a patch, or some cron job that looks for new users and logs into the IMAP server and creates their INBOX. This applies to SQL or other forms of user management as well. I'll probably go with dovecot for now, but it would really be nice to have something like the dovecot plugin (http://wiki2.dovecot.org/Plugins/Autocreate). It seems like we have a bunch of patches floating around. Perhaps I just don't understand what the problem with including them in the official release is. Thanks, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Another 2.4 upgrade horror story
> On 2012-09-25 19:05, Simon Beale wrote: >> The only gotcha I experienced was I forgot that cyrus was configured to >> hardlink mail, which of course was no longer the case after each mailbox >> was migrated, so my disk usage exploded. (But easily fixed/restored once >> identified). > > What did you use for restoring the hardlinks? freedup as well? > > I'm asking because I found a bug in freedup causing dataloss. I already > sent a patch fixing it to the author of freedup last november, but he > didn't release a new version yet. > > In case cyr_expire is running while freedup tries to hardlink files it is > possible to loose both the source freedup wants to link to and the copy > freedup still removes on error. Running cyr_expire and freedup (up to > 1.6-2) together is a really bad idea. > > If it's of interest I can provide my patch here, too. Hi, I have not used freedup for restoring the hardlinks but I'm interested in the patch. If it's not big could you post it here? Thanks, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Another 2.4 upgrade horror story
> Hi! > > On Tue, Sep 25, 2012 at 03:25:45PM +0200, Wolfgang Breyha wrote: >> Sebastian Hagedorn wrote, on 25.09.2012 14:01: >> > I realize that some of our problems were caused by infrastructure >> that's >> > not up to current standards, but nonetheless I would really urge you >> to >> > never again use an upgrade mechanism like that. Give admins a chance >> to >> > upgrade indexes in the background and over time. >> >> There is such an upgrade path using a murder environment and moving >> mailboxes >> between backends. We used that for our 150k user infrastructure and had >> no IO >> headaches at all. It was a good moment to update distribution, >> filesystems, >> hardware, as well. > > Could you elaborate on that? I considered that option, but seeing as > moving > even a couple dozen users from a backend to another using RENAME takes > hours > and one backend contains thousands of users, I decided to just live with > the ~1 > day of unbearable slowness. Or do you know of a fast way? I did the migration by moving mailboxes between backends of 1TB each, having scripted it up to only move employees when it was 00:00 - 06:00 in their local timezone, and left the script running on each v2.3 backend for a few days. Took a few weeks in all to migrate and upgrade all our backends in turn, but no one experienced any downtime. The only gotcha I experienced was I forgot that cyrus was configured to hardlink mail, which of course was no longer the case after each mailbox was migrated, so my disk usage exploded. (But easily fixed/restored once identified). It comes down to having spare backend(s) to move people on to, and time to do it patiently. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: Migrating to Office 365
___ From: Dan White [dwh...@olp.net] Sent: 25 June 2012 16:03 To: Simon Geary Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Migrating to Office 365 On 06/25/12 15:38 +0100, Simon Geary wrote: >Hello list, > >I need to migrate mailboxes from a few different Cyrus servers to Microsoft >Office 365 and am looking for a few tips. > >The migration process requires a CSV file to be prepared with a single >username and password that has access to every student mailbox. Is it >possible to create such a superuser account in Cyrus? > >Microsoft provide some tips on the required formatting of the CSV file for >other IMAP servers here, but I am struggling to find a format that works. >http://help.outlook.com/en-us/140/Ee730334.aspx > >Anyone have any ideas on how to accomplish this? I would guess that the Dovecot example should work. What's referred to as an Administrator should be the equivalent of a proxyservers entry in imapd.conf. e.g.: proxyservers: mailadmin For the proxy authentication to work, you will need to enable sasl authentication, and offer a mechanism which supports it: http://www.cyrussasl.org/docs/cyrus-sasl/2.1.25/mechanisms.php -- Dan White Thanks very much Dan, it is working now. After enabling SASL and adding the superuser account to proxyservers in imapd.conf I was able to format the CSV file using the Microsoft example. So for anyone who comes across the problem, this is the CSV format that did the trick. EmailAddress,UserName,Password,UserRoot stude...@qmul.ac.uk,superuser,password,studentaccount1 stude...@qmul.ac.uk,superuser,password,studentaccount2 stude...@qmul.ac.uk,superuser,password,studentaccount3 You saved me an awful lot of time, so thanks again. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Migrating to Office 365
Hello list, I need to migrate mailboxes from a few different Cyrus servers to Microsoft Office 365 and am looking for a few tips. The migration process requires a CSV file to be prepared with a single username and password that has access to every student mailbox. Is it possible to create such a superuser account in Cyrus? Microsoft provide some tips on the required formatting of the CSV file for other IMAP servers here, but I am struggling to find a format that works. http://help.outlook.com/en-us/140/Ee730334.aspx Anyone have any ideas on how to accomplish this? Cheers, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: IMAP error reported by server. Invalid body section.
> On Jun 22, 2012, at 5:10 PM, Simon Matter wrote:
>
>>> On 06/22/2012 09:43 AM, Dave McMurtrie wrote:
>>>> On 06/22/2012 06:35 AM, Adam Tauno Williams wrote:
>>>>> On Thu, 2012-06-21 at 13:07 -0300, Rodrigo Abantes Antunes wrote:
>>>>>> The source from horde3 is exactly the same as horde4
>>>>>
>>>>> That is expected. It isn't the message but the interpretation of the
>>>>> message. These evil messages contain many named parts separated by a
>>>>> boundry (the boundry value is declared in the header of the message).
>>>>> Then parts of a message can refer to other parts of the message. So
>>>>> either H4 can't correctly [or incorrectly!] parse the message into
>>>>> parts
>>>>> by boundry or one part references another part that isn't found.
>>>>>
>>>>> It would be useful to ask this question on the Horde / IMP mail list.
>>>>
>>>> I think this originated as a bug report to Horde and they think it's
>>>> the
>>>> IMAP server's fault.
>>>>
>>>> Rodrigo, can you forward the message to me?
>>>
>>> Hi. Rodrigo sent me the message. I wanted to confirm that the MIME
>>> structure was correct so I used munpack which was able to successfully
>>> unpack all the message parts. This isn't a guarantee that the MIME
>>> structure is correct, but at the very least I can't definitely say the
>>> message is malformed.
>>>
>>> I then imported the message into my mailstore. reconstruct was not
>>> pleased with it from the start:
>>>
>>> Jun 22 15:29:48 cyrusbe-d04 reconstruct[28021]: ERROR: message has more
>>> than 1000 header lines, not caching any more
>>
>> I did the same test on my box and reconstruct worked fine and I can view
>> the message with Squirrelmail and Thunderbird without any problems.
>>
>> What's your version of cyrus-imapd you tested with? I have tested with a
>> 2.4.16 server.
>>
>
> Interesting. The server I'm testing on isn't a released version, but
> rather a snapshot build from the caldav-2.4 Git branch. It should be
Without looking at it closely, I have two things in my setup which could
make the difference?
1) I have set lmtp_strict_rfc2821: 0
2) I have this patch:
--- cyrus-imapd-2.3.7/imap/message.c2006-10-28 22:18:08.0 +0200
+++ cyrus-imapd-2.3.7/imap/message.c.nobarenewlinescheck2006-10-28
22:21:55.0 +0200
@@ -256,8 +256,9 @@
r = IMAP_MESSAGE_CONTAINSNULL;
}
else if (*p == '\n') {
- if (!sawcr && (inheader || !allow_null))
- r = IMAP_MESSAGE_CONTAINSNL;
+ /* Do *NOT* check for RFC compliant line breaks (bare
newlines) */
+ /* if (!sawcr && (inheader || !allow_null))
+ r = IMAP_MESSAGE_CONTAINSNL; */
sawcr = 0;
if (blankline) {
inheader = 0;
Regards,
Simon
> fairly close to 2.4.16. Can you grab telemetry and see what
> Squirrelmail/tbird is requesting?
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: IMAP error reported by server. Invalid body section.
> On 06/22/2012 09:43 AM, Dave McMurtrie wrote: >> On 06/22/2012 06:35 AM, Adam Tauno Williams wrote: >>> On Thu, 2012-06-21 at 13:07 -0300, Rodrigo Abantes Antunes wrote: >>>> The source from horde3 is exactly the same as horde4 >>> >>> That is expected. It isn't the message but the interpretation of the >>> message. These evil messages contain many named parts separated by a >>> boundry (the boundry value is declared in the header of the message). >>> Then parts of a message can refer to other parts of the message. So >>> either H4 can't correctly [or incorrectly!] parse the message into >>> parts >>> by boundry or one part references another part that isn't found. >>> >>> It would be useful to ask this question on the Horde / IMP mail list. >> >> I think this originated as a bug report to Horde and they think it's the >> IMAP server's fault. >> >> Rodrigo, can you forward the message to me? > > Hi. Rodrigo sent me the message. I wanted to confirm that the MIME > structure was correct so I used munpack which was able to successfully > unpack all the message parts. This isn't a guarantee that the MIME > structure is correct, but at the very least I can't definitely say the > message is malformed. > > I then imported the message into my mailstore. reconstruct was not > pleased with it from the start: > > Jun 22 15:29:48 cyrusbe-d04 reconstruct[28021]: ERROR: message has more > than 1000 header lines, not caching any more I did the same test on my box and reconstruct worked fine and I can view the message with Squirrelmail and Thunderbird without any problems. What's your version of cyrus-imapd you tested with? I have tested with a 2.4.16 server. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: mailboxes.db vs IMAP client irregularities
> On 05/19/2012 01:51 PM, Stephen Ingram wrote: >> I'm running 2.4.13 from the invoca rpms on CentOS 5.8. I recently had >> an issue with a folder in a mailbox that would not show any >> subfolders. I created a new folder 'folder2' and moved all of the >> subfolders to it and then performed a reconstruct on the new set of >> folders and everything worked. Now I deleted the old folder 'folder' >> from the file system and then (after it wouldn't go away from the >> cyradm listing) used cyr_dbtool to manually remove it (and the >> subfolders) from the mailboxes.db file. The old folders and subfolders >> are now gone, however, I can't (using the IMAP client) rename >> 'folder2' back to 'folder' as when I do, the subfolders are not >> visible. >> >> I've dumped the mailboxes.db file to a flat file to look and see if >> there is anything in there that wasn't visible in cyradm or using >> cyr_dbtool show. Everything is as expected except there are some >> DELETED.user.xxx.folder entries at the top. Are you not allowed the >> create folders with the same name you've just deleted? Where are these >> DELETED folders actually stored and how long does it take them to go >> away? (I'm not using delayed expunge.) > > Sounds like you are using delayed delete. Mine show up in > /imap/mail/C/DELETED/ . How long they stay around depends on when you > run cyr_expire and what parameters you give it. > > > Man page entries: > > deletedprefix: DELETED > If "delete_mode" set to be "delayed", the prefix for > the deleted mailboxes hierarchy. The hierarchy delimiter will be > automatically appended. > > delete_mode: immediate > The manner in which mailboxes are deleted. "immediate" mode > is the default behavior in which mailboxes are removed > immediately. In "delayed" mode, mailboxes are renamed to a special > hiearchy defined by the "deletedprefix" option to be removed later by > cyr_expire. Just to clear this up, if he's using our invoca rpms then his man page reads: delete_mode: delayed The manner in which mailboxes are deleted. "immediate" mode is the the mode in which mailboxes are removed immediately. In "delayed" mode, mailboxes are renamed to a special hiearchy defined by the "deletedprefix" option to be removed later by cyr_expire. Note: This Invoca RPM build uses delayed by default instead of immedi- ate for delete_mode. Allowed values: immediate, delayed Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: (important) cyrus-imapd 2.4.16 released
> On Thu, 2012-04-19 at 11:00 +0100, Jeroen van Meeuwen wrote: >> Hi there, >> >> I'm forwarding this message posted to the announcement mailing list >> originally, to let you know any upgrades should target 2.4.16 as opposed >> to >> 2.4.15. >> >> We are pleased to announce the release of Cyrus IMAPd 2.4.16. >> [1] https://bugzilla.cyrusimap.org/show_bug.cgi?id=3651 > > The bug contains the comment: > Can we please confirm/deny this only breaks systems with fulldirhash: 1 > configured? > > Is the answer to that question "Yes"; sites that do not specify > "fulldirhash" or have a "fulldirhash: 0" in their imapd.conf are not > affected and do not need to rehash. I think that's correct. At least we have hashimapspool: true as default in our RPMs and fulldirhash is disabled by default and there are no problems AFAIK. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Lock problem with mupdate
> On Tue, 17 Apr 2012 05:43:41 -0400 Dave McMurtrie wrote: >> strace the server and find out what it's doing. Look at the logs on the >> server. Run netstat to confirm both sides still have an established >> connection and that iptables isn't silently dropping packets, etc. > > No iptables active, due to private net. SELinux? Same answer as above I guess? Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
RE: RPMs for cyrus-imapd v2.4.13?
> These are source RPMs. Are there any binary RPMs available? I'm kinda new > at Linux and don't know how to install a source RPM. (Also, will it use > the same directory structure as the binary RPMs?) Hi Larry, We do have binary RPMs for RHEL3, 4, 5 and 6 for ix86 and x86_64 in our YUM repositories which are only accessible for our paying customers. But, if you're interested in a current set of binary packages just let me know and I'll send them to you in private mail. And yes, the directory structure should still be compatible, as the packages provided by RedHat, Fedora or Kolab Systems are based on a old version of our RPMs - despite the fact that in certain RPMs the changelogs have been cleaned up :) Regards, Simon > > Thanks, Larry > > From: Chris Conn [mailto:cc...@abacom.com] > Sent: Thursday, January 19, 2012 12:03 PM > To: Rosenbaum, Larry M. > Cc: info-cyrus@lists.andrew.cmu.edu > Subject: Re: RPMs for cyrus-imapd v2.4.13? > > On 2012-01-19 11:40, Rosenbaum, Larry M. wrote: > Are there any RPMs available yet for cyrus-imapd v2.4.13? We're running > RHEL6. > > Larry M. Rosenbaum > Oak Ridge National Laboratory > > http://www.invoca.ch/pub/packages/cyrus-imapd/ > > I built them on a Centos 6.2 server and they worked well in my setup. > > Chris > > > > > > > > > > Cyrus Home Page: http://www.cyrusimap.org/ > > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: anysievefolder and autosievefolders gone in 2.4?
> Hello, > >> Sorry - I've been planning to do it for ages, and it just hasn't >> happened >> due to other things always being more pressing. It's on the "MUST HAVE" >> list for 2.5. > > ok, but where can I find a patch for the current 2.4.9 release of Ubuntu > 11.10? The latest release I can find at > http://email.uoa.gr/projects/cyrus/autosievefolder/ is from 2009 for the > 2.3.16 release. Check here: http://www.vx.sk/download/patches/cyrus-imapd/ If they don't apply, then you can extract the version in our current rpm: http://www.invoca.ch/pub/packages/cyrus-imapd/RPMS/ils-5/SRPMS/cyrus-imapd-2.4.13-1.el5.src.rpm I think I had to rework them. Simon > > Kind regards > Marten > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: High cpu usage for imapd processes on 2.4.12 (Invoca rpm)
> Hi, > > I've got an imap server being hammered by a program that is constantly > connecting via IMAP, searching, reading and deleting emails. At a very > rough estimate there'll be 10 to 20 new emails every minute that > arrive, are checked and deleted. > > The original machine is running 2.3.16 without replication and the > average load rarely went over 1. On the new server running 2.4.12 > (details below) I'm seeing the average load hover around 5 constantly. > The new box is running replication and this appears to be set up > correctly as the logs look pretty clean. top shows CPU percentage for > each imapd process running quite high. The largest mailbox contains > about 3000 emails and is usually at 50 to 60% CPU. > > I/O for the machine looks fine, I'm not seeing any iowait or high > usage with iostat. It's using local disks so not something I would > expect to be a problem. > > Can anyone suggest where I could start looking to find the cause? Or > point me in the direction of configuration options that are known to > have such an effect after upgrading between the two version? > > Thanks > Guy > > > New server: > name : Cyrus IMAPD > version: v2.4.12-Invoca-RPM-2.4.12-2el5_2 63b65d1c 2011-10-04 > vendor : Project Cyrus > support-url: http://www.cyrusimap.org > os : Linux > os-version : 2.6.18-238.el5 > environment: Built w/Cyrus SASL 2.1.22 > Running w/Cyrus SASL 2.1.22 > Built w/OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 > Running w/OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 > Built w/zlib 1.2.3 > Running w/zlib 1.2.3 > CMU Sieve 2.4 > TCP Wrappers > mmap = shared > lock = fcntl > nonblock = fcntl > idle = idled > > imapd.conf: > admins: cyradmin cyradmin-repl > configdirectory: /var/lib/imap > delete_mode: immediate > duplicate_db: skiplist > expunge_mode: default > hashimapspool: 1 > improved_mboxlist_sort: 1 > lmtp_downcase_rcpt: 1 > munge8bit: 0 > partition-default: /var/spool/imap > ptscache_db: skiplist > sasl_pwcheck_method: saslauthd > sendmail: /usr/sbin/sendmail > sievedir: /var/lib/imap/sieve > statuscache_db: skiplist > sync_authname: cyradmin-repl > server101_sync_host: 10.10.10.10 > server201_sync_host: 10.11.10.11 > sync_log: 1 > sync_log_channels: server101 server201 > sync_password: replpass > tls_ca_file: /etc/ssl/certs/domain.com.ca-bundle > tlscache_db: skiplist > tls_cert_file: /etc/ssl/certs/domain.com.crt > tls_key_file: /etc/ssl/certs/domain.com.key > unix_group_enable: 0 > normalizeuid: 1 Hi, I've not seen any such issues on my 2.4.x systems after upgrading from 2.3.x and I'm also wondering what's going on. So, is it the same hardware, os and everything and only cyrus changed? And then of course I don't know about replication how it affects it. What comes to mind is, did you try with "expunge_mode: delayed"? That can make quite a difference IIRC. Regards, Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: imapd 2.4.12 hangs
> I see, and I remember reading about it. > But, what will happen if I "make install" a skiplist version onto an older > BDB deployment > with running users? Will I need to run a reconstruct? Will I need to get > rid of older bdb files > manually? Also, do you think I will find any benefits by using skiplist on > NFS instead of BDB? Sorry, I can't help you here as we only do packaging on build hosts and then install resulting RPMs on production systems. What I'm sure is that you have to convert away from BDB before installing new binaries, because they won't do anything with any BDB file. I have never used NFS in this case. Simon > > Inviato da iPad > > Il giorno 21/ott/2011, alle ore 14:07, "Simon Matter" > ha scritto: > >>> Well, I've been using BDB for years on Cyrus. >>> Now I'm rebuilding a new distribution with the latest Cyrus, and I just >>> went on with the same >>> scheme. >>> How do I get rid of BDB? What will it use instead? >> >> skiplist, most databases default to skiplist anyway. Those which still >> default to BDB can be configured to use skiplist instead. Works fine for >> me and many others. >> >> Of course, I don't say your problems have to do with BDB, it's just >> possible. >> >> Simon >> >>> Gabriele. >>> -- >>> Da: Simon Matter >>> A: Gabriele Bulfon >>> Cc: info-cyrus@lists.andrew.cmu.edu >>> Data: 21 ottobre 2011 12.17.33 CEST >>> Oggetto: Re: imapd 2.4.12 hangs >>> Hi, I'm still having trouble running 2.4.12, once started, if I connect >>> to >>> port 143, I never >>> get the hello response, and the launched imapd goes in a loop. >>> These is the output of ldd against master, maybe some libraries are bad >>> versions? >>> Hi, >>> Do you build yourself? If so, do you really need BDB? I not, then you >>> may >>> want to build without BDB just to make sure to get rid of a possible >>> source of problems. >>> Regards, >>> Simon >>> linux-gate.so.1 =(0x009ff000) >>> libgssapi_krb5.so.2 =/sonicle/lib/libgssapi_krb5.so.2 (0x00a81000) >>> libkrb5.so.3 =/sonicle/lib/libkrb5.so.3 (0x00923000) >>> libk5crypto.so.3 =/sonicle/lib/libk5crypto.so.3 (0x00b76000) >>> libcom_err.so.3 =/sonicle/lib/libcom_err.so.3 (0x004a3000) >>> libkrb5support.so.0 =/sonicle/lib/libkrb5support.so.0 (0x00cc8000) >>> libresolv.so.2 =/lib/libresolv.so.2 (0x00a2e000) >>> libssl.so.0.9.8 =/sonicle/lib/libssl.so.0.9.8 (0x0030) >>> libcrypto.so.0.9.8 =/sonicle/lib/libcrypto.so.0.9.8 (0x0011) >>> libdb-4.8.so =/sonicle/lib/libdb-4.8.so (0x004a6000) >>> libpcre.so.0 =/sonicle/lib/libpcre.so.0 (0x00257000) >>> libpcreposix.so.0 =/sonicle/lib/libpcreposix.so.0 (0x00cdf000) >>> libz.so.1 =/sonicle/lib/libz.so.1 (0x00277000) >>> libc.so.6 =/lib/libc.so.6 (0x00655000) >>> libdl.so.2 =/lib/libdl.so.2 (0x0028f000) >>> libpthread.so.0 =/lib/libpthread.so.0 (0x00294000) >>> /lib/ld-linux.so.2 (0x00421000) >>> >>> Cyrus Home Page: http://www.cyrusimap.org/ >>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>> >> >> >> > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: imapd 2.4.12 hangs
> Well, I've been using BDB for years on Cyrus. > Now I'm rebuilding a new distribution with the latest Cyrus, and I just > went on with the same > scheme. > How do I get rid of BDB? What will it use instead? skiplist, most databases default to skiplist anyway. Those which still default to BDB can be configured to use skiplist instead. Works fine for me and many others. Of course, I don't say your problems have to do with BDB, it's just possible. Simon > Gabriele. > -- > Da: Simon Matter > A: Gabriele Bulfon > Cc: info-cyrus@lists.andrew.cmu.edu > Data: 21 ottobre 2011 12.17.33 CEST > Oggetto: Re: imapd 2.4.12 hangs > Hi, I'm still having trouble running 2.4.12, once started, if I connect to > port 143, I never > get the hello response, and the launched imapd goes in a loop. > These is the output of ldd against master, maybe some libraries are bad > versions? > Hi, > Do you build yourself? If so, do you really need BDB? I not, then you may > want to build without BDB just to make sure to get rid of a possible > source of problems. > Regards, > Simon > linux-gate.so.1 =(0x009ff000) > libgssapi_krb5.so.2 =/sonicle/lib/libgssapi_krb5.so.2 (0x00a81000) > libkrb5.so.3 =/sonicle/lib/libkrb5.so.3 (0x00923000) > libk5crypto.so.3 =/sonicle/lib/libk5crypto.so.3 (0x00b76000) > libcom_err.so.3 =/sonicle/lib/libcom_err.so.3 (0x004a3000) > libkrb5support.so.0 =/sonicle/lib/libkrb5support.so.0 (0x00cc8000) > libresolv.so.2 =/lib/libresolv.so.2 (0x00a2e000) > libssl.so.0.9.8 =/sonicle/lib/libssl.so.0.9.8 (0x0030) > libcrypto.so.0.9.8 =/sonicle/lib/libcrypto.so.0.9.8 (0x0011) > libdb-4.8.so =/sonicle/lib/libdb-4.8.so (0x004a6000) > libpcre.so.0 =/sonicle/lib/libpcre.so.0 (0x00257000) > libpcreposix.so.0 =/sonicle/lib/libpcreposix.so.0 (0x00cdf000) > libz.so.1 =/sonicle/lib/libz.so.1 (0x00277000) > libc.so.6 =/lib/libc.so.6 (0x00655000) > libdl.so.2 =/lib/libdl.so.2 (0x0028f000) > libpthread.so.0 =/lib/libpthread.so.0 (0x00294000) > /lib/ld-linux.so.2 (0x00421000) > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: imapd 2.4.12 hangs
> Hi, I'm still having trouble running 2.4.12, once started, if I connect to > port 143, I never > get the hello response, and the launched imapd goes in a loop. > These is the output of ldd against master, maybe some libraries are bad > versions? Hi, Do you build yourself? If so, do you really need BDB? I not, then you may want to build without BDB just to make sure to get rid of a possible source of problems. Regards, Simon > linux-gate.so.1 =(0x009ff000) > libgssapi_krb5.so.2 =/sonicle/lib/libgssapi_krb5.so.2 (0x00a81000) > libkrb5.so.3 =/sonicle/lib/libkrb5.so.3 (0x00923000) > libk5crypto.so.3 =/sonicle/lib/libk5crypto.so.3 (0x00b76000) > libcom_err.so.3 =/sonicle/lib/libcom_err.so.3 (0x004a3000) > libkrb5support.so.0 =/sonicle/lib/libkrb5support.so.0 (0x00cc8000) > libresolv.so.2 =/lib/libresolv.so.2 (0x00a2e000) > libssl.so.0.9.8 =/sonicle/lib/libssl.so.0.9.8 (0x0030) > libcrypto.so.0.9.8 =/sonicle/lib/libcrypto.so.0.9.8 (0x0011) > libdb-4.8.so =/sonicle/lib/libdb-4.8.so (0x004a6000) > libpcre.so.0 =/sonicle/lib/libpcre.so.0 (0x00257000) > libpcreposix.so.0 =/sonicle/lib/libpcreposix.so.0 (0x00cdf000) > libz.so.1 =/sonicle/lib/libz.so.1 (0x00277000) > libc.so.6 =/lib/libc.so.6 (0x00655000) > libdl.so.2 =/lib/libdl.so.2 (0x0028f000) > libpthread.so.0 =/lib/libpthread.so.0 (0x00294000) > /lib/ld-linux.so.2 (0x00421000) > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: singleinstancestore obsolete?
> Ramprasad wrote, on 19.10.2011 15:37: >> I think , writing a standalone index upgrade utility , like the ipurge , >> seems to be a reasonable thing to do >> >> >> If there was a light enough index upgrade possible ( only for inboxes .. >> not subfolders ) Then I could stop cyrus , fork probably around 100 >> parallel upgrades take a 2-3 hour downtime and then start services again > > Currently I'm considering the following way: > *) build a new backend with same partition count/names > *) output something like >"find ../partxx -type -f -links +2 -ls |sort" to a file for every > partition >on the old backend. -ls maybe replaced by special -printf. > *) move mailboxes from 2.3 to 2.4 with rename keeping partitions the same > *) write a script reading "find"-output and relink all the files with same >inodes. This can be done at any time and with low impact. > > I think that should be pretty safe if the script has enough safty belts in > place. Mails moved or deleted in the meantime are a special case. Don't > know > if it's worth to try hard to find them. I think for the singleinstancestore, you can redo it after migration with tools like hardlink or http://www.freedup.org/. IIRC I did this once and it worked fine - I think I was using a simple bash script as you suggested above. The only problem could be that you need the extra space while migration is going on. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
