Re: Slow delivery to Outlook 2k/2k3
Get people using crap like outlook to use pop3, their clients are not good enough for anything else anyway. Well, considering how well calendaring is implemented, along with contact integration, Outlook's far from 'crap', at least in the minds of the people happily using it. Just not with IMAP. It's really odd, apparently the folks on the Outlook development team either don't give a damn or have no idea how many users are being driven off the platform because of it's defects. It's really pretty surprising they've let it remain so shitty for so long. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Slow delivery to Outlook 2k/2k3
I have seen such problems occur when Outlook is left running for long periods of time. Believe it or not, a complete reboot of Windows is often necessary. Simply restarting Outlook is not always sufficient. I'll second that rather grim pronouncement. Outlook's IMAP handling is DEFECTIVE. It really cannot reliably stay connected to an IMAP server. It's been broken in all versions but 2003 seems to be the MOST defective. It plainly just doesn't work. I have seem it require a complete reboot of the PC to free it up. If you use an IMAP server don't depend on getting Outlook to work with it. Now, Outlook Express? That works, quite handily. -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: why does cyrus have to have a password
I would really not like people to be able to ssh into the cyrus account. Look into how your OS restricts logins. No doubt pam or whatever can be configured to specifically deny non-localhost logins to that account. Most handle root this way too. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: listen to a dynamic interface
You could always stop/start cyrus based on the ppp interface being live. That way it'd bind to whatever address is active and then only when the link is up. I suppose you could leave it live all the time and just restart it when the ppp link state changes. It's not common to run a mail server behind a dynamic address, they generally benefit from being on stable, always-on, connections. You *can* run them otherwise but it's usually not recommended. Other ideas like using a VPN come to mind but it depends on just how complex you need it to be. -Bill Kearney - Original Message - I want to set /etc/cyrus.conf to start another imap[s] proccess to listen in a dynamic ip interface (ppp0). AFAIU, you have to specify the ip address of the interface in the listen option. So, how would a ppp0 interface that has a dynamic ip that needs its own proccess (that's because of the ssl certificate) be configured? Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Outlook does not delete but displays deleted messages asstrike-trough
Thunderbird seems to be configured by default to _move_ the message to the Deleted Items-folder and thus they vanish from whatever folder they were. No they don't 'move'. They're simply marked as deleted, just as Outlook does, but they remain in the original folder. Try it, delete one in tbird and then connect to the very same mailbox using something like pine. You'll see the message is still there, just marked for deletion. The problem with this is wasted space. The user thinks the message is gone but it's still there taking up disk space and potentially reducing their quota. Leaving them with strikeouts at least introduces the user to the concept of 'purging their own trash'. I've found running schedule deletion tasks to be a VERY BAD IDEA. Users have all sorts of stupid notions about what being 'in the trash' means. Like they want me to go climb into the dumpster to go get the message they put in the trash 3 months ago... By the time you set a 'wide enough' window of time for automatic deletion you run out of disk space. Bottom line, I've found it's better to use strike-through marking and have the client do the purging. Some clients offer a feature to do it on leaving the application or folder, prompted by a dialog box. Works well for most users, especially the ones that want to delete it, but not really. Ugh, users A few users of ours moved over to Thunderbird because of this MS 'feature'. I think outlook's utterly crappy handling of IMAP is a more powerful motivator. Outlook Express, on the other hand, does a fine job of supporting IMAP. But the regular Outlook 2003 and past versions have had absolutely crappy IMAP handling. Such that it makes it almost impossible to use OL2003 against an IMAP server. I long since gave up on it for IMAP access. -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus postfix, message being corrupted when viewed by the client
You've tried two different computers? Running different mail client software? Does the exact same 'bad content' get delivered to an entirely different mail client program? Scare up a copy of pine, outlook express, thunderbird or mozilla suite and see if they all get the same corrupted content. When you speak of MIME sections all sorts of nonsense can develop. Let's be sure that all clients are getting exactly the same bad content before blaming cyrus. Likewise, what language and encoding are these messages? By the .cn TLD one might gather it's chinese? Encodings, charsets and languages are mess if you don't have all the right settings done in the right order. Make sure the underlying OS configuration for them is working properly. -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus postfix, message being corrupted when viewed by the client
I have not yet used another client , because logically, if it works on 1 server with the same patch levels, then it should work on the other server. Not if the client is defective or is unable to handle connecting to the server properly. K , just tested outlook express windows se(both servers ok) thunderbird osx(both servers ok) mail osx v205 (one server messed up, one server ok) Thought that would happen. grin It's very rare for an e-mail server to function normally but deliver corrupted messages. Since cyrus wasn't complaining in it's logs I figured it had to be something client-side. in the meantime i will get onto apple about a possible bug Sounds like that's the right place to start. You could go a step further and run 'ethereal' to trap the network packets. See if the data being sent to the clients is correct or not. If the packets on the wire are corrupted then there's some sort of handshaking nonsense going wrong. Since you've got some clients that can get GOOD content then unless the apple mail.app has a bug I'd start wondering about some sort of language/encoding problems. I'm betting the packets are good and the mail.app is getting seriously confused about the MIME structure. thanks bill. You're most welcome. Always try different clients before going to the extremes of changing the servers. -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
auth against active directory?
Has anyone setup their cyrus-imap server to authenticate against a Windows Active Directory domain? Any tips on doing it? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: auth against active directory?
Heh, 'easy enough' and LDAP rarely seem to be found together. Throw in SASL and it /really/ goes downhill. I figure it should be easy but given that I've never actually made a 'generic' LDAP connection to an active directory I'm not entirely sure where to start. And given the potential for amount of time fiddling with sasl is known to absorb I'm doubly cautious. -Bill Kearney - Original Message - I do alot of auth against our active directory for certain internal websites (using mod_ldap), but have had no need to do this for Cyrus yet. However, your domain controller is just an ldap server, for all intents and purposes. You can use saslauthd ldap auth, using your DC as the ldap server. The only thing I remember was that the filter was a little different, but you should be able to find that via google easy enough. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus WebMail
The optimal web mail system would run on the Cyrus server and would access mail directly without using imap protocol at all. Trouble is this also ends up being brittle. It's one thing to remain compatible with the IMAP spec. It's another thing entirely to develop and maintain a whole other spec for the filenames and internal setups. This means that any changes or improvements to the cyrus codebase would cause upgrade problems. The direct access tools would end up being stuck with old versions of the codebase, at least until someone got around to adapting them. And for what? It seems like a better idea to consider options available with cyrus-murder and proper network configuration. Mirapoint apparently have done this with their branch of Cyrus. But that's proprietary. Good, fast, cheap... pick two. That's always the rule. -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Does anyone know what these log messages mean?
Post your /etc/imapd.conf file. - Original Message - Nov 29 10:25:17 imap_svr imap[14331]: [ID 702911 auth.error] sql_select option missing Nov 29 10:25:17 imap_svr imap[14331]: [ID 702911 auth.error] auxpropfunc error no mechanism available Nov 29 10:25:17 imap_svr imap[14331]: [ID 702911 auth.debug] _sasl_plugin_load failed on sasl_auxprop_plug_init for plugin: sql Nov 29 10:25:26 imap_svr perl[14330]: [ID 702911 auth.notice] No worthy mechs found Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus WebMail
SquirrelMail!! Or it you want less bloat, try Ilohamail. I was wondering if someone tried to write some webmail application for Cyrus IMAPD. Using IMAP connection for Webmail are not this, what i want for today. Why not? Is there some documentation about Cyrus internal databases, or maybe API about this stuff ? If you use an IMAP webmail client it'll work with nearly all IMAP servers. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: handling unqualified names?
Have you tried virtdomains: userid ? This will disable the reverse lookup. Yes, it doesn't reverse but it also tacks on the domain.tld of the server. Or, more accurately, it tacks on the domain.tld of the interface on which the connection was made. So if 192.168.12.2 is 'mail.domain2.tld' then connections into it would be from '[EMAIL PROTECTED]'. Equally true if there are other IP addresses configured to respond to mail then their domain.tld will be used. It does work in cyrus and I think it also does work with SASL auxprop LDAP. It does not work with auxprop SQL where this auxprop plugin will always append the servername as a realm to unqualified userids. Ah, that explains it. I'll have to compare the source between the sql.c plugin and the others. you set virtdomains: userid and login with an unqualified userid the auxprop plugin will always append the servername. The 'auxprop' concept itself? Or the SQL plugin that's called via auxprop? I may be wrong here, but the defaultdomain setting is there to unqualify qualified userids. So if you login with [EMAIL PROTECTED] as in your example above the defaultdomain is stripped and the userid becomes joe. SASL auxprop plugin SQL will then append the servername as a realm leading to a lookup of [EMAIL PROTECTED] So the answer is that auxprop plugin SQL does not support unqualified userids, I think. Unfortunately you may be correct. Which may lead me back to using pam_mysql via saslauthd. Six steps forward, eight steps back, it seems. Here I was thinking it'd be more efficient to call SQL directly, but n! Yeesh. Of course by using PAM I get stuck not being able to use challenge-response, correct? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
virtdomains: userid?
The docs for /etc/imapd.conf read: virtdomains: off Enable virtual domain support. If enabled, the userâs domain will be determined by splitting a fully qualified userid at the last '@â' or '%' symbol. If the userid is unqualified, and the virtdomains option is set to on, then the domain will be determined by doing a reverse lookup on the IP address of the incoming network interface, otherwise the user is assumed to be in the default domain (if set). Allowed values: off, userid, on What actions does the userid value trigger? In testing I can't see that it does anything. Are the docs correct? Thanks, -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: virtdomains: userid?
Would this help your reverse DNS lookup trouble? Or will using the unqualified name still trigger a reverse lookup? It doesn't appear to help my situation, in that I'd like logins without a qualified name to use just the bare username and NOT append a realm onto it. Is this possible? -Bill Configuring Virtual Domains Introduction Virtual domains is the practice of hosting a service for more than one domain on one server. Cyrus IMAP has the ability to host IMAP/POP mailboxes for multiple domains (e.g. [EMAIL PROTECTED] and [EMAIL PROTECTED]) on a single server or Murder. In order to accomplish this, Cyrus needs to know which domain to look in when a mailbox is accessed. There are two ways in which Cyrus can determine the domain: * Fully qualified userid - the client logs in with a userid containing the domain in which the user belongs (e.g [EMAIL PROTECTED] or test%example.net) * IP address - the server looks up the domain based on the IP address of the receiving interface (useful for servers with multiple NICs or using IP aliasing) Both of these methods are active if the virtdomains option is set to on (or yes, 1, true) and can be used in conjunction with one another. If the virtdomains option is set to userid, then only the first method is used. Note that a fully qualified userid takes precedence over a domain obtained from the IP address. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cross-realm login denied?
Hello, I'm trying to use cyrus to connect using [EMAIL PROTECTED] where domain1.tld might be one of several domains. I'm trying to do the lookup using: sasl_sql_select: select accountuser.password from accountuser, virtual on accountuser.username=virtual.username where virtual.alias='[EMAIL PROTECTED]' When tested using the mysql client that select query does return a valid record. But when I try logging in that way using IMAP it reports: Nov 24 13:46:08 mailserver imap[11184]: cross-realm login [EMAIL PROTECTED] denied It would appear 'something' in the process is deciding to split apart the login username and then noticing it's not the 'correct' domain. The question is, what is making that decision and what data is it basing it's decision upon? The /etc/imapd.conf file defaultdomain: value does not appear to influence this. When I set it to the same 'domain1.tld' as one being requested it STILL complains about it being cross-realm. The hostname.domain of the box itself IS different. The box is on domain2.tld and the incoming request is for domain1.tld. So what gives here? What can I configure to let it login using what might be different domains? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
howto out of date?
It seems like most of the various howto websites aren't in sync with the current state of cyrus imap and sasl2. That is, since postfix and cyrus can both speak SQL natively where's the point in bothering with use of pam_mysql? Not to knock it, of course, but why bother using it along with pam and saslauthd when you can just use sql natively? Then there's the matter of whether or not to use encrypted passwords. If you use them then doesn't that limited using challenge-response? But is that really needed/supported in most clients? Having plain text passwords in the database has it's own risks but that traffic is generally not 'in the clear'. Using plain text passwords from the client to the server is a bad idea of course but how secure does this truly need to be, in regard to how it impacts password storage? If someone's setting up a brand new cyrus/posfix/squirrelmail/mysql server then what REALLY should be used? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyrus auth paths?
So, let me get this straight, using current versions of postfix, cyrus and sasl2 it's possible to authenticate in several different ways: postfix - 1) via sql directly in main.cf postfix - 2) via saslauthd a) using sql configured in /usr/lib/sasl2/smtpd.conf or b) using pam configured in /etc/pam.d/smtp (using pam_mysql) where smtpd.conf says to use pam. cyrus - 3) via sql directly in imapd.conf - 4) via saslauthd a) using sql configured in /usr/lib/sasl2/imapd.conf or b) using pam configured in /etc/pam.d/imap (using pam_mysql) where imapd.conf says to use pam. Does this about summarize the routes possible to basically accomplish the same thing? If so, what are the configuration syntaxes appropriate for each? In 2a 2b, postfix tells salsauthd what to use via the 'smtpd_sasl_application_name' variable. In 2a it would expect there to be SQL config directives in the smtpd.conf file. 2b would fall through to pam which would use smtp based on the port being looked up from /etc/services. Correct? How would cyrus-imap do the same thing in 4a 4b? By setting 'imap_sasl_application_name' , 'imapd_sasl_application_name' or something else? How does cyrus inform saslauthd a la postfix? Can it? How would saslauthd 'know' where to go looking for the config info needed? In 4b it's clear, it simply falls through to pam which handles it based on the port lookup from /etc/services. Again, sorry if this seems tedious to some of the more learned members of the lists. But I think if these are better documented to match up with the current versions of the various pieces involved it'll go a long way toward shaking off the notions about cyrus and sasl being complicated. -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: How to migrate only one domain in a multidomains cyrus imap server?
But there are some articles describing howto migrate or recover the whole cyrus imap server but no how to migrate one of all the domains in the cyrus imap server. Could you explain what you're trying to migrate? Where are the accounts now and where do you want to put them? One idea that comes to mind would be to use something like imapsync to move them. There may be other ways to move them around but if they're mail and you have two accounts then it might work. But it really depends on what is you're trying to do. Can you describe the problem better? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
right way to call saslauthd from cyrus?
I've asked similar question over in cyrus-sasl but that's about postfix, this is about cyrus itself. What's the current state of how to tell cyrus to authenticate using mysql? In the past it seems the combination was to have cyrus call saslauthd, which is turn uses PAM via pam_mysql to the database itself. This seemed to work fine using sasl1 but I cannot seem to make it work with sasl2. Old box had both, new box has only sasl2. I'm using these versions pam_mysql 0.8pre3, postfix-2.2.5, cyrus-sasl-2.1.21, cyrus-imap-2.1.12, mysaql-4.1.12, centos-4.2 (rhel4 So main.cf would have various lines like this: sender_canonical_maps = mysql:/etc/postfix/mysql-canonical.cf And :/etc/postfix/mysql-canonical.cf contains: hosts = localhost user = mail password = secret dbname = mail table = virtual select_field = alias where_field = username additional_conditions = and status = '1' limit 1 And /etc/pam.d/imap contains (edited for username/passwd of course and all each on a full line, no trailing \ char) auth sufficient pam_mysql.so verbose=1 sqllog=true user=mail passwd=secret \ host=localhost db=mail table=accountuser usercolumn=username \ passwdcolumn=password crypt=1 logtable=log logmsgcolumn=msg \ logusercolumn=user loghostcolumn=host logpidcolumn=pid logtimecolumn=time account required pam_mysql.so verbose=1 sqllog=true user=mail passwd=secret \ host=localhost db=mail table=accountuser usercolumn=username \ passwdcolumn=password crypt=1 logtable=log logmsgcolumn=msg \ logusercolumn=user loghostcolumn=host logpidcolumn=pid logtimecolumn=time But this causes a whole raft of errors in maillog: (more than this quantity appear, it may be related to the number of daemon spawned?) Nov 17 19:17:07 cbox imap[19003]: sql_select option missing Nov 17 19:17:07 cbox imap[19002]: auxpropfunc error no mechanism available Nov 17 19:17:07 cbox imaps[19003]: sql_select option missing Nov 17 19:17:07 cbox imaps[19002]: auxpropfunc error no mechanism available Nov 17 19:17:07 cbox lmtpunix[19003]: sql_select option missing Nov 17 19:17:07 cbox lmtpunix[19003]: auxpropfunc error no mechanism available Am I calling things wrong? Well, obviously I've got something wrong but I just can't seem to hit upon the right combination to get this thing running. I think it's a case to too many HOWTO docs spanning too many different versions all adding up to a mess. I sort of like how pam_mysql has logging options. Thus I thought using saslauthd and on to pam was the right means to maintain that functionality. HELP! -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Importing MBOX-formatted mailboxes into Cyrus?
Have you seen this page on the wiki: http://acs-wiki.andrew.cmu.edu/twiki/bin/view/Cyrus/MboxCyrusMigration Also consider using mbox to other format converters and then pulling them into cyrus. Or, potentially, setup a temporary server using something like the uw-imap server. It'll read directly from the mbox files. Then you could use imapsync to pull them from there onto the target cyrus box. Does the QMP server have any interfaces? POP? You could always setup some scripts and fetchmail to pull them off via POP and put them into cyrus. There's more than one way to solve the problem. Some are good for really BIG migrations and others are well suited for quick hacks. -Bill Kearney - Original Message - We are currently moving away from a POP3-based system (QuickMail Pro server and client) to IMAP, with Cyrus as the IMAP server and primarily Thunderbird as the client. Have already succesfully transitioned our users to using IMAP via Thunderbird. The problem we have run into is with older emails. QuickMail uses its own format for storing messages. I was able to convert these into standard MBOX format using a tool called qmptombox, which we were able to import into Thunderbird's Local Folders. Ideally, however, we would like to have these messages reside on the IMAP server itself. Are there any tools available that will take MBOX mailboxes and insert them into Cyrus? If not, if anyone has found a suitable workaround, I would love to hear it. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: duplicate suppression and syncronizing?
The duplicate supression database is used by lmtpd when delivering mail. It is not used at all by imapd. If it was amusing things would happen when people tried to copy messages around. Yes! That's the answer I need to hear. I agree, it would be quite a mess if it affected moving and copying using an IMAP client. But before I told it to --delete the source mailbox I wanted to be SURE. Somewhat tangentally, is there a way to use sieve or something else within cyrus to investigate duplicates in a given folder? I do know some mail is duplicated. Thunderbird has an extension available to dig through the messages to find duplicates. But is there something clever within sieve that can do it? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
duplicate suppression and syncronizing?
Hi all, What's the story with duplicate suppression and it's affect on syncronizing with tools like imapsync or mailsync? It occurs to be that if suppression is enabled and it's blocking sync then I may be seeing more errors than are accurate. Does anyone know if the syncing tools are impacted by duplicate suppression? If so, how would I reconfigure cyrus to help avoid problems during this sync interval? I'm willing to reconfig it temporarily if necessary. I'm moving a metric assload of mail from an aging uw-imap server over to cyrus and it's not going as smoothly as I'd hoped. HELP! -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: using imapsync and altnamespace?
Bill, I ran into this problem with one of these imapsync/mailutil/mailsync utilities, and what I wound up doing was, I sucked the mail over to a dummy folder, then manually copied the messages up, reconstructed the inbox, and then deleted the dummy folder. Which won't work very well considering I'm moving a mailbox that's got nigh-on a decade worth of mail and hundreds of nested folders. Were it a simple drag-and-drop I'd have gone that route. But no GUI clients offer decent drag-and-drop of entire hierarchies, let alone with the ability to ignore/continue when errors crop up. Thus going with imapsync or mailsync seems to be the more effective solution. I've managed to get imapsync to move things across but not without a fair number of errors. The trouble is there's not relatively simple way to get a grip on what's causing the errors in a way that'll lend itself to easy managing. Thus far the magic incantation for imapsync seems to be: imapsync --noauthmd5 --host1 sourcehost --user1 srcuser \ --passfile1 .pass1 --include '^mail' \ --host2 desthost --user2 destuser --passfile2 .pass2 --regextrans2 's/^mail\.//' --syncinternaldates Where I'm pulling folders starting with 'mail' from the [EMAIL PROTECTED] mailbox and then pushing them into folders on the the [EMAIL PROTECTED] while removing the 'mail.' prefix from them, along with making sure desthost dates match up with those on the srchost; using plain text authentication for the connection. Whew. The downside is the final report: Statistics Time : 20202 sec Messages transfered: 120788 Messages skipped : 12465 Total bytes transfered : 934257544 Total bytes skipped: 104881890 Total bytes error : 7828056 Detected 1450 errors I've yet to decide how to determine what didn't get moved. But hey, only 1450 out of 120k ain't terribly bad. The only other problem is it doesn't appear to be pulling the /var/spool/mail/srcuser INBOX folder. Not a big deal as I can run another sync to handle that one, or even just use simple drag-and-drop. I really question the sanity of how uw-imap allows for browsing ~/ as part of the IMAP tree, it's really a pain in the ass to work around sometimes. I'm sure it's useful for someone but I've always found it to be a lot more trouble than it's worth. So does anyone know *for sure* how the --delete function will behave using imapsync? Does it only delete that which it confirms was successfully transferred? If so then I could use that to determine what to do based on what got left behind. But before I turn it loose with the delete option I'd sure like to avoid having to reload it all again... -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: refusing bad mail based on headers?
I've already done this for several of them. The trouble is there's not a consistent set of bad characters in the corrupted header. A fair number came back with grep '/.X-Message-Status: n$' But that's not all of them. A few more came back with: grep -P '/\x13\x42/' Good tip on using sed to edit them in place. I'd used grep with redirection. That takes the additional step of moving them around one it's complete but it saves me from having to reload it all should something have gone horribly wrong. Six of one, half dozen of the other I suppose. -Bill Kearney - Original Message - If you've identified the offending header, you could delete it with: $ sed -i.bak -e '/^X-Message-Status/d' mailbox Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
moving mail from uw to cyrus, damaged headers?
Hello, Anyone know of a way to move all 'good mail' from a uw-imap imap server to a cyrus server and have it ignore mail with bad headers? It would appear I've got a considerable number of messages in my uw-imap mailbox that have headers that cyrus doesn't like. That is, if I try using mailsync, mailutil or even a regular IMAP client like Thunderbird, pine or OE the messages won't 'move' because cyrus complains about bad headers. I'm not looking to 'dummy up' cyrus' input routines. But I'd be willing to entertain patching cyrus to get it 'throw away' headers it doesn't like... When I check the source on the messages there are indeed bad headers. I think they got there by using hotwayd to POP the messages from hotmail. Pulled via fetchmail and delivered via dmail, but that's largely irrelevant at this point. They're 'in there' and they're defective but how can I skip over them? I want to move all the good ones off the uw-imap server over to cyrus. Once they good ones are moved I can try to figure out if the bad ones are worth putting any effort into handling. I'd even be willing to consider something that 'fixed' the mail on the uw-imap server first. Anyone know of decent tools for dealing with this sort of problem? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: moving mail from uw to cyrus, damaged headers?
try imapsync Tried it, no go, same problem: Couldn't append msg #6 (Subject:[Re: How to specify?]) to folder tt: Error trying to append: 111 NO Message contains invalid header (Basically it's telling me cyrus won't take the bogus message I put in the 'tt' folder) Likewise various other IMAP syncing tools upchuck on it. Cyrus just refuses (as it seems it should) to take the bogus header. So I suppose what I'm after is a 'mailbox cleaner-upper' for those corrupted messages. I've taken a stab at what's causing the trouble and it's lines with the hex chars \x13\x42 on them. At least that /seems/ to be the most commonly present combination of characters on the corrupted header line. It's basically some gibberish put on a line near the very start of a message (usually on a line somewhere between the 2nd and 8th in the header of the message). A bit of grep-fu digs them out: cd ~/mail grep -P '\x13\x42' -l -R * Fortunately it's only about 30 mailboxes (of some 300 or so) that are affected and they're (thanfully) all in mbox text files. If I can truly narrow down what characters are causing the trouble then the various tools should be able to function properly But is there a way to tell cyrus to do the dirty work of cleaning up the headers as a message is copied into it? -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cm non-INBOX folder: Permission denied
I have set up a normal user named tester, with a home directory. I would like cyrus-imapd to store the users mail inside of the users home directory. But, IIRC, that's not where cyrus stores mail. Cyrus uses it's own 'partitions' to store all mail, not the shell home directory. Thus trying to make folders in the ~/mail directory won't get you anywhere. However whenever I try to create a non-inbox folder I get the error. Here is an example using cyradm localhost cm try201 cm try201 createmailbox: Permission denied How about cm INBOX.try201 instead? # ls -l /home/tester drwxrwxrwx 2 cyrus mail 1024 Nov 11 23:20 mail Since cyrus ignores this that won't make a difference. At one time I was having an authentication problem with tester. The user tester seamed to have one password with saslpasswd2 and a different one with passwd, but I changed them both so that they are the same now. Cyrus doesn't care what the shell account might be using. Cyrus accounts are totally separate from the shell accounts. But then again, I'm not exactly a cyrus expert so feel free to correct me. -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
refusing bad mail based on headers?
Hi all, I'm trying to migrate a bunch of mail from a uw-imap server over to cyrus. Cyrus apparently dislikes some malformed headers some of the messages possess. Apparently a proxying script I'd been using had been randomly jamming some garbage into the X-Message-Status header and the uw-imap dmail and imapd daemon didn't care about it. Cyrus seems to care and refuses to accept the message. Trouble is this causes a fair bit of headaches with the mailutil, imapsync, imapcp, imapsync and probably a number of other utilities. They seem to want to transfer the mailbox in it's entirety and thus none get copied. Is there a way to a) have cyrus clean up the headers or b) ignore then and perhaps somehow flag them for later clean up? Thanks, -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
using imapsync and altnamespace?
Anyone out there using the altnamespace along with imapsync? I'm trying to move mail from a uw-imap server over to cyrus. imapsync (which doesn't appear to have it's own mailing list?) can preform a regex transform on the target but doesn't appear to be able to drop the leading seperator character. --regextrans2 's/^mail//' It ends up trying to do: From Folder [mail/somefolder] To Folder [.somefolder] How do I cajole it into dropping that leading period? What's the magic regex-fu needed? And I'm using the altnamespace so I can have sibling folders in the mailbox instead of stuff everything inside the Inbox. I could live with turning it off, doing the sync and then turning it back on again but would rather get it right instead. Thanks, -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: using imapsync and altnamespace?
How do I cajole it into dropping that leading period? What's the magic regex-fu needed? I think I managed to stumble across it on my own: --regextrans2 's/^mail\.//' -Bill Kearney Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
