Re: what is the best way to migrate
Rudy Gevaert wrote: Timo Schoeler wrote: http://www.linux-france.org/prj/imapsync/ helped me several times. it's an awesome tool imho ;) I'm still looking for a way to do the sync without knowing the password of the user. Any idea's? I've used a dummy ldap backend with a copy of all our accounts.. but with the same known password. You are down during the migration.. but it's fairly easy to setup and cutover. Jared Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: New kid on the block: Zimbra?
[EMAIL PROTECTED] wrote: So, anybody here notice http://www.zimbra.com yet? I have been talking to this company since last november.. and I'm in the process of doing an eval. I have an enterprise cyrus/sqm install with about 2500 accounts. I can't yet speak for speed... as the performance tests will come a little later... but in my environment it's not just about raw speed. This is a very cool combination of software and it supports (or will support) HA and load balanced clustering with gfs and all sorts of plugins for linking the content of messages out to external data.. like it will recognize a ups tracking number and you can hover over it in webmail and it tells you the status in a little mouseover popup. You can also link to internal apps.. like being able to right click on a company PO number and check status.. approve/decline etc. This has the potential to be much more than just another mail server. Jared Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Imap timeout with 27k messages...
Hello all.. I have a situation here where an 'exempt' user has accumulated nearly 27k messages and 1.5G of mail in their sent items folder and now any attempt to access this folder has imap timeout problems and stuck processes. The cyradm utility is also not able to work with the folder... attempts to rename.. or reconstruct it results in a stuck process. This under RHEL3 with reiserfs and cyrus 2.2.3.. I've reviewed the changelog and I don't see anything obviously related to this up to 2.2.12. Any ideas on what might be causing this... or what I can do to fix it... short of deleting the folder? Thanks, Jared --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Imap timeout with 27k messages...
Ken Murchison wrote: Jared Watkins wrote: Hello all.. I have a situation here where an 'exempt' user has accumulated nearly 27k messages and 1.5G of mail in their sent items folder and now any attempt to access this folder has imap timeout problems and stuck processes. The cyradm utility is also not able to work with the folder... attempts to rename.. or reconstruct it results in a stuck process. This under RHEL3 with reiserfs and cyrus 2.2.3.. I've reviewed the changelog and I don't see anything obviously related to this up to 2.2.12. Any ideas on what might be causing this... or what I can do to fix it... short of deleting the folder? Are you sure that reconstruct gets stuck or are you just not waiting long enough for it to finish? Cyrus has no problem with mailboxes of this size. The info-cyrus archive mailbox at CMU (which I read the list from) currently has 36K messages in it. Well... I've run reconstruct on large mailboxes before... and when things are normal.. I can look at a top listing and see reconstruct at the top the list.. using lots of resources while it runs. In this case... it's not using any resources.. it shows no evidence that it's doing anything.. and when I ctrl-c it... it exits immediately with no error messages. What could cause reconstruct to act this way? I may try moving some stuff around by hand... as DL suggested... but I'm not crazy about that approach since I don't know the cause of the problem. Part of me just wants to solve it the BOFH way to encourage the user to keep better control over their stuff... but for now I can't do that. Thanks, Jared --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: backup without stopping the imap server?
John Madden wrote: I'm using LVM snapshot on linux box and it work perfectly But a filesystem-level snapshot isn't a clear copy of what's uncommitted to the DB's. I still haven't heard how bad a situation it is if the db's in the 'db' directory are corrupted -- what do you do then? AFAIK the only really important data that can't be easily replaced is the mailbox list database. So I do regular dumps of that file and keep the last several on hand in plain text format. The database indexes in each user folder can be rebuilt with the reconstruct command if there are any corruption problems... Remember.. the per user 'databases' are not storing the entire message.. only some cached metadata to make response times better for the client. Also.. there are two automatic backups of the important stuff from /var/imap if you had to use one of those. Jared --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: backup without stopping the imap server?
Andreas Hasenack wrote: On Mon, Jun 13, 2005 at 12:35:25PM -0400, Jared Watkins wrote: AFAIK the only really important data that can't be easily replaced is the mailbox list database. So I do regular dumps of that file and keep What exactly is this? Really just the names of all the folders and user.USER mailboxes? Can't this be reconstructed by just inspecting the /var/spool/imap directory hierarchy where all mailboxes reside? What is the black magic here that prevents this from being reconstructed? It depends... if your setup is simple enough.. yes you could write a script to scan your folders and rewrite the text dump format of this file. It also contains what imap partition the folder is on.. the owner.. and all permissions on the folder. If you use a lot of shared mailboxes.. there is no other way to recover the permissions info that I know of. Yes you could get by without it.. maybe.. but notice I didn't say it was impossible to rebuild this data.. only that it wasn't easily replaced. Jared --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus/OpenLDAP Administration SQL
jon johnston wrote: I'm very new to Cyrus. I am looking for a method by which someone with no linux skills can manage user accounts in Cyrus, and preferably, OpenLDAP simultaneously. Any input here is greatly appreciated. Thanks I have an admin tool in progress that does exactly that... I've gotten stalled with it lately as my company is switching from Iplanet ldap to active directory... but I've tested it with openldap in the past. It's in CVS right now... and the docs are not complete... once it's configured though it's pretty easy to use. The helpdesk and other admins where I work use it to manage about 2500 accounts. http://sourceforge.net/projects/ldapcyradm/ You can see some screen shots of it here... http://snowcrash.homeip.net/ldapcyradm/ Jared --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Hardware RAID Level Performance
Andrew Morgan wrote: You may want to look into Dell's AX100 SAN (a rebranded version of the EMC Clariion AX100). These use SATA drives with a FC front end. They are relatively inexpensive for the amount of storage you can get, if your I/O needs match. You can also go a little more upscale with the CX300/500/700 models which support a mix of FC and SATA hard drives and offer greater expandability. Whether these solutions are appropriate for storing mail is left as an exercise for the reader... :) I've had the chance to test about a dozen different storage systems... FC and ATA... I have tried to run CX200 and 300s in production with a mix of FC and ATA drives.. using the ata for simple file server space.. and let me just say.. don't go there. The emc ata performance was so bad.. after 4 months of them tinkering with it we eventually sent it back for all FC drives. Their ata systems couldn't match any of the other ata disk arrays I've tested. One good but lesser known company is http://www.technomagesinc.com/ I have about 8TB worth of their ata disk in production... with FC and U160 connectivity. The boxes are all off the shelf and proven hardware.. nothing exotic and proprietary.. and they run embedded linux. Very straight forward systems that just work.. and very good support. Jared --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Invalid flag in Append command
Hello all...
I'm trying to migrate about 2400 users from iplanet 4x to cyrus 2.2.3...
my first batch of test accounts went without a problem... but now I've
hit a snag with the following in the telemetry output...
APPEND INBOX (\Recent) {somenumber}
BAD Invalid flag in Append command
I looked in imapd.c and found the following function which checks for
known flags...
static int isokflag(char *s)
{
if (s[0] == '\\') {
lcase(s);
if (!strcmp(s, \\seen)) return 1;
if (!strcmp(s, \\answered)) return 1;
if (!strcmp(s, \\flagged)) return 1;
if (!strcmp(s, \\draft)) return 1;
if (!strcmp(s, \\deleted)) return 1;
/* uh oh, system flag i don't recognize */
return 0;
} else {
/* valid user flag? */
return imparse_isatom(s);
}
}
I'm guessing it's not as simple as adding Recent as a known type... but
what's the story on this... and is there an easy way around it? I
couldn't find anything relevant in the archives.
Thanks,
Jared
---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Invalid flag in Append command
Cyrus Daboo wrote:
Bottom line is: its a client bug - fix the client.
In this case.. the client is the imapmigrate tool from the Cyrus IMAPd
Utilities project on sourceforge.
Just to have this in the archive... around line 514 I've added a line
that should ignore any Recent flags...
for $msg ($oldimap-search(ALL)) {
my $msgtext = $oldimap-message_string($msg);
my $flags = $oldimap-flags($msg);
my $flg = ;
for (@flags) {
if ($_ eq \\Recent) { next; } # Added line
$flg .= $_
}
chomp $flg;
$newimap-append_string($newfolder, $msgtext, $flg);
}
That seems to take care of it...
Jared
---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAP load testing tool?
Has anyone found or written a tool to load test an IMAP server (Cyrus in particular) which simulates client (reader) traffic? I have used this one by eTesting labs with some success... it's a pita to setup... but it does work well. http://www.veritest.com/benchmarks/svrtools/email/t1intro.asp?visitor=X Jared
Re: Management tool?
Did you ever get an answer to your question? I'm working on just such a tool.. but with an ldap backend. So far.. it supports working with static/dynamic groups.. public folders.. and even alternate cyrus partitions.. I have scripts that pull info out of ldap each hour and write out postfix format files for the heavily used info. Jared Hi All Is there a tool available (perl or php) that will allow my users to manage their email accounts? I have tried web-cyadmin but did not get it working( is there a easy to follow doc for this? Non academic in style please or rtfm comments ;-) been there, done that,got nowhere) I ask this as manually having to addusers, saslpasswd2 there password, edit postfix files, postmap postfix for every adjustment is at the least tedious. I must say though that both apps work wonderfully ;-) Just ease of management is lacking. Any help or guidance is very much appreciated. Dave C --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.471 / Virus Database: 269 - Release Date: 10/04/2003
Pausing Cyrus for Snapshots?
I've not been able to find the answer to this in the docs or mailing list... I'd like to use the snapshot feature of the linux volume manager to take block level snaps of the cyrus databases and mailstore at regular intervals. To ensure consistency of the databases I assume you need to shut down cyrus... but I was looking for a way to do this without disconnecting imap and pop clients... a way to suspend operations.. flush the databases and sync the disks before taking a snap. Thanks, Jared -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Re: MUPDATE master server
No kidding. I'm looking forward to the donation that makes developing that possible. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper Any idea what it would take to make this happen... ballpark? Jared - -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
cyrdump and cyrrestore?
I was just curious about the status of thse two tools... they look very useful... I tried compiling the cyrrestore in 2.1.11 but had no luck... and I see nothing in the archives about these... Much thanks, jared -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Re: SNMP cyrus monitoring.
Christian Schulte wrote: Jared Watkins schrieb: Hmmm... Well I ran the snmpconf perl script and allowed public read access in snmpd.conf.. but when I try to walk the enterprises tree I don't see any cyrus stuff there... in fact.. I don't get any results if I use enterprises.anything. Do I need to give the cyrus user permission to write to snmp somehow? Is there a config option in cyrus.conf that is not there by default? Cyrus is not logging anything about attempting to connect to snmp and snmpd is not logging any errors... I don't know what to look for here... can anyone shed some light? How do the cyrus stats become known to snmpd? I'm almost snmp clueless... Jared By the way: Did you copy master/CYRUS-MASTER.mib to /usr/local/share/snmp/mibs ? Ahh... no I had missed that file... but.. in the mean time I also discovered that cyrus was not picking up the snmp libs when I compiled it... even though they were there and I'd specified it on the configure line. So I updated the snmp libs.. with a src rpm and then cyrus saw the libs.. but would not compile. I had a similar error to that other thread... I'm on redhat 7.2 here. That's as far as I got with it so far.. I'm loosing interest in snmp... =[ If I see some answers in the compile problem thread I'll try them... otherwise I'm done with this for now. jared -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Re: SNMP cyrus monitoring.
I don't know a lot about the innards of snmp... but here is my problem. I configured cyrus with the --with-ucdsnmp flag.. but when I query with snmpwalk it gives no result.. as if that part of the tree is not known to snmp. I ran the query using the base OID from your scripts. I'm doing this on my test RH 7.3 box with the rpm packages of snmp. What else has to be done to get this working for cyrus? Jared Scott Adkins wrote: --On Thursday, January 23, 2003 5:58 PM +0600 Dmitry Novosjolov [EMAIL PROTECTED] wrote: Hi All, has anybody succeded in using SNMP statistics of cyrus IMAP server ? If so, can you please point me in right direction of how to monitor the activity of Cyrus-imapd-2.1.11 server? I've heard about togowar, but cyrus documents are empty in this chapter ... -- Best regards, Novosjolov Dmitry I actually wrote a couple scripts that monitor the server. One script is just meant to be called from the command line (snmp_query) and displays the results in a clean easy to understand format. The sample output is as follows: Cyrus IMAP Server v2.0.16 Thu Jan 23 09:10:55 EST 2003 Up 5 days, 0:36:29 Services Forks Running Maximum --- -- -- -- imap 204631 48 pop3 11582 11 imaps14181 16722127 pop3s392223 72 imaps_silky 355222 62 lmtp 135032 113 lmtpunix 150 0 1 === == == == Total26359 17822434 NOTES --- Forks = Total number of forks since server was started. Running = Total number of processes currently running. Maximum = Maximum number of processes running concurrently. The other script (check_cyrus) is for logging and I use a cron job that runs it periodically and appends the output to a log file. The output of that script looks like the following (all on one line, though): 01/23 09:11 5 days, 0:37:08 imap=31/48 pop3=2/11 imaps=1672/2127 pop3s=23/72 imaps_silky=22/62 lmtp=32/113 lmtpunix=0/1 The above data should be easy enough to parse so that you can push it through a grapher, such as gnuplot or maybe even excel. I will attach both scripts. Maybe if the CMU folks like it, they can put it in the contrib directory? Oh, I actually just called the snmpwalk program directly and didn't use the SNMP perl module. Maybe somebody else can retrofit the script to do that. Anyawys, it should require just minor tweaking to get it to work on your system. Scott -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Re: SNMP cyrus monitoring.
Hmmm... Well I ran the snmpconf perl script and allowed public read access in snmpd.conf.. but when I try to walk the enterprises tree I don't see any cyrus stuff there... in fact.. I don't get any results if I use enterprises.anything. Do I need to give the cyrus user permission to write to snmp somehow? Is there a config option in cyrus.conf that is not there by default? Cyrus is not logging anything about attempting to connect to snmp and snmpd is not logging any errors... I don't know what to look for here... can anyone shed some light? How do the cyrus stats become known to snmpd? I'm almost snmp clueless... Jared Scott Adkins wrote: Taking a stab in the dark, do you have an SNMP server running on the machine? If you have the snmp tools installed, then check chkconfig with the following command to see if you got the service configured to start at boot time. Before you turn on the snmp service via the /etc/init.d scripts, make sure that you have it configured properly (I am thinking in terms of access control). Anyways, that is my stab in the dark guess ;) Oh, you will have to restart your cyrus server after you start the snmp server, that way, cyrus will initialize properly with the snmp server and start sending it stats. Scott --On Thursday, January 23, 2003 12:36 PM -0500 Jared Watkins [EMAIL PROTECTED] wrote: I don't know a lot about the innards of snmp... but here is my problem. I configured cyrus with the --with-ucdsnmp flag.. but when I query with snmpwalk it gives no result.. as if that part of the tree is not known to snmp. I ran the query using the base OID from your scripts. I'm doing this on my test RH 7.3 box with the rpm packages of snmp. What else has to be done to get this working for cyrus? Jared Scott Adkins wrote: --On Thursday, January 23, 2003 5:58 PM +0600 Dmitry Novosjolov [EMAIL PROTECTED] wrote: Hi All, has anybody succeded in using SNMP statistics of cyrus IMAP server ? If so, can you please point me in right direction of how to monitor the activity of Cyrus-imapd-2.1.11 server? I've heard about togowar, but cyrus documents are empty in this chapter ... -- Best regards, Novosjolov Dmitry I actually wrote a couple scripts that monitor the server. One script is just meant to be called from the command line (snmp_query) and displays the results in a clean easy to understand format. The sample output is as follows: Cyrus IMAP Server v2.0.16 Thu Jan 23 09:10:55 EST 2003 Up 5 days, 0:36:29 Services Forks Running Maximum --- -- -- -- imap 204631 48 pop3 11582 11 imaps14181 16722127 pop3s392223 72 imaps_silky 355222 62 lmtp 135032 113 lmtpunix 150 0 1 === == == == Total26359 17822434 NOTES --- Forks = Total number of forks since server was started. Running = Total number of processes currently running. Maximum = Maximum number of processes running concurrently. The other script (check_cyrus) is for logging and I use a cron job that runs it periodically and appends the output to a log file. The output of that script looks like the following (all on one line, though): 01/23 09:11 5 days, 0:37:08 imap=31/48 pop3=2/11 imaps=1672/2127 pop3s=23/72 imaps_silky=22/62 lmtp=32/113 lmtpunix=0/1 The above data should be easy enough to parse so that you can push it through a grapher, such as gnuplot or maybe even excel. I will attach both scripts. Maybe if the CMU folks like it, they can put it in the contrib directory? Oh, I actually just called the snmpwalk program directly and didn't use the SNMP perl module. Maybe somebody else can retrofit the script to do that. Anyawys, it should require just minor tweaking to get it to work on your system. Scott -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK-- -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Latest on Cyrus and GFS?
I've not been able to find the answer to the GFS question in the archives.. so here goes. If one were using a dual attach scsi cabinet.. or fibre channel.. would it possible to use cyrus on GFS from two or more servers all with r/w access? I'm thinking of a setup that would have all user mailboxes visible to all cyrus servers.. but clients would be directed to different systems with perdition to balance the load. So if one server needed to come down.. you would simply take that system out of the pool and direct the users hitting it to the remaining cyrus servers.. increasing their load... but leaving the clients none the wiser. Jared -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Migration Pointers?
Hello all... I'm working on a migration from a single linux Iplanet (aka Netscape aka Sun One) messaging server with Iplanet ldap backend for the user database TO cyrus on linux with an active directory backend. I have about 2500 accounts on this sytem.. and from the mta logs I go through about 100k messages and 5GB a day of mail volume usually with delivery peaks of no more than 300 messages a minute. Most users get their mail via pop.. but we will encourage more of them to use imap along with sieve. We use a wide range of client apps.. all versions of outlook.. outlook express.. eudora.. netscape.. mozilla.. act.. and some others. (This has me most worried) We also require the use of SSL/TLS for any logins from outside our lan. I'm planning to deliver mail via lmtp from our postfix relays... letting them deal with virtual user/domain issues through ldap lookups... we have only one namespace with the first initial last name style of usernames. I'm already well on my way to finishing some web based cyrus/AD admin tools.. and AD migration scripts... Hardware wise... it will be dell on either a dual or quad xenon system.. with 2G of ram.. and either an 7 disk local raid array or a FC attached SAN. For the actual data migration... I have a simple plan. Since most people use pop.. I'll just cut mail delivery over to the new system.. and provide a page on our intranet for people to move their old mail from one server to the other.. this way I already have their password since they are logged into our intranet. Has anyone done a migration like this before? I'm looking for some gotchas or left field sort of problems I may encounter... I've also read some about performance and recovery best practices.. but I'm sure there is more I could learn from all you fine people. I hope to catch most of the problems through my testing process.. but you never really know what you are going to have problems with until it's in the wild of production. Thanks in advance... Jared -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Re: backup mail server
How long have you been using this setup... and have you ever had any problem... or needed to do a failover.. or a failback? Do you dedicate nics for the replication.. are they GB? Do you have any stats on the typical data rates for the replication under normal load? I've thought about using this setup as a poor mans san for a simple hot spare... but I've not known of anyone actually doing it in production.. so I'm interested to know your experiences. I'll assume you are running on hardware raid.. and doing backups... but what about using the LVM and snapshots? Much thanks you brave soul... Jared Lee wrote: - Are you using other tools like heartbeat or in the same kind ? If yes which tool ? Yes, we're using hearbeat. Heres the requisite config: /etc/ha.d/haresources: servname.host.com 100.102.248.46 datadisk::drbd0 cyrus postfix - From your drbd configuration file I can see that you are using /dev/sda6 as physical disk, is that your Cyrus partition (/var/spool/imap) ? sda6 is our data partitiion where we keep /var/spool/imap /var/imap/ /var/spool/mail and all of our configuration files. L -- -BEGIN GEEK CODE BLOCK- Version: 3.12 GIT/S/B d- s-:+ a- C$ UL$ P--- L+++$ E--- W+++ N++ o+ K- w O- M-- !V PS+ PE Y++ PGP++ t+ 5- X+ R* tv+ b++ DI+ D G e+ h+ r+++ z* --END GEEK CODE BLOCK--
Re: OT LVM Snapshot for backups (was: Best way to backup cyrus syste)
Christian Schulte wrote: Jared Watkins wrote: I would think you could use LVM the linux volume manager along with it's snapshot feature.. and then any sort of backup program you want. I'm not using this method yet... but I will be soon... is anyone else out there running LVM or some other volume manager under linux in production? Jared I have had many many troubles with the linux LVM itself. I would rather suggest using a well configured linux softare raid. This is much more stable than LVM! I lost 250GB because of LVM and did not have a backup. With software raid on linux this would not have happened That's a little vague... How long ago was this.. and what was going on when you had the failure? Was it an LVM failure.. or something related.. like a non redundant dead drive.. or file system issue? Software raid is not exactly a replacement for LVM.. espicially when talking about the snapshot feature..
Re: OT LVM Snapshot for backups (was: Best way to backup cyrus syste)
I would think you could use LVM the linux volume manager along with it's snapshot feature.. and then any sort of backup program you want. I'm not using this method yet... but I will be soon... is anyone else out there running LVM or some other volume manager under linux in production? Jared [EMAIL PROTECTED] wrote: Dear all, I would like to know the best way of backup/restore cyrus system. What are the steps required? Are there online backup methods available? What are the files required to backup and how can I restore it on the same machine/ another machine? Many thanks! Boris --- PLEASE READ: The information contained in this e-mail is confidential and intended for the named recipient(s) only. If you are not an intended recipient of this e-mail you must not copy, distribute or take any further action in reliance upon it and you should delete it and notify the sender immediately. E-mail is not a secure method of communication. GuruBase Technology Limited cannot accept responsibility for the accuracy or completeness of this message or any attachment(s). This transmission could contain viruses, be corrupted, destroyed, incomplete, intercepted, lost or arrive late. If verification of this e-mail is sought then please request a hard copy. Unless otherwise stated any views or opinions presented are solely those of the author and do not represent those of GuruBase Technology Limited. This e-mail is intended for information purposes only.
Re: Murder / LDAP / SASL Problem... END
Just to give this thread some closure... I've abandoned the effort to get this to work for now... Since I can always add it on later... I'll just wait a bit until these issues are shaken out a little more... I would think that in a large environment where a murder might be used... it will be common to use an LDAP backend... so I'll be lurking.. seeing what others come up with. In the mean time... I'll be putting in a virtualized san.. with the ability to do local mirrors and long distance replication... making the boot disk and data 'portable' and that should be enough to cover for single machine hardware failures.. and there is always per user transport ldap lookups in postfix for multiple cyrus stores... that should cover my bases for the next year or two. Thanks for the input... jared Rob Siemborski wrote: On Thu, 31 Oct 2002, Jared Watkins wrote: Do you have a copy of the entire log I could look at (since you've already sent the passwords to a public list, I'm guessing you don't really care about them any more)? The only log entry I get on the backend.. even with CYRUS_VERBOSE turned up... is this: Oct 31 11:30:53 is8000new imapd[19749] badlogin: [10.10.100.42] PLAIN [SASL (-4): no mechanism available: security flags do not match required] The tcpdump log for this action follows... ignore the differences in time stamps.. This is all I've been able to go by for logs... if there is some way of getting more detailed logs from cyrus.. let me know and I'll try that. Oh and no I'm not concerned with passwords.. these are all test systems on a private network. Is imtest selecting PLAIN as its mechanism? I have a feeling you're getting screwed because in general you can't use PLAIN without an external security layer (e.g. TLS) present. One thing you can try is removing the backend1_mechs line from your frontend's imapd.conf, and see if that makes it do the same thing that imtest is doing. I'm not sure what the correct approach is in your situation with currently-written code, since you really want to be using DIGEST-MD5 or another challenge-response mechanism that supports proxying to authenticate to the backends, but you need to keep the full user database in LDAP (and the full user list needs to be able to authenticate to the backends, as referrals are always a possibility). It may be worthwhile to look into the LDAP auxprop patch to make this possible (or you can try having a sasldb2 with just the frontend's id's in it, and fall back to PLAIN for the rest, but this requires clients to not get upset when authenticating via DIGEST-MD5 fails). The bad way to fix this is to change this line in imapd.c: secprops = mysasl_secprops(SASL_SEC_NOPLAINTEXT); to: secprops = mysasl_secprops(0); (it occurs twice) -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: Murder / LDAP / SASL Problem...
Using imtest I was able to login as you suggested... and sniffing the traffic I saw that it did an ldap lookup to verify the 'murder' proxy user... but why is not doing the ldap search when the auth attempt is made from the frontend server? If you see my original email... I don't have a problem until I try to select the inbox for the test account... that's when I get the error... and no ldap lookups are taking place from the backend system. imtest -u testuser -a slaveuser backend.your.dom Also.. in your example line should that be '-a proxyuser' instead of slaveuser? It was my understanding that the 'slaveuser' was only used in communicating with the mupdate master... and the proxy_authname user was used in the connection to the backend. jared Rob Siemborski wrote: On Wed, 30 Oct 2002, Jared Watkins wrote: What's not working: Although I'm able to authenticate with a test account to the front end system... I am not able to select the inbox. When I try to select the inbox there is a pause of around 5 seconds then I see the following errors: IMAP: NO Server(s) unavailable to complete operation Frontend: login: localhost.localdomain[127.0.0.1] test1 plaintext Frontend: couldn't authenticate to backend server: authentication failure Backend: badlogin: [ip of frontend] PLAIN [SASL (-4): no mechanism available: security flags do not match required] When this happens... I know from sniffing the network that neither front or back system is doing an ldap lookup to verify the proxy users password... so I assume that's why it is failing... it has nothing to verify the proxy_authname against. This isn't what is being indicated by the logs and the behavior you suggest. If you can authenticate to the frontend as the test user, then the frontend is happy that the test user is a-ok. No authentications to the backend happen until you select a mailbox. Have you tried doing something like: imtest -u testuser -a slaveuser backend.your.dom and seeing if you can proxy authenticate that way? -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: Murder / LDAP / SASL Problem...
Rob Siemborski wrote: On Thu, 31 Oct 2002, Jared Watkins wrote: Using imtest I was able to login as you suggested... and sniffing the traffic I saw that it did an ldap lookup to verify the 'murder' proxy user... but why is not doing the ldap search when the auth attempt is made from the frontend server? If you see my original email... I don't have a problem until I try to select the inbox for the test account... that's when I get the error... and no ldap lookups are taking place from the backend system. I don't know why there isn't a lookup from the frontend. What mechanism is it trying to use (since it's obviously succeeding). I'm guessing there's something strange about your configuration on the frontend, but I'm really worried by the fact that it seems to be working. When I login to port 143 using telnet to the _frontend_ that system will do an ldap lookup and verify the password of the test user. I am able to list the mailboxes (since they are being provided by mupdate) but when I attempt to select a mailbox... it fails. When it fails... I can see the front end attempting to connect to the backend over imap.. but the backend system replies with 'NO Error authenticating'I also noticed, while using tcpdump, that the backend system is sending this error message before the frontend can supply a username... It almost seems like the back end system does not like something about the greeting message from the front... Now... if I do the exact same thing.. (login with telnet to port 143 with my test user) on the backend system... everything works normally... it will do an ldap lookup to verify the password... and I'm able to select mailboxes... imtest -u testuser -a slaveuser backend.your.dom What mechanism does it decide to use? I am using ldap for all my authentications... or is that not what you mean? For now I have not compiled any strong mechs.. so everything should be plain text to keep things simple for testing. Jared
Re: Murder / LDAP / SASL Problem...
Rob Siemborski wrote:
On Thu, 31 Oct 2002, Jared Watkins wrote:
When I login to port 143 using telnet to the _frontend_ that system will
do an ldap lookup and verify the password of the test user. I am able to
list the mailboxes (since they are being provided by mupdate) but when I
attempt to select a mailbox... it fails. When it fails... I can see
the front end attempting to connect to the backend over imap.. but the
backend system replies with 'NO Error authenticating'I also noticed,
while using tcpdump, that the backend system is sending this error
message before the frontend can supply a username...
Do you have a copy of the entire log I could look at (since you've
already sent the passwords to a public list, I'm guessing you don't
really care about them any more)?
The only log entry I get on the backend.. even with CYRUS_VERBOSE turned
up... is this:
Oct 31 11:30:53 is8000new imapd[19749] badlogin: [10.10.100.42] PLAIN
[SASL (-4): no mechanism available: security flags do not match required]
The tcpdump log for this action follows... ignore the differences in
time stamps.. This is all I've been able to go by for logs... if there
is some way of getting more detailed logs from cyrus.. let me know and
I'll try that. Oh and no I'm not concerned with passwords.. these are
all test systems on a private network.
11:47:09.908689 darklord.amcity.com.57464 is8000new.amcity.com.imap: S
2130791435:2130791435(0) win 5840 mss 1460
,sackOK,timestamp 656477180 0,nop,wscale 0 (DF)
0x 4500 003c 47f7 4000 4006 b23a 0a0a 642aE..G:..d*
0x0010 0a0a c84c e078 008f 7f01 4c0b ...L.xL.
0x0020 a002 16d0 117b 0204 05b4 0402 080a.{..
0x0030 2721 0bfc 0103 0300 '!..
11:47:09.909046 is8000new.amcity.com.imap darklord.amcity.com.57464: S
3574424286:3574424286(0) ack 2130791436 win
5792 mss 1460,sackOK,timestamp 59580368 656477180,nop,wscale 0 (DF)
0x 4500 003c 4000 4006 fa31 0a0a c84cE...1...L
0x0010 0a0a 642a 008f e078 d50d 62de 7f01 4c0c..d*...x..b...L.
0x0020 a012 16a0 b650 0204 05b4 0402 080a.P..
0x0030 038d 1fd0 2721 0bfc 0103 0300 '!..
11:47:09.909106 darklord.amcity.com.57464 is8000new.amcity.com.imap: .
ack 1 win 5840 nop,nop,timestamp 656477180
59580368 (DF)
0x 4500 0034 47f8 4000 4006 b241 0a0a 642aE..4GA..d*
0x0010 0a0a c84c e078 008f 7f01 4c0c d50d 62df...L.xL...b.
0x0020 8010 16d0 e4e5 0101 080a 2721 0bfc'!..
0x0030 038d 1fd0
11:47:11.911213 is8000new.amcity.com.imap darklord.amcity.com.57464: P
1:60(59) ack 1 win 5792 nop,nop,timestamp
59580568 656477180 (DF)
0x 4500 006f 7d0f 4000 4006 7cef 0a0a c84cE..o}...|L
0x0010 0a0a 642a 008f e078 d50d 62df 7f01 4c0c..d*...x..b...L.
0x0020 8018 16a0 4244 0101 080a 038d 2098BD..
0x0030 2721 0bfc 2a20 4f4b 2069 7338 3030 306e'!..*.OK.is8000n
0x0040 6577 2e61 6d63 6974 792e 636f 6d20 4379ew.amcity.com.Cy
0x0050 7275 7320 494d 4150 3420 7632 2e31 2e39rus.IMAP4.v2.1.9
0x0060 2073 6572 7665 7220 7265 6164 790d 0a .server.ready..
11:47:11.911316 darklord.amcity.com.57464 is8000new.amcity.com.imap: .
ack 60 win 5840 nop,nop,timestamp 65647738
0 59580568 (DF)
0x 4500 0034 47f9 4000 4006 b240 0a0a 642aE..4G..d*
0x0010 0a0a c84c e078 008f 7f01 4c0c d50d 631a...L.xL...c.
0x0020 8010 16d0 e31a 0101 080a 2721 0cc4'!..
0x0030 038d 2098
11:47:11.911467 darklord.amcity.com.57464 is8000new.amcity.com.imap: P
1:25(24) ack 60 win 5840 nop,nop,timestamp
656477380 59580568 (DF)
0x 4500 004c 47fa 4000 4006 b227 0a0a 642aE..LG'..d*
0x0010 0a0a c84c e078 008f 7f01 4c0c d50d 631a...L.xL...c.
0x0020 8018 16d0 e704 0101 080a 2721 0cc4'!..
0x0030 038d 2098 4130 3120 4155 5448 454e 5449A01.AUTHENTI
0x0040 4341 5445 2050 4c41 494e 0d0a CATE.PLAIN..
11:47:11.911683 is8000new.amcity.com.imap darklord.amcity.com.57464: .
ack 25 win 5792 nop,nop,timestamp 59580568
656477380 (DF)
0x 4500 0034 7d10 4000 4006 7d29 0a0a c84cE..4}...})...L
0x0010 0a0a 642a 008f e078 d50d 631a 7f01 4c24..d*...x..c...L$
0x0020 8010 16a0 e332 0101 080a 038d 2098.2..
0x0030 2721 0cc4 '!..
11:47:14.920742 is8000new.amcity.com.imap darklord.amcity.com.57464: P
60:89(29) ack 25 win 5792 nop,nop,timestam
p 59580870 656477380 (DF)
0x 4500 0051 7d11 4000 4006 7d0b 0a0a c84cE..Q}...}L
0x0010 0a0a 642a 008f e078 d50d 631a 7f01 4c24..d*...x..c...L$
0x0020 8018 16a0 0d13 0101 080a 038d 21c6
Murder / LDAP / SASL Problem...
I'm trying to setup a murder for testing... I have two physical machines... one running a backend.. the other running the mupdate master and as a frontend. I'm using SASL 2.1.9 and cyrus 2.1.9 on both systems. My latest compile time options are as follows: SASL --with-openssl=/usr/lib --with-saslauthd --enable-krb4=no --with-ldap --disable-anon --disable-cram --disable-digest --disable-otp --enable-plain --enable-login --disable-srp --with-opie=no --with-gssapi=no IMAP --with-auth=unix --enable-fulldirhash --with-mboxlist-db=skiplist --with-dbdir=/usr/include/db3 --with-ucdsnmp --enable-murder --with-krb4=no --with-sasl=/usr/lib/sasl2 My backend system has the following in imapd.conf configdirectory: /var/imap partition-default: /var/spool/imap admins: cyrus sasl_pwcheck_method: saslauthd sasl_mech_list: plain allowplaintext: yes lmtp_allowplaintext: yes altnamespace: yes proxyservers: murder tls_cert_file: /var/imap/server.pem tls_key_file: /var/imap/server.pem mupdate_server: my front end system ip mupdate_password: murder mupdate_authname: mupdatebackend1 My front end system has the following imapd.conf configdirectory: /var/imap partition-default: /tmp admins: cyrus mupdatebackend1 slave1 sasl_pwcheck_method: saslauthd sasl_mech_list: plain allowplaintext: yes mupdate_server: localhost mupdate_port: 2004 mupdate_password: murder mupdate_authname: slave1 backend1_password: murder backend1_mechs: plain proxy_authname: murder I also have ldap entries for mupdatebackend1, slave1, murder, cyrus and my test accounts. What works: Before I started on murder.. I had a working mail system with a postifx mta (also using ldap) and a standalone cyrus using ldap to authenticate and accepting deliveries over lmtp from postfix. Now.. deliveries are still working to the backend system... mupdate is working... I am able to use cyradm as the cyrus user to create and delete mailboxes when connected to the backend system and using telnet... I am able to authenticate as one of my test accounts to port 143 to the front end system. What's not working: Although I'm able to authenticate with a test account to the front end system... I am not able to select the inbox. When I try to select the inbox there is a pause of around 5 seconds then I see the following errors: IMAP: NO Server(s) unavailable to complete operation Frontend: login: localhost.localdomain[127.0.0.1] test1 plaintext Frontend: couldn't authenticate to backend server: authentication failure Backend: badlogin: [ip of frontend] PLAIN [SASL (-4): no mechanism available: security flags do not match required] When this happens... I know from sniffing the network that neither front or back system is doing an ldap lookup to verify the proxy users password... so I assume that's why it is failing... it has nothing to verify the proxy_authname against. Any ideas on how to get this sorted out? Thanks, Jared
