Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
On 10/24/07, Ralph Angenendt [EMAIL PROTECTED] wrote: Matt Shields wrote: Because of the way that the infrastructure is (biz reasons) we are not doing shared storage, we have numerous IMAP servers that we distribute accounts across. As we add more users, we image up a new IMAP server. For our business's scaling purposes this was the best plan. What I am having a problem is how do I get postfix to transfer the email to the particular IMAP server that the user account is on. I know that I need to use lmtp and transport, but all the examples I have seen show forwarding all email to 1 IMAP server. I would like Postfix to do a lookup for each mailbox and determine which IMAP server to deliver it to. Having no idea how that fits into your already existing infrastructure, but the Cyrus IMAPD Aggregator (also known as Cyrus IMAPD Murder) looks like something which should be evaluated - you probably can even drop the mysql database, as it really doesn't matter to which of the lmtp/imapd proxies you connect to. From what I understand about Cyrus Murder, it is for replicating your user data across multiple servers, which is good if you want to load balance multiple IMAP servers and you don't have a shared storage backend. As mentioned we have a web frontend that checks mysql when the user logs in to see which imap server the account resides on. Everyone, I have figured it out. I do plan on posting after I finish documenting the steps (for those interested). -matt ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
On 10/23/07, Christopher Chan [EMAIL PROTECTED] wrote: Matt Shields wrote: Data changes too frequently to generate the file every x number of minutes across all smtp servers. You have to support instantly deliverable mailboxes for new accounts? Yes, don't ask me why, it's a business thing. The mysql db isn't a single server. It's a master (read/write) with multiple replicas for read access. Those replicas are load balanced with LVS (heartbeat/ldirectord/ipvsadm). The postfix(smtp) incoming and outgoing servers are also load balanced with LVS. So database read speed is not an issue. Believe me, we know how to build large high traffic sites, the only problem we're having is the exact syntax on using transport_maps or virtual_transport with multiple lmtp transports, and I think I got that figured out with the transport_maps. Will post more later. I assume that you are aware that transport_maps is called multiple times. Recipient_maps in rdbms tables generate at least two lookups (one for smtpd, one for cleanup) but when you add transport_maps, that will at least explode to one per subdomain of the sender address (you can mitigate a lot of that with the domain setting in the map configuration file) as trivial-rewrite tries to build its triples for addresses. ___ Yes, we're aware, that why we have mysql setup with multiple incoming and outgoing smtp servers that read from a large cluster of replicated mysql servers (read-only). Not saying we won't look at creating a cron to dump maps to a local file, we might do that in the future, but just for right now we have enough horsepower to deal with what we have. -matt ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
On Oct 24, 2007, at 1:38 PM, Craig White wrote: On Wed, 2007-10-24 at 21:21 +0800, Christopher Chan wrote: I thought the usual ways of doing this were to either use a high-performance NFS server (netapp filer...) and maildir format so you can run imap from any client facing server, or to keep the delivery host information in an LDAP attribute that you find when validating the address. This is the 'I have the money' way of doing this ;-) last I checked, openldap, postfix and cyrus-imapd were free. What is the money reference? cyrus-imapd doesn't use maildir but rather it's own methodology which is similar to maildir but keeps all the mail in it's own partition instead of users folders. It doesn't use system for quota management but has quota management built in. It seems much more sane and permits 'virtual users' which is/can be a virtue of ldap based accounts. I'm guessing that money referred to the netapp filer Tony S ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
On Wed, Oct 24, 2007 at 10:38:41AM -0700, Craig White wrote: On Wed, 2007-10-24 at 21:21 +0800, Christopher Chan wrote: I thought the usual ways of doing this were to either use a high-performance NFS server (netapp filer...) and maildir format so you can run imap from any client facing server, or to keep the delivery host information in an LDAP attribute that you find when validating the address. This is the 'I have the money' way of doing this ;-) last I checked, openldap, postfix and cyrus-imapd were free. What is the money reference? Last I checked, cyrus-imapd could not provide reliable service when the datastore was on NFS. -- /\oo/\ / /()\ \ David Mackintosh | [EMAIL PROTECTED] | http://www.xdroop.com pgpjRNEwkeH7o.pgp Description: PGP signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
Heck, I see lots of circles where they wouldn't trust mysql for an enterprise application so it seems clear that you are not talking about stability or performance but rather familiarity and the amount of trust you have in what you know. I would expect openldap to blow the doors off a mysql db but what do I know? I deal in circles 100 user accounts (small businesses). Wow it's amazing how off topic and how many opinions you get on a mailing list, when all you wanted to know was how do I specially do this or that. That's why I stated what my environment was. But, since numerous people have stated how mysql is inadequate to do what we want to do or in general for any task. We currently use mysql in a replicated environment with LVS to balance the connections for our main websites that is all dynamic. Last time I checked we were sustaining thousands of visitors per second 24 hours a day, which equaled about 3-4 thousand queries per second. So, if it can handle that load and Google trusts it in their infrastructure, then I'm not gonna replace it. It does what I need, it's reliable, it's fast and it has proven that it scales well. I think the main problem when people say you shouldn't use this product or that product because it's not good enough is they haven't set it up properly. They haven't taken the time to tune the server, the daemon, and the application. Let's face it anyone can write a query to a database (like select * from table) and if you put enough load behind it your performance is gonna suck no matter what your app or database is. But if you take time to tune your code and your database and design it so it can scale, you can efficiently use applications like mysql. Anyway, back to my original request. You can use the transport_maps feature to dynamically lookup lmtp transports on a per account basis. I have figured it out, and for those that are curious I will post when I've finished documenting everything. -matt ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
Matt Shields wrote: Anyway, back to my original request. You can use the transport_maps feature to dynamically lookup lmtp transports on a per account basis. I have figured it out, and for those that are curious I will post when I've finished documenting everything. I thought the ltmp transport was local by definition. How does that work when you need delivery to happen on a different host? -- Les Mikesell [EMAIL PROTECTED] ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
Les Mikesell wrote: Matt Shields wrote: Anyway, back to my original request. You can use the transport_maps feature to dynamically lookup lmtp transports on a per account basis. I have figured it out, and for those that are curious I will post when I've finished documenting everything. I thought the ltmp transport was local by definition. How does that work when you need delivery to happen on a different host? lmtp was to save having to queue the email on the delivery box and then deliver via a local lda. ltmp is kind of like using smtp to talk directly to the lda. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
Matt Shields wrote: I'm trying to set up a large scale email system that supports 100,000+ IMAP accounts. We have an existing frontend web interface that does a lookup on a mysql db to figure out which IMAP server to connect to for each user. For the email infrastructure we have decided on Postfix and Cyrus. We have configured both to use mysql to get the virtual user information. Because of the way that the infrastructure is (biz reasons) we are not doing shared storage, we have numerous IMAP servers that we distribute accounts across. As we add more users, we image up a new IMAP server. For our business's scaling purposes this was the best plan. What I am having a problem is how do I get postfix to transfer the email to the particular IMAP server that the user account is on. I know that I need to use lmtp and transport, but all the examples I have seen show forwarding all email to 1 IMAP server. I would like Postfix to do a lookup for each mailbox and determine which IMAP server to deliver it to. Anyone have a working example that they could share? It would be greatly appreciated. http://www.postfix.org/MYSQL_README.html Then you can create a view out of your existing data schema to fit the postfix needed schema. -Ross __ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
mouss wrote:
mouss wrote:
Matt Shields wrote:
I'm trying to set up a large scale email system that
supports 100,000+
IMAP accounts. We have an existing frontend web interface
that does a
lookup on a mysql db to figure out which IMAP server to
connect to for
each user. For the email infrastructure we have decided on Postfix
and Cyrus. We have configured both to use mysql to get the virtual
user information.
Because of the way that the infrastructure is (biz
reasons) we are not
doing shared storage, we have numerous IMAP servers that
we distribute
accounts across. As we add more users, we image up a new
IMAP server.
For our business's scaling purposes this was the best plan.
What I am having a problem is how do I get postfix to transfer the
email to the particular IMAP server that the user account is on. I
know that I need to use lmtp and transport, but all the examples I
have seen show forwarding all email to 1 IMAP server. I would like
Postfix to do a lookup for each mailbox and determine which IMAP
server to deliver it to.
There are primarily two ways:
[virtual aliase]
you can use virtual_alias_maps to redirect [EMAIL PROTECTED] to
[EMAIL PROTECTED], provided the final server accepts
such addresses.
If the final server doesn't accept these, and you use smtp
to relay to,
then you can write the addresses back, using smtp_generic_maps.
[transport]
an laternative is to use use (per-user) transport_maps.
something like
[EMAIL PROTECTED] relay:[hostN.example.com]
In bothe approaches, the mappings can be generated using
sql statements
(mostly CONCAT). something like
...
query = SELECT concat('relay:[', host, '.example.com]')
FROM User
where '%u' = user and '%d' = domain
you get the idea I hope.
True, it may be better to just have a cron job dump out
new static maps every 15 minutes or so then to have the
MTA query on every delivery especially for 100K accounts.
-Ross
__
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
On 10/23/07, mouss [EMAIL PROTECTED] wrote:
There are primarily two ways:
[virtual aliase]
you can use virtual_alias_maps to redirect [EMAIL PROTECTED] to
[EMAIL PROTECTED], provided the final server accepts such addresses.
If the final server doesn't accept these, and you use smtp to relay to,
then you can write the addresses back, using smtp_generic_maps.
[transport]
an laternative is to use use (per-user) transport_maps. something like
[EMAIL PROTECTED] relay:[hostN.example.com]
In bothe approaches, the mappings can be generated using sql statements
(mostly CONCAT). something like
...
query = SELECT concat('relay:[', host, '.example.com]')
FROM User
where '%u' = user and '%d' = domain
you get the idea I hope.
Anyone have a working example that they could share? It would be
greatly appreciated.
Forward's aren't acceptable. There is a way to do it with the
transport function and lmtp on a account by account basis. I'm
looking for real world configs from someone that has this working.
-matt
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
On Oct 23, 2007, at 12:28 PM, Matt Shields wrote: Forward's aren't acceptable. There is a way to do it with the transport function and lmtp on a account by account basis. I'm looking for real world configs from someone that has this working. Not condoning, but providing some links: http://middleware.internet2.edu/dir/docs/ldap-recipe.htm#E-MailRouting http://www.postfix.org/LDAP_README.html#example_virtual The transport function will tell you how to deliver to a particular server, but I'm not sure you are going to get the kind of efficiency you probably want thinking of the user account to server mapping as part of the transport functions, though suggestions have been made that will meet that way of thinking. Regardless what method you use to generate the maps, be it mysql, ldap or flat file, you will want the maps available to each edge host on the box themselves, so either storing copies of the flat files, a local copy of the mysql database or a local a local directory (none of them being the masters, more functioning like caching only name servers.) I'm partial to flat files for smaller maps and LDAP for larger ones, but there are arguments all the way around, some of which depend on local admin familiarity with whichever tech. Forward's aren't acceptable. There is a way to do it with the transport function and lmtp on a account by account basis. I'm looking for real world configs from someone that has this working. Depending on how you define forwards, it is not going to be possible for you to not have forwards, unless you have a large number of domains pointing directly at your delivery point servers and have only a certain number of domains per individual server. --Chris ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
Matt Shields wrote:
Data changes too frequently to generate the file every x number of
minutes across all smtp servers.
The mysql db isn't a single server. It's a master (read/write) with
multiple replicas for read access. Those replicas are load balanced
with LVS (heartbeat/ldirectord/ipvsadm). The postfix(smtp) incoming
and outgoing servers are also load balanced with LVS. So database
read speed is not an issue. Believe me, we know how to build large
high traffic sites, the only problem we're having is the exact syntax
on using transport_maps or virtual_transport with multiple lmtp
transports, and I think I got that figured out with the
transport_maps. Will post more later.
the syntax is simple, but depends on the structure of your tables.
transport_maps =
...
proxy:mysql:/etc/postfix/maps/mysql/transport
...
# cat /etc/postfix/maps/mysql/transport
hosts = 192.0.2.33 ...
user = youruser
password = yourpassword
dbname = yourdbname
query = select
concat('lmtp:', host)
from
yourtable
where
mailbox = '%s'
The above assumes a simple {`mailbox`, `host`} structure. you'll need to
adjust the sql query to your table structure.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users
Matt Shields wrote: Data changes too frequently to generate the file every x number of minutes across all smtp servers. You have to support instantly deliverable mailboxes for new accounts? The mysql db isn't a single server. It's a master (read/write) with multiple replicas for read access. Those replicas are load balanced with LVS (heartbeat/ldirectord/ipvsadm). The postfix(smtp) incoming and outgoing servers are also load balanced with LVS. So database read speed is not an issue. Believe me, we know how to build large high traffic sites, the only problem we're having is the exact syntax on using transport_maps or virtual_transport with multiple lmtp transports, and I think I got that figured out with the transport_maps. Will post more later. I assume that you are aware that transport_maps is called multiple times. Recipient_maps in rdbms tables generate at least two lookups (one for smtpd, one for cleanup) but when you add transport_maps, that will at least explode to one per subdomain of the sender address (you can mitigate a lot of that with the domain setting in the map configuration file) as trivial-rewrite tries to build its triples for addresses. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
