When you crunch the numbers, you don't need the latest-and-greatest processor -- you need ones that will do an adequate job, at the right price, from which you can scale at low cost. Simple, manageable.
However, one of my concerns is the backend mailstore. The load on the MySQL (or PostgreSQL) server could get very high here -- and it doesn't seem anyone has done any "real world" testing of DBMail in this scenario.
DBMail is a new project - so it still has some growing pains to work through, as does any new project. I'd be concerned about using it in production until such a time where things have been tweaked and the feature-set has matured, etc. (that isn't meant to be a negative remark)
In an Enterprise-wide setup, there really needs to be some form of balancing between multiple DB's (or mailstores, however you refer to it).
I've read about mail.com's setup, where they cache inbound mail into an mbox file, until the user logs in, at which point the mail is then piped into the database server - thus reducing the overall load on the server.
Frankly, I'm also concerned about MySQL's performance under such potentially harsh conditions. That remains to be seen.
I'm hoping some others will speak up and participate in a useful way in this thread. This is how a project such as this can gain momentum.
There are probably very few of us who have actually implemented Enterprise-wide mail infrastructures -- there are some useful docs on the net, but a situation like this is really one that needs to be customized per the situation. I had a couple of good people chatting with me when I was considering Qmail, but they disappeared when I mentioned our company would be using Postfix. Not very helpful.
Forrest Kevin Baker wrote:
Forrest, (Long reply) I am also a new dbmail user and testing options. We are definitely moving forward with an implementation of dbmail with similar requirements. We are just now setting up our test environment. Basically my major concerns with a solution are fail over and scalability. We explored a number of options that would load balance by segmenting mailboxes into server clusters. Each cluster being inexpensive machines with large local drives. One primary, one secondary (hot spare) server. This would give us scalability and reasonable fail over. OPTIONS: ---postfix+cyrus murder---- While it seems to solve most problems, the setup is very complex... with little support for RDBMS that would helps with central account administration. Plus cyrus docs flat out say not to use NAS... only SAN, significantly increasing overhead. ---postfix+cyrus+mysql--- Got the mysql for settings and other but for fail over we would need to implement something like DRBD at the filesystem level. This seems like a good option, but would require a very custom OS configuration to handle the DRBD... DRDB also seems to require a high throughput dedicated connection between cluster nodes for any performance... we can't provide that. ---postfix+dbmail+mysql--- This seems to solve everything and is easy to setup. 2 node clusters. Each cluster having a primary and secondary. Secondary uses standard mysql replication with primary as "master" giving us a hot spare. We will probably look into master master replication over the cluster in the future if things go well. We will be using simple ip takeover to handle fail over to the secondary if the primary goes down. Mail is then routed through postfix to the correct server cluster based on mysql configuration for load balancing. So to scale we just add a cluster every x number of users. No need for remote storage solutions... we will get better performance off the cheap local drives, have fail over through the cluster and scale by adding clusters. Each cluster is so cheap it seems to offset the advantage of SAN or NAS... although that was definitely something we were looking at early on. This also has the advantage of low initial overhead. We can easily start with a single cluster and just add as we go. So that it... we are still testing but it looks good so far. No real benchmarks yet. I will likely post in the next week or so as we complete more testing. The major down side I see for you is storage.. where you are looking to offer GB+ per user.. I would guess you *will* likely need a SAN or NAS solution... I'd love to hear about MySQL performance over NAS... that would make this solution even more interesting. Hope this helps.. I'm psyched to see this as a discussion item. The more discussion we get the more this configuration can be refined... tested. Love to see a howto focused specifically around this topic.. I'll have a lot to contrib soon. Kevin Baker Mission Vi Inc. http://www.missionvi.comI'm going to guess that DBMail isn't yet ready for prime-time. It doesn't appear to have the benefit (yet) of wide-spread exposure to Enterprise-scale environments. It's an approach I had been considering, as a possible solution to the many issues I find myself facing with a "standard" configuration (which is inevitably more complex). I see problems with system load on the database server at high-peak -- without a plan to load balance, in whatever way, this could downspiral quickly. No single point of failure is a goal I have for my design (as much as possible). It would be cool to have GMail's GFS for this project ;-) But I don't see that happening anytime soon.. ;-) There are other projects out there - like GFS from RedHat, open-sourced -- which is more geared toward SAN. One of the goals of my design is to keep it simple, if possible -- manageability is a desired end-result. Thanks, Forrest Paul J Stevens wrote:Forrest Aldrich wrote:o IMAP sort and indexing - the load this would create on the system - this is a major concern; could this be distributed amoungst different databases to load-balance?Sort and search is currently implemented in the database client (dbmail-imapd), not serverside. This makes searching and sorting at present very slow when it concerns many messages.o Distributing the accounts evenly across the backend server(s) and which filesystem to useDbmail doesn't use the filesystem for storage, just the database. Dbmail currently only uses a single storage database. Using mysql's replication with automatic fallover is easy, but running a multi-master setup won't work due to possible overlap in generated message id's.o DBmail supports multiple databases - we need some load balancing with reasonable tcp/ip cutoff points to avoid any downspiral performance issues - no single point of failure.Well, there's been talk of using sqlrelay for something like that. But at present the sqlrelay driver setup is on the wishlist only.o Quotas - how granular is DBMail's quota - we'd like to offer x GB of space per user, and aggregate that amoungst an offering of 8 mailboxes per "master account" - the logical approach to this?Quota are per mailbox. There is no concept of groups in the code, only an unused client_id field in the usertable.o Capacity monitoringIndeed, wouldn't a snmp driver be nice.o Deliver to the DB immediately or to a temp mbox file, then to the db upon login? Concerned about the load on the database server(s)As said dbmail doesn't do filebased storage. That would directly conflict with it's distributed nature: the dbmail frontend handling the delivery may well be a very different machine from the one handling the imap connect.Are there any Enterprise-scale implementations of DBMail at this time?Hmm, Ilja? Sorry, but dbmail aint gmail just yet :-)_______________________________________________ Dbmail mailing list Dbmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail_______________________________________________ Dbmail mailing list Dbmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
