On Thu, Feb 1, 2007, "Matthew T. O'Connor" <[email protected]> said:
> Aaron Stone wrote: >> On Thu, Feb 1, 2007, Marc Dirix <[EMAIL PROTECTED]> said: >> >>>> Throw memcached >>>> into the picture and the dbmail-imapd process can be told by dbmail-lmtpd >>>> "Hey, I just updated the INBOX for JoeSchmoe, and the current status is in >>>> memcache!" and not a single database query will need to executed on the >>>> IMAP side. >>> Sounds nice, but only if there is no more query optimization possible. >> >> Not every query can return in <10ms, no matter how optimized, and not >> every piece of information is worth keeping in a ready-to-deliver format >> in the database. I would even bet that where a completely >> ready-to-delivery response *is* kept in the database, that same item could >> come from memcached a *lot* faster. > > Perhaps, but I think there is still alot of low hanging fruit that > simple DB schema / query optimization can solve directly, which BTW will > still help us in the memcached world since the data has to be fetched > from the DB at some point. I'm not saying I would ever ignore that. We also need to add a lot more instrumentation to be able to profile single commands and the queries that they call upon. I've got a few ideas for doing this in the 2.2 branch so that we can continue to refine performance for 2.2 over the coming months and years. >> It also just dawned on me that we should probably become dependent upon >> memcached as a requirement. By sucking the memcached code into >> dbmail-clusterd (or whatever we call it). It's a BSD-3 license, and it >> would just be a matter of adding a few additional commands for our >> purposes. I'll hash this ideas out some more and post it on the wiki. > > Whoa slow down there... First I think that memcached should always be > an option. For a small install, things already work just fine as they > are and I would hate to make things more complicated for people who are > happy with a simple setup on one server. Also, from a quick glance, > this should be fairly easy to do, since all programs have to be able to > get data from the DB anyway, it should be easy to just skip the memached > calls on a system that doesn't want it. If it means nothing more that starting dbmail-clusterd in addition to dbmail-imapd and dbmail-lmptd, and we can immediately begin scaling out based on that one additional piece of caching / messaging infrastructure, I'm all for adding that small level of complexity for everyone. I have a personal distaste for systems that aren't so easy to set up in the first place, and are even harder to scale. In this day and age, anything that isn't expressly designed for trivial use, and yet isn't built to scale way, way up and out from day one, is basically a waste of bytes. Every time someone goes and builds a monster sized email system or website, they basically have to build it from the ground up. That's silly. Let's push the state of the art already! > Lets get it working and prove that it's a significant improvement before > we start requiring it. Right, again, just working through some ideas. If we don't throw some spaghetti at the wall, we'll never know what sticks ;-) Aaron _______________________________________________ Dbmail-dev mailing list [email protected] http://twister.fastxs.net/mailman/listinfo/dbmail-dev
