Re: [rlug] mdir sau mbox
On 8/17/06, Teo [EMAIL PROTECTED] wrote: Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Care este diferenta intre cele 2 alternative? Performanta? Flexibilitate? Integritate? Daca se foloseste mdir, cum susnt structurate mail-urile? Se face cate un director pt fiecare mesaj? Multumesc! ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug Postez un articolas interesant care face o comparatie intre stocare de mailuri in sistem de fisiere si stocare de mailuri in dburi (mbox, exchange, etc) si prezinta de ce este mult mai avantajoasa prima solutie: http://www.memoryhole.net/~kyle/databaseemail.html Database Back-end for Email The (mostly) bad idea that just won't die Introduction There has been a recent discussion on the qmail mailing list regarding using a relational database (a.la. MySQL or Postgres or Oracle) as the back end storage for email. This is an idea that comes up many times, in many contexts, and I am writing this to hopefully educate those who have come up with the same idea again. Thanks to Charles Cazabon and the rest of the qmail mailing list for the ideas contained here. The primary motivation typically behind desires to store email in a database tends to be a desire to search through email quickly. This is an understandable desire, and the urge to go to a database is obvious: databases are built for searching. Delivering mail to a database has been done many times by many people. One example is the DBMail project. Another example is the Microsoft Exchange server, which uses a relational database to store everything. However, storing email in a database is, generally, a bad idea UNLESS you have very particular, very unusual, very specific requirements that justify using a database and which overcome the drawbacks inherent in using a database. Storing email in a database rather than a file-system is usually a bad idea for many reasons, which tend to fall into two categories: Databases Are Bad for Email, and Email is Bad for Databases. Databases Are Bad for Email In general, a good email system must have certain specific characteristics: Reading email must be fast. (goto) Many users must be able to access their mail at the same time, quickly. (goto) Backup must be easy. (goto) Corruption must be easy to recover from. (goto) Let's take them one at a time: 1. Reading email must be fast. In a file-system using (for example) Maildir format for storing messages, the way reading an email works is: I send a request for a message to (for example) my POP3 server, which trivially constructs the path of the file I want, and asks the OS for it. The OS fetches the message (say, mmap()'s it into the server's process space), and returns, at which point the POP3 server sends it to me (which transfers the data back into the OS to talk to the network). The speed of this operation can be significantly improved if the server and OS both support the sendfile() interface. If they can, then this operation is much simpler and MUCH faster: the POP3 server tells the OS to send the file containing the email to me, and the OS returns to the POP3 server when it has been done. This operation (reading an email) is more difficult if the message is stored in even a simple database (such as an mbox-format file, or an sqlite file). In such a case the POP3 server is forced to do several disk accesses (going in and out of the kernel several times) to find out first where the message is stored in the file by reading the database meta data, and then fetching the message out of the file into a local buffer, and then sending that buffer out to the client. Reading this meta data is slow even if the entire database has been mmap()'d into the server's memory space, because the whole thing will most likely not fit into cache or memory at the same time, a condition which is exacerbated if there are many users trying to read their email at the same time. The more complex the database is, the more complex this operation becomes, and thus the slower it becomes. 2. Many users must be able to access their mail at the same time, quickly. The last word of this requirement, quickly, is the difficult part, of course. Generally, which solution is best depends on what kind of concurrency you're going to need to handle. The solution is different if your user base is 2,000 people checking their email occasionally versus 50,000 people whose clients check their email every five minutes. It's possible to use a poor OS/file-system combination with this, though this sort of thing gets down to very basic sysadmin training. For example, many OS's (e.g. Linux before 2.6) use a single lock for each file-system. This makes dealing with concurrent access slower (and harder) than it needs to be. Choosing a good OS/file-system implementation can improve the
Re: [rlug] mdir sau mbox
Teo wrote: Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Care este diferenta intre cele 2 alternative? Performanta? Flexibilitate? Integritate? Daca se foloseste mdir, cum susnt structurate mail-urile? Se face cate un director pt fiecare mesaj? maildir is the way to go. mbox is very obsolete. si n-ai zis ce server de pop/imap folosesti pentru a citi mesajele. ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
Re: [rlug] mdir sau mbox
On Thursday 17 August 2006 16:36, Teo wrote: Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Nu stiu cum legi cu postfix clamav-amavis-SA dar daca o faci prin procmail atunci tehnic merg ambele metode la fel. Care este diferenta intre cele 2 alternative? Implementare tehnica. Un mbox inseamna un singur fisier pt toata casuta de mail (de obicei fiind un format simplu in care pur si simplu sunt concatenate toate mesajele unul dupa altul, unele MUA-uri totusi avand o baza de date, vezi tbird), iar la Maildir ai cate un fisier pt fiecare mesaj. Performanta? Depinde de use-case. In cazul unui singur utilizator probabil ca viteza e comparabila desi eu am avut experiente neplacute pt foldere de mail prea mari pe mbox vs maildir (evident a modifica un director cu foarte multe fisiere prin a sterge un fisier din el e operatie simpla fata de a sterge o parte dintr-un fisier mare, ma rog depinde de implementare). Exista o viteza suplimentara care se vede prin NFS la mbox vs maildir datorita faptului ca maildir specifica clar cum se scriu fisierele si toate operatiile se bazeaza pe operatii primare atomica (mutari de fisiere) pe cand a lucra cu mbox peste NFS e mult mai complex (cu lockd merge cica dar clar e mai lent). Flexibilitate? Banuiesc ca poti face aceleasi operatii pe ambele metode :) Integritate? procmail stie Maildir/mbox, kmail stie maildir/mbox, postfix parca stia maildir/mbox deci ar trebui in ziua de azi maildir sa fie destul de suportat fata de cum era situatia acum ceva ani. Un alt aspect nespecificat de tine este robustetea. In general un maildir e mult mai robust fata de un mbox din cauza simplitatii implementarii, practic nu prea ai ce sa busesti intr-un maildir pe cand la un mbox exista multe lucruri nasoale care se pot intampla sa-ti buseasca toate mesajele (mai ales daca mbox-ul e o baza de date ceva). -- Mihai RUSU Email: [EMAIL PROTECTED] GPG : http://dizzy.roedu.net/dizzy-gpg.txt WWW: http://dizzy.roedu.net Linux is obsolete -- AST ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
RE: [rlug] mdir sau mbox
Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Care este diferenta intre cele 2 alternative? Performanta? Flexibilitate? Integritate? Daca se foloseste mdir, cum susnt structurate mail-urile? Se face cate un director pt fiecare mesaj? Multumesc! ___ Depinde mult de cati useri ai si de ce filesystem folosesti daca te uiti la performanta si cat de mare ajunge mailu la useri, pentru mail pana in 20megi pe server MBOX e ok peste 20megi maildir si reiserfs DW ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
Re: [rlug] mdir sau mbox
În data de Jo, 17-08-2006 la 16:36 +0300, Teo a scris: Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Depinde de foarte multe lucruri. Un bun punct de start: http://tinyurl.com/e8y84 Care este diferenta intre cele 2 alternative? Performanta? Flexibilitate? Integritate? http://en.wikipedia.org/wiki/Maildir http://en.wikipedia.org/wiki/Mbox Daca se foloseste mdir, cum susnt structurate mail-urile? Se face cate un director pt fiecare mesaj? Vezi mai sus. Se face cate un fisier pentru fiecare mesaj. Multumesc! Cu placere, Sami. -- This message was scanned for viruses by BitDefender for Linux Mail Servers. For more information please visit http://www.bitdefender.com/ ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
RE: [rlug] mdir sau mbox
On Thursday 17 August 2006 16:36, Teo wrote: Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Nu stiu cum legi cu postfix clamav-amavis-SA dar daca o faci prin procmail atunci tehnic merg ambele metode la fel. ___ Nu o face prin procmail o face prin amavisd-new, daca folosesti SA ar fi idée buna sa folosesti si rbl's si poate postgrey DW ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
Re: [rlug] mdir sau mbox
Serverul are 300 de useri, dintre care unii isi sterg mesajele cu clientul de mail, insa unii le pastreaza. Fisierle ajung uneori la peste 100M, si citind cele scrise de voi, inteleg ca un mdir e mult mai robust iar viteza de lucru creste. Exista vreo modalitate de a transforma mbox in mdir pentru toti userii? In cazul in care se trece de la mdir la mbox, se mai poate folosi SquirrelMail? On 8/17/06, David Williams [EMAIL PROTECTED] wrote: Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Care este diferenta intre cele 2 alternative? Performanta? Flexibilitate? Integritate? Daca se foloseste mdir, cum susnt structurate mail-urile? Se face cate un director pt fiecare mesaj? Multumesc! ___ Depinde mult de cati useri ai si de ce filesystem folosesti daca te uiti la performanta si cat de mare ajunge mailu la useri, pentru mail pana in 20megi pe server MBOX e ok peste 20megi maildir si reiserfs DW ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
Re: [rlug] mdir sau mbox
On Thu, 17 Aug 2006 16:36:18 +0300 Teo wrote: Daca se foloseste mdir, cum susnt structurate mail-urile? Se face cate un director pt fiecare mesaj? Daca prin mdir intelegi Maildir, atunci: http://www.qmail.org/man/man5/maildir.html http://cr.yp.to/proto/maildir.html Incearca sa mergi pe maildir, in ziua de azi e f. bine suportat, iar riscul de a-ti busi mailboxul pastrat in format mailbox e mult mai mic decat la mbox. Oricum, pe solutia aleasa de tine cu clamav + amavis + SA (teribil de lenta, mai pune postgrey rbl si-l incetinesti si mai tare) cred ca nu accesul la mailbox e pe moment gatuirea in sistem, ci scanarea. (Sugestii OT: nu scana antispam mailurile care pleaca de la tine ci doar ceea ce intra, economisesti resurse si nici nu risti sa-ti marchezi newsletterele ca spam :D. Iar daca folosesti servere RBL la SMTP, nu le mai folosi si in antispam) -- Adi Pircalabu (PGP Key ID 0x04329F5E) -- This message was scanned for viruses by BitDefender for Linux Mail Servers. For more information please visit http://www.bitdefender.com/ ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
Re: [rlug] mdir sau mbox
Teo wrote: Serverul are 300 de useri, dintre care unii isi sterg mesajele cu clientul de mail, insa unii le pastreaza. Fisierle ajung uneori la peste 100M, si citind cele scrise de voi, inteleg ca un mdir e mult mai robust iar viteza de lucru creste. la dimensiuni mici, performantele sint comparabile. la dimensiuni mari ale mbox, performantele de multe ori scad simtitor. depinde insa f mult de la caz la caz. wu-imap de pilda incearca sa aduca in memorie tot mailboxul, asa ca daca nu ai foarte mult RAM, incepe sa faca swapping... iar toata masina se va tirii recomandarea mea ar fi totusi mdir Exista vreo modalitate de a transforma mbox in mdir pentru toti userii? da. vezi un mesaj al meu de acum vreo 2 sapt. In cazul in care se trece de la mdir la mbox, se mai poate folosi SquirrelMail? squirrel sta de vorba direct cu serverul IMAP, nu ii pasa in ce format sint mesajele stocate de server -- Never underestimate the havoc that can be wreaked by misunderstanding the documentation. ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
RE: [rlug] mdir sau mbox
Exista vreo modalitate de a transforma mbox in mdir pentru toti userii? In cazul in care se trece de la mdir la mbox, se mai poate folosi SquirrelMail? ___ Acu ceva timp in urma am folosit scriptul lui Greg #Usage: # mb2md mbox maildir # where mbox must exist, and maildir must not exist. if [ $# -ne 2 ] ; then echo usage: $0 mbox maildir 2 exit 1 fi mbox=$1 maildir=$2 if [ -e $maildir ] ; then echo error: $maildir already exists 2 exit 1 fi if [ ! -e $mbox ] ; then echo error: $mbox does not exist 2 exit 1 fi for p in formail reformail ; do pp=`which $p` if [ -n $pp -a -x $pp ] ; then helper=$pp break fi done if [ -z $helper ]; then echo error: either formail or reformail is required 2 exit 1 fi for d in cur new tmp ; do mkdir -p $maildir/$d done $helper -s addtomaildir $maildir $mbox Din experienta mie mia fost mai incet odata ce mam mutat pe maildir cu useri care avea mailbox de 200megi din cauza FS'ului, pe reiserfs cu multe file mici merge brici acum Squirrel ...nu conteaza e la fel pentru squirrel DW ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
Re: [rlug] mdir sau mbox
Exista programul mbox2mdir si prietenii lui care ia un mbox si il transforma in maildir. Cu putine cunostinte de shell scripting se poate face si pentru 500 de useri. Iar de squirrelmail, pe el il cam doare in basca de ce format folosesti tu la mailbox, pentru ca el lucreaza cu serverul de imap (sau eventual de pop3, dar slabe sanse - de fapt squirrelmail nici nu cred ca stie de pop3, ci doar impu). Va trebui sa convingi insa serverul de imap sa lucreze cu acele mdir-uri, ceea ce poate insemna eventual schimbarea serverului de pop3/imap folosit. Alex Citat Teo [EMAIL PROTECTED]: Serverul are 300 de useri, dintre care unii isi sterg mesajele cu clientul de mail, insa unii le pastreaza. Fisierle ajung uneori la peste 100M, si citind cele scrise de voi, inteleg ca un mdir e mult mai robust iar viteza de lucru creste. Exista vreo modalitate de a transforma mbox in mdir pentru toti userii? In cazul in care se trece de la mdir la mbox, se mai poate folosi SquirrelMail? On 8/17/06, David Williams [EMAIL PROTECTED] wrote: Salut, imi puteti spune va rog, daca este mai avantajos a se folosi mdir sau mbox in conditiile in care se foloseste postfix+clamav+amavis+spamassassin? Care este diferenta intre cele 2 alternative? Performanta? Flexibilitate? Integritate? Daca se foloseste mdir, cum susnt structurate mail-urile? Se face cate un director pt fiecare mesaj? Multumesc! ___ Depinde mult de cati useri ai si de ce filesystem folosesti daca te uiti la performanta si cat de mare ajunge mailu la useri, pentru mail pana in 20megi pe server MBOX e ok peste 20megi maildir si reiserfs DW ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug ___ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
