Re: librmb: Mail storage on RADOS with Dovecot
On 22 Sep 2017, at 14.18, mj wrote: > First, the Github link: > https://github.com/ceph-dovecot/dovecot-ceph-plugin > > I am not going to repeat everything which is on Github, put a short summary: > > - CephFS is used for storing Mailbox Indexes > - E-Mails are stored directly as RADOS objects > - It's a Dovecot plugin > > We would like everybody to test librmb and report back issues on Github so > that further development can be done. > > It's not finalized yet, but all the help is welcome to make librmb the best > solution for storing your e-mails on Ceph with Dovecot. It would be have been nicer if RADOS support was implemented as lib-fs driver, and the fs-API had been used all over the place elsewhere. So 1) LibRadosMailBox wouldn't have been relying so much on RADOS specifically and 2) fs-rados could have been used for other purposes. There are already fs-dict and dict-fs drivers, so the RADOS dict driver may not have been necessary to implement if fs-rados was implemented instead (although I didn't check it closely enough to verify). (We've had fs-rados on our TODO list for a while also.) BTW. We've also been planning on open sourcing some of the obox pieces, mainly fs-drivers (e.g. fs-s3). The obox format maybe too, but without the "metacache" piece. The current obox code is a bit too much married into the metacache though to make open sourcing it easy. (The metacache is about storing the Dovecot index files in object storage and efficiently caching them on local filesystem, which isn't planned to be open sourced in near future. That's pretty much the only difficult piece of the obox plugin, with Cassandra integration coming as a good second. I wish there had been a better/easier geo-distributed key-value database to use - tombstones are annoyingly troublesome.) And using rmb-mailbox format, my main worries would be: * doesn't store index files (= message flags) - not necessarily a problem, as long as you don't want geo-replication * index corruption means rebuilding them, which means rescanning list of mail files, which means rescanning the whole RADOS namespace, which practically means rescanning the RADOS pool. That most likely is a very very slow operation, which you want to avoid unless it's absolutely necessary. Need to be very careful to avoid that happening, and in general to avoid losing mails in case of crashes or other bugs. * I think copying/moving mails physically copies the full data on disk * Each IMAP/POP3/LMTP/etc process connects to RADOS separately from each others - some connection pooling would likely help here
Re: Different users having same home/mail_location?
Hello Peter, I'm using a virtual users model in dovecot where all the mail is stored under the same OS user account anyway, so file/dir permissions won't be a problem. The system users I was referring to are actually completely separate user accounts at the moment with their own uid/gid (and passwordless sudo between them), but I guess they could use same uid/gid and just have separate entries in passwd/shadow. Anyway at the moment I'm not concerned about getting the authentication part working, but whether there's something in dovecot that will bite my sitting muscles if mail_location is the same for two users. I'm aware of the shared folders functionality mentioned by Marcus and that will probably be my fallback, if sharing a mail_location is not viable. Same mail_location would just be simpler. Tapio On 24.9.2017 0:34, Peter Chiochetti wrote: Hello Tapia, You are going to great lengths :) I assume, that PAM working means, "user" and "otheruser" are the names of system-users with the same uid, home etc? Then I do not see, why dovecot, all defaults and authenticating against PAM won't work out of the box, shouldn't it? Peter Am 2017-09-23 um 22:08 schrieb Tapio Sokura: Hello, Will I be breaking something if I have two users share the same home and/or mail_location (maildir format) in dovecot? Mail will only be delivered to the primary user, the other would be used every now and then to login via imap (webmail). This would be on a CentOS 7 box running dovecot from the packaging system, i.e. dovecot-2.2.10-8.el7 currently. The reason I'd like to do this is to have two ways to login to the same mailbox, one using username + password and the other otherusername + otherpassword + otp-token. Basically this "otherusername" would be for cases where I don't trust the client computer to not have a keylogger etc badness in it, but still need to see my mail. I have this kind of a dual-login setup via PAM for regular system user authentication and it's working ok. I'm just wondering about the dovecot part, whether dovecot will do Unexpected Things if more than one user have the same home and mail_location? If it makes any difference, the users can have separate homes, but the mailbox should be the same for usability. Tapio
Re: Different users having same home/mail_location?
Hello Tapia, You are going to great lengths :) I assume, that PAM working means, "user" and "otheruser" are the names of system-users with the same uid, home etc? Then I do not see, why dovecot, all defaults and authenticating against PAM won't work out of the box, shouldn't it? Peter Am 2017-09-23 um 22:08 schrieb Tapio Sokura: Hello, Will I be breaking something if I have two users share the same home and/or mail_location (maildir format) in dovecot? Mail will only be delivered to the primary user, the other would be used every now and then to login via imap (webmail). This would be on a CentOS 7 box running dovecot from the packaging system, i.e. dovecot-2.2.10-8.el7 currently. The reason I'd like to do this is to have two ways to login to the same mailbox, one using username + password and the other otherusername + otherpassword + otp-token. Basically this "otherusername" would be for cases where I don't trust the client computer to not have a keylogger etc badness in it, but still need to see my mail. I have this kind of a dual-login setup via PAM for regular system user authentication and it's working ok. I'm just wondering about the dovecot part, whether dovecot will do Unexpected Things if more than one user have the same home and mail_location? If it makes any difference, the users can have separate homes, but the mailbox should be the same for usability. Tapio
Re: Different users having same home/mail_location?
On 2017-09-23 20:08:35 +, Tapio Sokura wrote: > Will I be breaking something if I have two users share the same home and/or > mail_location (maildir format) in dovecot? Mail will only be delivered to > the primary user, the other would be used every now and then to login via > imap (webmail). > > This would be on a CentOS 7 box running dovecot from the packaging system, > i.e. dovecot-2.2.10-8.el7 currently. The reason I'd like to do this is to > have two ways to login to the same mailbox, one using username + password > and the other otherusername + otherpassword + otp-token. Basically this > "otherusername" would be for cases where I don't trust the client computer > to not have a keylogger etc badness in it, but still need to see my mail. > > I have this kind of a dual-login setup via PAM for regular system user > authentication and it's working ok. I'm just wondering about the dovecot > part, whether dovecot will do Unexpected Things if more than one user have > the same home and mail_location? If it makes any difference, the users can > have separate homes, but the mailbox should be the same for usability. why not just a shared folder? -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
Different users having same home/mail_location?
Hello, Will I be breaking something if I have two users share the same home and/or mail_location (maildir format) in dovecot? Mail will only be delivered to the primary user, the other would be used every now and then to login via imap (webmail). This would be on a CentOS 7 box running dovecot from the packaging system, i.e. dovecot-2.2.10-8.el7 currently. The reason I'd like to do this is to have two ways to login to the same mailbox, one using username + password and the other otherusername + otherpassword + otp-token. Basically this "otherusername" would be for cases where I don't trust the client computer to not have a keylogger etc badness in it, but still need to see my mail. I have this kind of a dual-login setup via PAM for regular system user authentication and it's working ok. I'm just wondering about the dovecot part, whether dovecot will do Unexpected Things if more than one user have the same home and mail_location? If it makes any difference, the users can have separate homes, but the mailbox should be the same for usability. Tapio
Fwd: Bug#876364: dovecot-sieve: Just discovered imap_sieve/sieve_imapsieve is not set up to work with virtual mailboxes.
Hi... The output of my doveconf -n is as folllows. # 2.2.32 (dfbe293d4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.20 (7cd71ba) doveconf: Warning: /etc/dovecot/dovecot.conf line 105: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf line 47) doveconf: Warning: /etc/dovecot/dovecot.conf line 105: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf line 47) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.2 ext4 auth_default_realm = othersider92.com auth_mechanisms = plain login first_valid_uid = 100 log_path = /var/log/dovecot.log mail_location = maildir:/var/mail/%d/%n mail_plugins = " virtual" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { auto = create special_use = \Drafts } mailbox Junk { auto = create autoexpunge = 1 weeks special_use = \Junk } mailbox Promotions { auto = subscribe } mailbox Sent { auto = create special_use = \Sent } mailbox Trash { auto = create autoexpunge = 30 days special_use = \Trash } prefix = separator = / type = private } namespace virtual { disabled = yes hidden = yes list = no location = virtual:/var/mail/virtual prefix = virtual/ separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { imapsieve_mailbox1_before = file:/etc/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * imapsieve_mailbox3_before = file:/etc/sieve/report-spam-promo.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_name = Promotions imapsieve_mailbox4_before = file:/etc/sieve/report-ham-promo.sieve imapsieve_mailbox4_causes = COPY imapsieve_mailbox4_from = Promotions imapsieve_mailbox4_name = * sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/sieve/conf.d/after sieve_before = /etc/sieve/conf.d/before sieve_global_extensions = +editheader +vnd.dovecot.pipe sieve_pipe_bin_dir = /usr/local/bin sieve_plugins = sieve_extprograms sieve_imapsieve } protocols = imap lmtp service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = wrote: > Control: tags -1 + moreinfo upstream > > [Forwarding this to the Dovecot mailing list, just in case someone can > help] > > Hi, > > Thanks for the report! See my comments inline. > > On 11:56 Thu 21 Sep , Thurgood Angelou wrote: > > Package: dovecot-core > > Version: 1:2.2.32-2 > > > > I've just discovered a bug where the sieve plugin (especially IMAP) > > will not work with a virtual mailbox. I recently found this out when > > testing a virtual mailbox setup and I use imap_sieve for SPAM > > filtering so to lose that would be painful. > > > > This is what is reported in dovecot's debug log... > > > > Sep 21 10:53:01 imap(perso...@othersider92.com): Panic: file > mail-index-map.c: line 549 (mail_index_map_lookup_seq_range): assertion > failed: (first_uid > 0) > > Sep 21 10:53:01 imap(perso...@othersider92.com): Error: Raw backtrace: > /usr/lib/dovecot/libdovecot.so.0(+0x9f0a2) [0x7f4226fd50a2] -> > /usr/lib/dovecot/libdovecot.so.0(+0x9f19a) [0x7f4226fd519a] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4226f65cf8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map_lookup_seq_range+0x120) > [0x7f422733dae0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xe8d1d) > [0x7f422734cd1d] -> /usr/lib/dovecot/libdovecot-st > orage.so.0(mail_index_lookup_seq+0xf) [0x7f4227350e9f] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x82aa) > [0x7f422678a2aa] -> > /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so(+0x708c) > [0x7f422657d08c] -> /usr/lib/dovecot/libdovecot-st > orage.so.0(mailbox_transaction_commit_get_chan
Re: Sieve vacation and srs
Op 9/22/2017 om 10:32 PM schreef azu...@pobox.sk: > Hi, > > is this already fixed? > > https://www.dovecot.org/pipermail/dovecot/2017-July/108688.html Yes: https://github.com/dovecot/pigeonhole/commit/70c218b5627b90b071c7196d5c6ee2063cc3515f The next release will include it. Regards, Stephan.