Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-29 Thread ncjeffgus

On Thu, 29 Mar 2012 07:04:26 +0300, Timo Sirainen wrote:

On 29.3.2012, at 5.07, Jeff Gustafson wrote:


That fixed some errors, but it still is having some sort of trouble
with that command:

# time doveadm -o mail=maildir:/home/bu/user.mdbox import -u
u...@domain.com maildir:/home/users/user%domain.com/Maildir/  all
doveadm(u...@domain.com): Error: Can't find namespace for mailbox 
Trash
doveadm(u...@domain.com): Error: Can't find namespace for mailbox 
test


Oh, you don't have prefix= namespace? If you have e.g.
prefix=INBOX. namespace then use:

time doveadm -o mail=maildir:/home/bu/user.mdbox import -u
user@domain maildir:/home/users/user%domain.com/Maildir/ INBOX all


Oh! I should have known that was the problem.
This was very, very fast. This test is maildir to maildir:

# time doveadm -o mail=maildir:/home/bu/test import -u u...@domain.com 
maildir:/home/users/user%domain.com/Maildir INBOX all


real0m0.412s
user0m0.036s
sys 0m0.088s

But it was just as slow to import into mdbox:

# time doveadm -o mail=mdbox:/home/bu/test2 import -u u...@domain.com 
maildir:/home/users/user%domain.com/Maildir INBOX all


real7m12.738s
user6m46.161s
sys 0m7.046s

mbox... still pretty fast:

# time doveadm -o mail=mbox:/home/bu/test3 import -u u...@domain.com 
maildir:/home/users/user%domain.com/Maildir INBOX all


real0m58.534s
user0m52.264s
sys 0m5.762s

sdbox seems a little on the slow side too:

# time doveadm -o mail=sdbox:/home/bu/test4 import -u u...@domain.com 
maildir:/home/users/user%domain.com/Maildir INBOX all


real6m11.616s
user6m6.924s
sys 0m4.579s

Does information help? It seems that [sm]dbox is on the slow side for 
the purpose of doing backups.


...Jeff



[Dovecot] Problem about dovecot Panic

2012-03-29 Thread FABIO FERRARI
Good morning,
we have 2 Redhat Enterprise 5.7 machines, they are a cluster with some
mail services in it (postfix and dovecot 2).

The version of dovecot is dovecot-2.0.1-1_118.el5 (installed via rpm).

From last week we have this dovecot problem: suddenly dovecot doesn't
accept any new connections, the dovecot.log file reports lines like these

Mar 15 12:38:54 secchia dovecot: imap: Panic: epoll_ctl(add, 5) failed:
Invalid argument
Mar 15 12:38:54 secchia dovecot: imap: Error: Raw backtrace:
/usr/lib64/dovecot/libdovecot.so.0 [0x36ea436de0] -
/usr/lib64/dovecot/libdovecot.so.0 [0x36ea436e3a] - /usr/lib64/dovecot/
libdovecot.so.0 [0x36ea4362e8] -
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118)
[0x36ea441498] - /usr/lib64/dovecot/libdovecot.so.0(io_add+0x8f)
[0x36ea440b7f] - /usr/li
b64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6)
[0x36ea430c16] - dovecot/imap(main+0x10a) [0x41773a] -
/lib64/libc.so.6(__libc_start_main+0xf4) [0x36ea01d994] - dovecot/
imap [0x408179]
Mar 15 12:38:54 secchia dovecot: master: Error: service(imap): child 14514
killed with signal 6 (core dumps disabled)
Mar 15 12:38:54 secchia dovecot: master: Error: service(imap): command
startup failed, throttling
Mar 15 12:39:50 secchia dovecot: imap-login: Error: master(imap): Auth
request timed out (received 0/12 bytes)
Mar 15 12:39:51 secchia dovecot: imap-login: Error: master(imap): Auth
request timed out (received 0/12 bytes)
Mar 15 12:39:51 secchia dovecot: imap-login: Error: master(imap): Auth
request timed out (received 0/12 bytes)
Mar 15 12:39:52 secchia dovecot: imap-login: Error: net_connect_unix(imap)
failed: Resource temporarily unavailable
Mar 15 12:39:52 secchia dovecot: imap-login: Error: net_connect_unix(imap)
failed: Resource temporarily unavailable
Mar 15 12:39:52 secchia dovecot: imap-login: Error: master(imap): Auth
request timed out (received 0/12 bytes)
Mar 15 12:39:53 secchia dovecot: imap-login: Error: net_connect_unix(imap)
failed: Resource temporarily unavailable
Mar 15 12:39:53 secchia dovecot: imap-login: Error: net_connect_unix(imap)
failed: Resource temporarily unavailable
Mar 15 12:39:54 secchia dovecot: imap-login: Error: net_connect_unix(imap)
failed: Resource temporarily unavailable
Mar 15 12:39:54 secchia dovecot: imap: Error: Login client disconnected
too early
Mar 15 12:39:54 secchia dovecot: imap: Error: Login client disconnected
too early
Mar 15 12:39:54 secchia dovecot: imap: Error: Login client disconnected
too early
Mar 15 12:39:54 secchia dovecot: imap: Error: Login client disconnected
too early
Mar 15 12:39:55 secchia dovecot: imap: Panic: epoll_ctl(add, 5) failed:
Invalid argument

and the kern.log file reports

Mar 15 12:38:52 secchia kernel: dlm: closing connection to node 1
Mar 15 12:39:04 secchia kernel: lpfc :83:00.0: 1:(0):2753 PLOGI
failure DID:010400 Status:x9/x32900
Mar 15 12:39:04 secchia kernel: lpfc :03:00.0: 0:(0):2753 PLOGI
failure DID:010400 Status:x9/x32900
Mar 15 12:41:14 secchia kernel: lpfc :03:00.0: 0:(0):2753 PLOGI
failure DID:010400 Status:x9/x32900
Mar 15 12:41:15 secchia kernel: lpfc :83:00.0: 1:(0):2753 PLOGI
failure DID:010400 Status:x9/x32900
Mar 15 12:42:11 secchia kernel: dlm: got connection from 1

can you help us?

thanks in advance

Fabio Ferrari



Re: [Dovecot] Problem about dovecot Panic

2012-03-29 Thread Javier Miguel Rodríguez
  

We had the same problem. Reboot with an older kernel
(2.6.18-274.17.1.el5 works for us). It is known bug of RHEL, see this
bugzilla: 

https://bugzilla.redhat.com/show_bug.cgi?id=681578 

Regards


Javier 

On Thu, 29 Mar 2012 10:15:32 +0200 (CEST), FABIO FERRARI
wrote: 

 Good morning,
 we have 2 Redhat Enterprise 5.7 machines,
they are a cluster with some
 mail services in it (postfix and dovecot
2).
 
 The version of dovecot is dovecot-2.0.1-1_118.el5 (installed
via rpm).
 
 From last week we have this dovecot problem: suddenly
dovecot doesn't
 accept any new connections, the dovecot.log file
reports lines like these
 
 Mar 15 12:38:54 secchia dovecot: imap:
Panic: epoll_ctl(add, 5) failed:
 Invalid argument
 Mar 15 12:38:54
secchia dovecot: imap: Error: Raw backtrace:

/usr/lib64/dovecot/libdovecot.so.0 [0x36ea436de0] -

/usr/lib64/dovecot/libdovecot.so.0 [0x36ea436e3a] -
/usr/lib64/dovecot/
 libdovecot.so.0 [0x36ea4362e8] -

/usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118)

[0x36ea441498] - /usr/lib64/dovecot/libdovecot.so.0(io_add+0x8f)

[0x36ea440b7f] - /usr/li

b64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6)

[0x36ea430c16] - dovecot/imap(main+0x10a) [0x41773a] -

/lib64/libc.so.6(__libc_start_main+0xf4) [0x36ea01d994] - dovecot/

imap [0x408179]
 Mar 15 12:38:54 secchia dovecot: master: Error:
service(imap): child 14514
 killed with signal 6 (core dumps
disabled)
 Mar 15 12:38:54 secchia dovecot: master: Error:
service(imap): command
 startup failed, throttling
 Mar 15 12:39:50
secchia dovecot: imap-login: Error: master(imap): Auth
 request timed
out (received 0/12 bytes)
 Mar 15 12:39:51 secchia dovecot: imap-login:
Error: master(imap): Auth
 request timed out (received 0/12 bytes)

Mar 15 12:39:51 secchia dovecot: imap-login: Error: master(imap): Auth

request timed out (received 0/12 bytes)
 Mar 15 12:39:52 secchia
dovecot: imap-login: Error: net_connect_unix(imap)
 failed: Resource
temporarily unavailable
 Mar 15 12:39:52 secchia dovecot: imap-login:
Error: net_connect_unix(imap)
 failed: Resource temporarily
unavailable
 Mar 15 12:39:52 secchia dovecot: imap-login: Error:
master(imap): Auth
 request timed out (received 0/12 bytes)
 Mar 15
12:39:53 secchia dovecot: imap-login: Error: net_connect_unix(imap)

failed: Resource temporarily unavailable
 Mar 15 12:39:53 secchia
dovecot: imap-login: Error: net_connect_unix(imap)
 failed: Resource
temporarily unavailable
 Mar 15 12:39:54 secchia dovecot: imap-login:
Error: net_connect_unix(imap)
 failed: Resource temporarily
unavailable
 Mar 15 12:39:54 secchia dovecot: imap: Error: Login client
disconnected
 too early
 Mar 15 12:39:54 secchia dovecot: imap: Error:
Login client disconnected
 too early
 Mar 15 12:39:54 secchia dovecot:
imap: Error: Login client disconnected
 too early
 Mar 15 12:39:54
secchia dovecot: imap: Error: Login client disconnected
 too early

Mar 15 12:39:55 secchia dovecot: imap: Panic: epoll_ctl(add, 5)
failed:
 Invalid argument
 
 and the kern.log file reports
 
 Mar
15 12:38:52 secchia kernel: dlm: closing connection to node 1
 Mar 15
12:39:04 secchia kernel: lpfc :83:00.0: 1:(0):2753 PLOGI
 failure
DID:010400 Status:x9/x32900
 Mar 15 12:39:04 secchia kernel: lpfc
:03:00.0: 0:(0):2753 PLOGI
 failure DID:010400 Status:x9/x32900

Mar 15 12:41:14 secchia kernel: lpfc :03:00.0: 0:(0):2753 PLOGI

failure DID:010400 Status:x9/x32900
 Mar 15 12:41:15 secchia kernel:
lpfc :83:00.0: 1:(0):2753 PLOGI
 failure DID:010400
Status:x9/x32900
 Mar 15 12:42:11 secchia kernel: dlm: got connection
from 1
 
 can you help us?
 
 thanks in advance
 
 Fabio Ferrari
 

[Dovecot] File/folder permission issues in 2.1.3

2012-03-29 Thread Christian Rößner
Hi,

I figured out that Dovecot does not honer secondary groups with 
auth/auth-worker (??), if doing LDAP/TLS stuff. I had to use file system acls 
to add the user vmail to /etc/ssl/private and to the corresponding key file:


doveconf -n

# 2.1.3: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS 
auth_master_user_separator = *
auth_mechanisms = plain login
auth_verbose = yes
hostname = mail.roessner-net.de
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_access_groups = vmail
mail_gid = vmail
mail_location = mdbox:~/mdbox
mail_plugins = autocreate quota acl fts fts_solr zlib mail_log notify
mail_privileged_group = mail
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
namespace {
  list = children
  location = mdbox:%%h/mdbox
  prefix = shared/%%u/
  separator = /
  subscriptions = no
  type = shared
}
namespace inbox {
  inbox = yes
  location = 
  mailbox Deleted Messages {
special_use = \Trash
  }
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox Sent Messages {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  mailbox junkmail {
special_use = \Junk
  }
  prefix = 
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/master-users
  driver = passwd-file
  master = yes
  pass = yes
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  acl = vfile
  acl_shared_dict = file:/var/mail/virtual/shared-mailboxes.db
  autocreate = Trash
  autocreate2 = Sent
  autocreate3 = Drafts
  autocreate4 = junkmail
  autosubscribe = Trash
  autosubscribe2 = Sent
  autosubscribe3 = Drafts
  autosubscribe4 = junkmail
  fts = solr
  fts_solr = break-imap-search url=http://localhost:8080/solr/
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
  mail_log_fields = uid box msgid size
  quota = dict:User quota::file:%h/mdbox/dovecot-quota
  quota_rule = *:storage=300M:messages=2
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  zlib_save = gz
  zlib_save_level = 6
}
protocols = imap pop3 lmtp sieve
service auth-worker {
  unix_listener auth-worker {
user = vmail
  }
  user = vmail
}
service auth {
  unix_listener auth-userdb {
mode = 0600
user = vmail
  }
  user = vmail
}
service dict {
  unix_listener dict {
mode = 0600
user = vmail
  }
}
service lmtp {
  inet_listener lmtp {
address = ::1
port = 24
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
user = vmail
  }
  user = dovecot
}
ssl_ca = /ca/psw_net/SSL123_CA_Bundle.pem
ssl_cert = /ca/psw_net/mail_roessner-net_de.crt
ssl_key = /ca/psw_net/mail_roessner-net_de.key
userdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
verbose_proctitle = yes
protocol lmtp {
  mail_plugins = autocreate quota acl fts fts_solr zlib mail_log notify sieve
}
protocol imap {
  imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags
  mail_max_userip_connections = 50
  mail_plugins = autocreate quota acl fts fts_solr zlib mail_log notify 
imap_quota imap_acl imap_zlib
}



Normally, mail is placed under /var/mail/virtual as user vmail, group vmail. Is 
there something wrong with my config that prevents switching to secondary 
groups?



/etc/dovecot/dovecot-ldap.conf.ext:


uris = ldap://ldap0.roessner-net.de/ ldap://db.roessner-net.de/
sasl_bind = yes
sasl_mech = EXTERNAL
tls = yes
tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt
tls_cert_file = /etc/ssl/certs/mx0.roessner-net.de.pem
tls_key_file = /etc/ssl/private/mx0.roessner-net.de.key.pem
tls_require_cert = hard
base = ou=people,ou=it,dc=roessner-net,dc=de
user_attrs = rnsMSQuota=quota_rule=*:storage=%$,rnsMSMailboxHome=home
user_filter = ((objectClass=rnsMSDovecotAccount)(rnsMSRecipientAddress=%u))
pass_attrs = rnsMSDeliverToAddress=user,userPassword=password
pass_filter = 
((objectClass=rnsMSDovecotAccount)(rnsMSRecipientAddress=%u)(rnsMSEnableDovecot=TRUE))
iterate_attrs = rnsMSDovecotUser=user
iterate_filter = (objectClass=rnsMSDovecotAccount)
default_pass_scheme = CRYPT



Thanks in advance.

-Christian

---
Roessner-Network-Solutions
Bachelor of Science Informatik
Nahrungsberg 81, 35390 Gießen
F: +49 641 5879091, M: +49 176 93118939
USt-IdNr.: DE225643613
http://www.roessner-network-solutions.com



smime.p7s
Description: S/MIME cryptographic signature


Re: [Dovecot] dbox vs. mdbox

2012-03-29 Thread Miguel Afonso
On Thu, Mar 29, 2012 at 12:30 AM, Timo Sirainen t...@iki.fi wrote:


 The main problem is that it's difficult to do any real world tests with
 IMAP, especially when users are using many different kinds of IMAP clients.
 So I'm very interested in hearing some numbers (and disk IO graphs for a
 few weeks would be great) before your migration and after your migration,
 but the numbers for your tests might not mean all that much.


I was considering using the imaptest tool to simulate IMAP activity. I
would keep the same machine configuration, only varying the mailbox format
while running imaptest against each setup for a few hours/days.

I'm now converting the original Maildir format to both dbox formats and
I'll give it a try. I'll share some graphs afterwards.


Re: [Dovecot] Sieve fileinto and year/month folders.

2012-03-29 Thread Benny Pedersen

Den 2012-03-28 17:50, Xavier Beaudouin skrev:


require fileinto;

if address :is [From, To] dovecot@dovecot.org {
   fileinto INBOX.mls.%Y.%m.dovecot;
}



is this valid sieve ?

This will fill any mails into INBOX.mls.2012.03.dovecot uppon 
receiving...


not all sieve have date support, and imho no one have macro supported


I don't know if some sieve guru can tell me how to do that... ?


why not keep lmtp ? :)

http://sieve.info






Re: [Dovecot] dsync redesign

2012-03-29 Thread Stan Hoeppner
On 3/28/2012 3:54 PM, Jeff Gustafson wrote:
 On Wed, 2012-03-28 at 11:07 -0500, Stan Hoeppner wrote:
 
 Locally attached/internal/JBOD storage typically offers the best
 application performance per dollar spent, until you get to things like
 backup scenarios, where off node network throughput is very low, and
 your backup software may suffer performance deficiencies, as is the
 issue titling this thread.  Shipping full or incremental file backups
 across ethernet is extremely inefficient, especially with very large
 filesystems.  This is where SAN arrays with snapshot capability come in
 really handy.
 
   I'm a new employee at the company. I was a bit surprised they were not
 using iSCSI. They claim they just can't risk the extra latency. I

The tiny amount of extra latency using a software initiator is a non
argument for a mail server workload, unless the server is undersized for
the workload--high CPU load and low memory constantly.  As I said, in
that case you drop in an iSCSI HBA and eliminate any possibility of
block latency.

 believe that you are right. It seems to me that offloading snapshots and
 backups to an iSCSI SAN would improve things. 

If you get the right unit you won't understand how you ever lived
without it.  The snaps complete transparently, and the data is on the
snap LUN within a few minutes, depending on the priority you give to
internal operations, snaps/rebuilds/etc, vs external IO requests.
Depending on model

 The problem is that this
 company has been burned on storage solutions more than once and they are
 a little skeptical that a product can scale to what they need. There are

More than once?  More than once??  Hmm...

 some SAN vendor names that are a four letter word here. So far, their
 newest FC SAN is performing well.

Interesting.  Care to name them (off list)?

   I think having more, small, iSCSI boxes would be a good solution. One
 problem I've seen with smaller iSCSI products is that feature sets like
 snapshotting are not the best implementation. It works, but doing any
 sort of automation can be painful.

As is most often the case, you get what you pay for.

 The snap takes place wholly within the array and is very fast, without
 the problems you see with host based snapshots such as with Linux LVM,
 where you must first freeze the filesystem, wait for the snapshot to
 complete, which could be a very long time with a 1TB FS.  While this
 occurs your clients must wait or timeout while trying to access
 mailboxes.  With a SAN array snapshot system this isn't an issue as the
 snap is transparent to hosts with little or no performance degradation
 during the snap.  Two relatively inexpensive units that have such
 snapshot capability are:
 
   How does this work? I've always had Linux create a snapshot. Would the
 SAN doing a snapshot without any OS buy-in cause the filesystem to be
 saved in an inconsistent state? I know that ext4 is pretty good at
 logging, but still, wouldn't this be a problem?

Instead of using SAN as a generic term for a box, which it is not,
please use the terms SAN for storage area network, SAN array or
SAN controller when talking about a box with or without disks that
performs the block IO shipping and other storage functions, SAN switch
for a fiber channel switch, or ethernet switch dedicated to the SAN
infrastructure.  The acronym SAN is an umbrella covering many
different types of hardware and network topologies.  It drives me nuts
when people call a fiber channel or iSCSI disk array a SAN.  These can
be part of a SAN, but are not themselves, a SAN.  If they are direct
connected to a single host they are simple disk arrays, and the word
SAN isn't relevant.  Only uneducated people, or those who simply don't
care to be technically correct, call a single intelligent disk box a
SAN.  Ok, end rant on SAN.

Read this primer from Dell:
http://files.accord.com.au/EQL/Docs/CB109_Snapshot_Basic.pdf

The snapshots occur entirely at the controller/disk level inside the
box.  This is true of all SAN units that offer snap ability.  No host OS
involvement at all in the snap.  As I previously said, It's transparent.
 Snaps are filesystem independent, and are point-in-time, or PIT copies
of one LUN to another.  Read up on LUN if you're not familiar with the
term.  Everything in SAN storage is based on LUNs.

Now, as the document above will tell you, array based snapshots may or
may not be a total backup solution for your environment.  You need to
educate yourself and see if this technology is a feature that fits your
file backup and disaster avoidance and recovery needs.

 http://www.equallogic.com/products/default.aspx?id=10613

 http://h10010.www1.hp.com/wwpc/us/en/sm/WF04a/12169-304616-241493-241493-241493.html

 The Equallogic units are 1/10 GbE iSCSI only IIRC, whereas the HP can be
 had in 8Gb FC, 1/10Gb iSCSI, or 6Gb direct attach SAS.  Each offer 4 or
 more host/network connection ports when equipped with dual controllers.
  There are 

Re: [Dovecot] SIS and restoring from backups

2012-03-29 Thread Charles Marcus

On 2012-03-28 7:57 PM, Timo Sirainen t...@iki.fi wrote:

It's easy to restore a full backup. And it's easy to restore specific
users if you have the full backup easily accessible (just run doveadm
import with proper settings pointing to backup). What's difficult is
if you just want to restore a specific user from the backup and can't
easily do random access to all files. Then you'll first need to
restore the user's dbox files and then somehow figure out which
attachments to restore from the SIS directory.


Well, I think I'm not going to worry about this, since you recently said:

On 2012-03-24 9:16 AM, Timo Sirainen t...@iki.fi wrote:
 On 24.3.2012, at 14.54, Charles Marcus wrote:
 I was also thinking of asking about how to provide read-only access
 to these backup snapshots to the users in some kind of special
 namespace, so that they could all essentially go 'back in time' to
 grab any emails that they may have inadvertently deleted...

 This should be possible, just point the namespace to such snapshot.
 You may need to point CONTROL dir to some temporary directory and
 index dir as well to either temp or to memory.

If we really can get these snapshots to automatically show up under a 
'Backups' namespace, with each users folders under each snapshot showing 
by date, so they can easily 'go back in time' and retrieve anything they 
want from them, that totally eliminates any need for me to do individual 
restores... :)



I'd guess that with rsnapshot + Maildir you can get duplicate Maildir
files if the rsnapshot is accessing a large maildir at the same time
as user is changing a message flag. Dovecot usually notices these
duplicates and logs a warning about them.


This won't be a problem wither, because our new system will be 
performing filesystem snapshots for rsnapshot to use as a source.


Thanks again!

--

Best regards,

Charles


Re: [Dovecot] dsync redesign

2012-03-29 Thread Robin
On 3/29/2012 5:24 AM, Stan Hoeppner wrote:
 This happens with a lot of fan boys.  There was so much hype
 surrounding ZFS that even many logically thinking people were frothing
 at the mouth waiting to get their hands on it.  Then, as with many/most
 things in the tech world, the goods didn't live up to the hype.

The problem with zfs especially is that there are so many different 
implementations, with only the commercial Sun, er, Oracle paid Solaris having 
ALL of the promised features and the bug-fixes to make them safely usable.  For 
those users, with very large RAM-backed Sun, er, Oracle, hardware, it probably 
works well.

FreeBSD and even the last versions of OpenSolaris lack fixes for some wickedly 
nasty box-bricking bugs in de-dup, as well as many of the sexy features in 
zpool that had people flocking to it in the first place.  

The bug database that used to be on the OpenSolaris portal by Sun's gone dark, 
but you may have some luck through archive.org.  I know when I tried it out for 
myself using the Community Edition of Solaris, I did feel annoyed by the 
bait-and-switch, and the RAM requirements to run de-dupe with merely adequate 
performance were staggering if I wanted to have plenty of spare block cache 
left over for improving performance overall.

Sun left some of the FOSS operating systems a poison pill with its CDDL 
licence, which is the main reason why the implementations of zfs on Linux are 
immature and is being re-implemented with US DOE sponsorship, ostensibly in a 
GNU compatible licence.

zfs reminds me a great deal of TIFF - lots of great ideas in the White Paper, 
but an elusive (or very very costly) white elephant to acquire.  Rapidly 
changing, bleeding edge, and hot  new are not descriptors for filesystems 
I want to trust more than a token amount of data to.

=R=


Re: [Dovecot] LDAP Lookup not returning value in maxStorage

2012-03-29 Thread Andrew Bruce


On 28/03/2012 19:25, Nikita Koshikov wrote:

On Wed, 28 Mar 2012 09:39:37 +1300
Bruce, Andrew wrote:


On 28 March 2012 09:36, Bruce, Andrewabr...@tumnus.co.nz  wrote:

On 27 March 2012 19:14, Nikita Koshikovkoshi...@gmail.com  wrote:

On Tue, 27 Mar 2012 13:57:04 +1300
Bruce, Andrew wrote:

Hi there,

We're setting up a Dovecot virtual email setup - we've got everything
working perfect with LDAP logins authenticating against AD and so
forth, but we're having issues with retrieving the maxStorage value
from AD (this is a pre-setup field in AD that we'd like to use to set
per user quotas).

In our LDAP lookup, we have the maxStorage entry listed under
user_attrs for the quota (user_attrs =
maxStorage=quota_rule=*:storage=%$M), and in the debug logs, can see
it trying to get the entry, but it fails with:
Mar 27 13:19:27 auth: Debug: ldap(username@site,192.168.1.5): user
search: base=dc=site,dc=local scope=subtree
filter=((objectClass=person)(| (userPrincipalName=username@site)
(|(mail=username@site)(samAccountName=username@site
fields=maxStorage
Mar 27 13:19:27 auth: Debug: ldap(username@site,192.168.1.5): no
fields returned by the server

At this point, we then see the default quota applied.


Try to change your quota rule to be like:
maxStorage=quota_rule=*:bytes=%$
^
And put the value in bytes to maxStorage - if I remember correct - this is 
integer field and no K\M\G values is valid here.

PS We successfully using maxStorage field to obtain non-default quota from AD, 
dovecot version 2.0.x

If we change the name of the field from maxStorage to instanceType we
see the value show up in the logs and passed through to the quota
system and applied successfully:
Mar 27 11:09:01 auth: Debug: ldap(username@site,192.168.1.5): user
search: base=dc=site,dc=local scope=subtree
filter=((objectClass=person)(| (userPrincipalName=username@site)
(|(mail=username@site)(samAccountName=username@site
fields=instanceType
Mar 27 11:09:01 auth: Debug: ldap(username@site,192.168.1.5): result:
instanceType(quota_rule=*:storage=%$M)=*:storage=4M
Mar 27 11:09:01 auth: Debug: master out: USER   3901227009
username@sitequota_rule=*:storage=4M


Which seems a bit weird.

If we use ldapsearch and pass it the same search string and look for
the field maxStorage, we clearly see the field and the value being
returned.  The result looks the same if we also lookup instanceType.

We're using Dovecot 2.0.9.

Does anyone have any idea as to why we can't use this field?

Thanks,

Andrew

Tried your suggestion Nikita, no joy unfortunately.  It still looks
like the value never gets returned from the LDAP server to Dovecot.
It definitely has something in the field (equivalent of 10GB, but in
bytes as suggested) and I changed the user_attrs also, but still get
the same no fields returned by the server error message.

Modifying the user_attrs to lookup from a different field
(instanceType) definitely works.

What exact version are you using - perhaps it's a problem with our
copy of 2.0.9.

Thanks,

Andrew

maybe you met restriction of ldap port 
3268?(http://wiki2.dovecot.org/AuthDatabase/LDAP)

Dead on - it was a restriction of ldap port 3268 - as soon as we pointed 
ldapsearch at the same port, we got the same result - some of the fields 
were missing.  It all makes perfect sense and I wish I noticed that 
earlier.


Now need to work out why Dovecot can get the fields and username back 
from ldap on port 389, but it can't do the auth through it like it could 
with 3268.


Thanks Nikita for your help.

Andrew


Re: [Dovecot] doveadm sync impac problem

2012-03-29 Thread Timo Sirainen
On 22.3.2012, at 19.09, Martin Schitter wrote:

 Am 2012-03-22 03:46, schrieb Gedalya:
 
 doveadm sync/backup via impac puts the same message all over the place...
 
 Thanks Martin, I've set up a test platform to investigate this further
 but I've been short on time...
 
 after some debugging a few more remarks about this problem:
 
 the bug only appears on recursive folder hierarchies.
 if you specity option -m INBOX everything works fine.
 
 for recursive hierarchies the rawlog (-o imapc_rawlog_dir=...) shows that 
 UID FETCH 1:* FLAGS will be called for all folders but UID FETCH NNN 
 (INTERNALDATE) and UID FETCH NNN (BODY.PEEK[]) only happens for the 
 messages in first found subfolder! the last message in this folder will 
 substitute all other messages on the target side... :(
 
 has anyone a clue how to fix this problem in the source code?

http://hg.dovecot.org/dovecot-2.1/rev/078697a32109 should fix it.



[Dovecot] Dovecot migration from any IMAP/POP3 server

2012-03-29 Thread Timo Sirainen
With the latest hg version / upcoming v2.1.4 you can do a perfect migration to 
Maildir using imapc/pop3c backends:

http://wiki2.dovecot.org/Migration/Dsync

The main new feature here is the pop3-migration plugin that matches messages 
from IMAP and POP3 servers together, so that when dsync needs to request POP3 
UIDL for some IMAP message it's actually looked up from the POP3 server.



[Dovecot] newbie: keep getting same emails in mail client

2012-03-29 Thread neuronetv

dovecot-2.0.9-2.el6_1.1.i686
I've just set up dovecot in centos 6.2 (server install) and finally got it
working (kind of). I set up a unix user (not a virtual user) sent a test
email to this user but in my mail client I keep getting this test email over
and over again. I don't think the fault is with the email client because
other emails work fine and never duplicate and I've tweaked the account
settings too, so it must be something I've done wrong in the dovecot setup.
Here is my dovecot.conf file:

# Dovecot configuration file
protocols = pop3 imap
disable_plaintext_auth = no
mail_location = mbox:~/mail:INBOX=/var/spool/mail/unix-username
ssl_cert = /etc/pki/dovecot/certs/dovecot.pem
ssl_key = /etc/pki/dovecot/private/dovecot.pem
passdb {
  driver = pam
}
userdb {
  driver = passwd
}

thanks for any pointers.
-- 
View this message in context: 
http://old.nabble.com/newbie%3A-keep-getting-same-emails-in-mail-client-tp33544893p33544893.html
Sent from the Dovecot mailing list archive at Nabble.com.



Re: [Dovecot] Dovecot migration from any IMAP/POP3 server

2012-03-29 Thread Gedalya

On 3/29/2012 10:27 PM, Timo Sirainen wrote:

With the latest hg version / upcoming v2.1.4 you can do a perfect migration to 
Maildir using imapc/pop3c backends:

http://wiki2.dovecot.org/Migration/Dsync

The main new feature here is the pop3-migration plugin that matches messages 
from IMAP and POP3 servers together, so that when dsync needs to request POP3 
UIDL for some IMAP message it's actually looked up from the POP3 server.



Bravo!!