date:20180521

Re: Cannot delete folder

2018-05-21 Thread Linda A. Walsh


Yves Goergen wrote:

To be honest, I can't follow you. I see that I have inconsistent
permissions (but all the same owner and group). I did use a different
mail server software in the past, but that's long ago and I can't even
remember when I switched to Dovecot. I think it was Courier-IMAP before.

Maildirs for new mailboxes are still created by one of my own scripts so
maybe that script creates the maildirs with some wrong permissions. I'll
first have to look up what all those funny letters actually mean.
They're too rare to remember.

---
hopefully quick explanation of letters 
  +-d=directory

  | usr(me) (r=read,w=write,x=cross(ing)...allows
   seeing things below this directory
   but 'r' controls seeing things *in*
   this directory)
  | ||| group
  | ||| ||| everyone else
My base permissions were : d rwx rwx r-x
most were: :   d rwx rws r-x  The 's' on the group-triplet means
 set-group on files+dirs below this

The exact permissions aren't so important other than to know that
1) when it creates an internal directory (like .imap/whatever), those
  it followed the permissions on my original mbox.
2) when it creats an internal mbox (like INBOX, Trash, Sent...etc) it may
   use whatever it's own umask allows.  Umasks are often set to system 
defaults
   like 022 or 002 on 'friendly systems -- they *remove* a bit from a 
permission set on a file.  Those perms are in octal (0-7), with octal 
you need
3 bits 0111 = 1*4 + 1*2 + 1*1 -- the high bit (multiplied by highest 
amount, in this case 4, corresponds to the read permission, the 2 
corresponds to write permission and the 1 corresponds to execute on 
files files, or cross-through
for directories.  So a umask of 022 would filter out the 2nd bit in each 
permission mask (i.e. the write bit for groups (the middle number) and 
the write
bit for 'other' (the 3rd group)  Since I allow groups to write, I have 
002 so
people in the same group can write. 


   Windows will use something like 077 on your home directory -- only the
user has any access, by default.  the 7's mask out access for any 
created files.


If the process that runs dovecot runs with a umask different from you, 
by default it might create differently permissioned files.  If you 
create a directory manually, that also might be different from what you 
normally see.


   In your case 'Trash' had different permissions -- a directory or 
file that would have been created by dovecot.  so it may be running with 
different default

permissions (settable via the usmask), than you.

   Hope that helps some rather than confuses more...was a quick summary 
of stuff.

Re: Cannot delete folder

2018-05-21 Thread Yves Goergen

In an effort to analyse this, I think I have found the cause for the 
inconsistent permissions.


  mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Trash/cur"

will create .Trash/cur with mode 700. But if .Trash doesn't already 
exist, it will be created with some default mode which is not 700.


  mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Trash"
  mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Trash/cur"

This creates both directories explicitly and with mode 700.

I could get a list of all directories with unexpected mode with

  find /var/mail/virtual -perm -go=rx

And this command fixed the directories (tested in the range of a single 
domain for now):


  find /var/mail/virtual -perm -go=rx |xargs chmod go-rx

Please let me know if there are any problems with this procedure. I 
don't expect having the initial issue again with corrected maildir 
folder permissions and the corrected maildir creation script.


-Yves



Von: Aki Tuomi
Gesendet: Mo, 2018-05-21 10:49 +0200

I guess you have had different version(s) or setting(s) in the past, and
you can probably fix this by setting perms to same as the new folders.

Aki


On 21.05.2018 11:46, Yves Goergen wrote:

I couldn't find anything relevant between all the debug log lines. Is
that in /var/log/mail.log?

Here's the lines I think are related to creating and deleting the folder:


May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace :
/var/mail/virtual/.de/yg/.Test doesn't exist yet, using default
permissions
May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace :
Using permissions from /var/mail/virtual/.de/yg: mode=0700
gid=default
May 21 10:36:23 mond2 dovecot: imap(yg@.de): Debug: Namespace :
/var/mail/virtual/.de/yg/.Trash.Test doesn't exist yet, using
default permissions


That's it.

Thunderbird shows an error message with: [CANNOT] Renaming not
supported across conflicting directory permissions (0.000 + 0.000 secs).

-Yves


Von: Aki Tuomi
Gesendet: So, 2018-05-20 16:00 +0200

Assuming '8' = mail, i cannot see anything wrong here...

Can you turn on 'mail_debug=yes' and attempt folder deletion, and
check logs after that?

Aki


On 20 May 2018 at 16:19 Yves Goergen 
wrote:


Here's the output of 'doveconf -n':

# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
doveconf: Warning: NOTE: You can get a new clean config file with:
doveconf -n > dovecot-new.conf
doveconf: Warning: Obsolete setting in /etc/dovecot/local.conf:21:
ssl_parameters_regenerate should have 'hours' suffix
# OS: Linux 4.4.0-124-generic x86_64 Ubuntu 16.04.3 LTS
auth_mechanisms = plain login
disable_plaintext_auth = no
first_valid_uid = 8
imap_idle_notify_interval = 5 mins
last_valid_uid = 8
listen = 5.9.48.194, 2a01:4f8:161:40c9::2, 127.0.0.1, ::1
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_gid = 8
mail_plugins = quota
mail_privileged_group = mail
mail_uid = 8
namespace inbox {
     inbox = yes
     location =
     mailbox Drafts {
   special_use = \Drafts
     }
     mailbox Junk {
   special_use = \Junk
     }
     mailbox Sent {
   special_use = \Sent
     }
     mailbox "Sent Messages" {
   special_use = \Sent
     }
     mailbox Trash {
   special_use = \Trash
     }
     prefix =
}
passdb {
     args = /etc/dovecot/dovecot-sql.conf.ext
     driver = sql
}
plugin {
     quota = maildir
}
protocols = imap pop3
service imap-login {
     process_limit = 300
}
ssl_cert = 
Date: 19/05/2018 18:51 (GMT+02:00)
To: Dovecot Mailing List 
Subject: Re: Cannot delete folder

The issue still exists. Can anybody explain to me why dovecot creates
IMAP folders with the wrong filesystem permissions? Every new folder
that I create in the mailbox through Thunderbird (IMAP) has the wrong
permission and cannot be deleted anymore.

This looks like a dovecot bug to me.

-Yves



Von: Yves Goergen
Gesendet: Di, 2018-03-06 20:55 +0100

Hi there,

I cannot delete IMAP folders in new mailboxes. When trying to delete a
folder in Thunderbird I get the error message "renaming not supported
across conflicting directory permissions". Looking up the error on the
web led me to checking the mailbox directory's permissions. They're
inconsistent in new mailboxes. While every single directory in an old
mailbox has "drwx--S---", some of the directories in a new mailbox have
"drwxr-sr-x" instead. I'm not sure what that means. The folders with the
later permissions are "Drafts", "Sent" and "Trash". Folders with the
first permissions are "Junk", "Archives" and "Archives.2018". All
folders were created by Thunderbird automatically. Now I wanted to
delete "Archives.2018".

What's the issue here? Why do folders created by the same MUA in the
same event on the same mail server have different permissions and why
can't I delete one of them (which would probably move it into

Re: Cannot delete folder

2018-05-21 Thread Yves Goergen


I've looked up the mode letters and tried this command:

chmod go-rx .Trash

The Trash subdirectories already have the same mode as the mailbox 
directory. With the changed mode on .Trash, the deletion did work. So I 
guess my maildirs were created with the wrong mode. I'm using this 
script to create them:


---8<-

mkdir -p -m 700 "/var/mail/virtual"
mkdir -p -m 700 "/var/mail/virtual/$1"
mkdir -p -m 700 "/var/mail/virtual/$1/$2"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/cur"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/new"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/tmp"

# Create default folders
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Trash/cur"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Trash/new"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Trash/tmp"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Sent/cur"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Sent/new"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Sent/tmp"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Drafts/cur"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Drafts/new"
mkdir -p -m 700 "/var/mail/virtual/$1/$2/.Drafts/tmp"

# Set up subscriptions
cat <"/var/mail/virtual/$1/$2/subscriptions"
Trash
Sent
Drafts
EOF

---8<-

Not sure why this doesn't work. It should create directories with 
drwx-- but it seems to result in the strange drwxr-sr-x. The SGID 
bit (sticky on group) seems to be inherited from the parent directory. 
Also not sure why I would need that.


Actually I believe that the mail user should have full access to all 
maildirs, and no other user. drwx-- should be perfectly sufficient 
for that.


Maybe Courier-IMAP (which I've used long long ago before Dovecot) needed 
these weird modes. Does anybody know something about that?


-Yves




Von: Aki Tuomi
Gesendet: Mo, 2018-05-21 10:49 +0200

I guess you have had different version(s) or setting(s) in the past, and
you can probably fix this by setting perms to same as the new folders.

Aki


On 21.05.2018 11:46, Yves Goergen wrote:

I couldn't find anything relevant between all the debug log lines. Is
that in /var/log/mail.log?

Here's the lines I think are related to creating and deleting the folder:


May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace :
/var/mail/virtual/.de/yg/.Test doesn't exist yet, using default
permissions
May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace :
Using permissions from /var/mail/virtual/.de/yg: mode=0700
gid=default
May 21 10:36:23 mond2 dovecot: imap(yg@.de): Debug: Namespace :
/var/mail/virtual/.de/yg/.Trash.Test doesn't exist yet, using
default permissions


That's it.

Thunderbird shows an error message with: [CANNOT] Renaming not
supported across conflicting directory permissions (0.000 + 0.000 secs).

-Yves


Von: Aki Tuomi
Gesendet: So, 2018-05-20 16:00 +0200

Assuming '8' = mail, i cannot see anything wrong here...

Can you turn on 'mail_debug=yes' and attempt folder deletion, and
check logs after that?

Aki


On 20 May 2018 at 16:19 Yves Goergen 
wrote:


Here's the output of 'doveconf -n':

# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
doveconf: Warning: NOTE: You can get a new clean config file with:
doveconf -n > dovecot-new.conf
doveconf: Warning: Obsolete setting in /etc/dovecot/local.conf:21:
ssl_parameters_regenerate should have 'hours' suffix
# OS: Linux 4.4.0-124-generic x86_64 Ubuntu 16.04.3 LTS
auth_mechanisms = plain login
disable_plaintext_auth = no
first_valid_uid = 8
imap_idle_notify_interval = 5 mins
last_valid_uid = 8
listen = 5.9.48.194, 2a01:4f8:161:40c9::2, 127.0.0.1, ::1
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_gid = 8
mail_plugins = quota
mail_privileged_group = mail
mail_uid = 8
namespace inbox {
     inbox = yes
     location =
     mailbox Drafts {
   special_use = \Drafts
     }
     mailbox Junk {
   special_use = \Junk
     }
     mailbox Sent {
   special_use = \Sent
     }
     mailbox "Sent Messages" {
   special_use = \Sent
     }
     mailbox Trash {
   special_use = \Trash
     }
     prefix =
}
passdb {
     args = /etc/dovecot/dovecot-sql.conf.ext
     driver = sql
}
plugin {
     quota = maildir
}
protocols = imap pop3
service imap-login {
     process_limit = 300
}
ssl_cert = 
Date: 19/05/2018 18:51 (GMT+02:00)
To: Dovecot Mailing List 
Subject: Re: Cannot delete folder

The issue still exists. Can anybody explain to me why dovecot creates
IMAP folders with the wrong filesystem permissions? Every new folder
that I create in the mailbox through Thunderbird (IMAP) has the wrong
permission and cannot be deleted anymore.

This looks like a dovecot bug to me.

-Yves



Von: Yves Goergen
Gesendet: Di, 2018-03-06 20:55 +0100

Hi there,

I cannot delete IMAP folders in new mailboxes. When trying to delete a
folder in Thunderbird I get the

Re: Cannot delete folder

2018-05-21 Thread Yves Goergen

To be honest, I can't follow you. I see that I have inconsistent 
permissions (but all the same owner and group). I did use a different 
mail server software in the past, but that's long ago and I can't even 
remember when I switched to Dovecot. I think it was Courier-IMAP before.


Maildirs for new mailboxes are still created by one of my own scripts so 
maybe that script creates the maildirs with some wrong permissions. I'll 
first have to look up what all those funny letters actually mean. 
They're too rare to remember.


-Yves



Von: Linda A. Walsh
Gesendet: Mo, 2018-05-21 20:20 +0200

Yves Goergen wrote:

The issue still exists. Can anybody explain to me why dovecot creates
IMAP folders with the wrong filesystem permissions?

On a lark, I looked through my dirs @ permissions.  Shorted lines a bit
so they'd fit w/o extra lines between them using:
(get rid of text before permissions, and shorten user/group to a few letter)
find . -type d -ls|sed -r 's/^\s*\S+\s+\S+\s+// ; s/linda(group)?/usr/g'

Made it wasy to look down the 1st column and find where things were
different.
FWIW, using mbox format.

My base permissions were : drwxrwxr-x
most were:   : drwxrwsr-x
I noticed that subdirs I
created *by hand* (to later
move folders into), often
had different permissions:  drwxrwsr--

The first auto-dir created
by dovecot (.imap under
top dir) had some perms
drawn from the base  : drwxrwxr--
First auto-subdir created
(INBOX) had  : drwxrwxr-x
BUT...had a different
group  -- one that I used
ages ago before making sure
that the username based group
took priority.
(I make each user's primary
group one based on their
username).

Another imap created subdir
(.imap/Junk) : drwxrwx---

took it's perms from the base
file.  Most of my mbox files
have : -rw-rw-r--
but Junk was : -rw-rw (thus no 'r' bit on .imap/Junk)


In all the cases, the folders that were different
derived from manually created mboxes, *OR* a few
auto-created MBOXES (like INBOX ... and likely 'Trash' --
but before using dovecot, I already had a 'Trash' mbox, so it
just used it w/the same perms.

In your case, the permissions likely derive from the
umask that dovecot was(is?) running with when it created
the 'builtin' files (like Trash, Inbox, etc..)

Does that fit your case?

[no subject]

2018-05-21 Thread Rebel Mee

Re: Cannot delete folder

2018-05-21 Thread Linda A. Walsh


Yves Goergen wrote:
The issue still exists. Can anybody explain to me why dovecot creates 
IMAP folders with the wrong filesystem permissions?
On a lark, I looked through my dirs @ permissions.  Shorted lines a bit 
so they'd fit w/o extra lines between them using:

(get rid of text before permissions, and shorten user/group to a few letter)
find . -type d -ls|sed -r 's/^\s*\S+\s+\S+\s+// ; s/linda(group)?/usr/g'

Made it wasy to look down the 1st column and find where things were 
different.

FWIW, using mbox format.

My base permissions were : drwxrwxr-x
most were:   : drwxrwsr-x
I noticed that subdirs I
created *by hand* (to later
move folders into), often
had different permissions:  drwxrwsr--

The first auto-dir created
by dovecot (.imap under
top dir) had some perms
drawn from the base  : drwxrwxr--
First auto-subdir created
(INBOX) had  : drwxrwxr-x
  BUT...had a different
group  -- one that I used
ages ago before making sure
that the username based group
took priority.
(I make each user's primary
group one based on their
username).

Another imap created subdir 
(.imap/Junk) : drwxrwx---


took it's perms from the base
file.  Most of my mbox files
have : -rw-rw-r--
but Junk was : -rw-rw (thus no 'r' bit on .imap/Junk)


In all the cases, the folders that were different
derived from manually created mboxes, *OR* a few
auto-created MBOXES (like INBOX ... and likely 'Trash' --
but before using dovecot, I already had a 'Trash' mbox, so it
just used it w/the same perms.

In your case, the permissions likely derive from the
umask that dovecot was(is?) running with when it created
the 'builtin' files (like Trash, Inbox, etc..)

Does that fit your case?

Re: Dovecot blacklist?

2018-05-21 Thread Aki Tuomi

Or you can implement a policy server yourself. :)The protocol is not 
complicated, json over http. See https://wiki.dovecot.org/Auth/Policy


---Aki TuomiDovecot oy
 Original message From: Aki Tuomi  Date: 
21/05/2018  19:13  (GMT+02:00) To: Marc Perkel , 
dovecot@dovecot.org Subject: Re: Dovecot blacklist? 

https://github.com/PowerDNS/weakforced cn be used for this.

---Aki TuomiDovecot oy
 Original message From: Marc Perkel  Date: 
21/05/2018  19:09  (GMT+02:00) To: dovecot@dovecot.org Subject: Dovecot 
blacklist? 
Just wondering if there is an easy way to have dovecot do a blacklist 
lookup as a negative authentication so that if the IP is on a blacklist 
then authentification fails even if they get the password right.

If this works I have a blacklist everyone can use.

Re: Dovecot blacklist?

2018-05-21 Thread Aki Tuomi


https://github.com/PowerDNS/weakforced cn be used for this.

---Aki TuomiDovecot oy
 Original message From: Marc Perkel  Date: 
21/05/2018  19:09  (GMT+02:00) To: dovecot@dovecot.org Subject: Dovecot 
blacklist? 
Just wondering if there is an easy way to have dovecot do a blacklist 
lookup as a negative authentication so that if the IP is on a blacklist 
then authentification fails even if they get the password right.

If this works I have a blacklist everyone can use.

Re: SSL error after upgrading to 2.31

2018-05-21 Thread Aki Tuomi

You forgot to cc the list.
ssl_ca is used only for validating client certificates.
---Aki TuomiDovecot oy
 Original message From: Marc Perkel  Date: 
21/05/2018  18:25  (GMT+02:00) To: Aki Tuomi  Subject: 
Re: SSL error after upgrading to 2.31 





On 05/21/2018 07:54 AM, Aki Tuomi
  wrote:



  
  Does ssl_cert file contain intermediates?
  

  
  

  
  

  



No - but the ssl_ca does.




  

---
Aki Tuomi
Dovecot oy
  
  

  
  
 Original message 
From: Marc Perkel  
Date: 21/05/2018 16:32 (GMT+02:00) 
To: dovecot@dovecot.org 
Subject: SSL error after upgrading to 2.31 



  
  After upgrading to 2.31 I'm getting
this error. Not sure what I'm doing wrong.



No (No signatures could be verified because the chain contains
only one certificate and it is not self signed.)

  

  

  ssl = yes

  

  ssl_cert =

Re: SSL error after upgrading to 2.31

2018-05-21 Thread Aki Tuomi

Does ssl_cert file contain intermediates?


---Aki TuomiDovecot oy
 Original message From: Marc Perkel  Date: 
21/05/2018  16:32  (GMT+02:00) To: dovecot@dovecot.org Subject: SSL error after 
upgrading to 2.31 

After upgrading to 2.31 I'm getting this
  error. Not sure what I'm doing wrong.

  

  No (No signatures could be verified because the chain contains
  only one certificate and it is not self signed.)





ssl = yes



ssl_cert =

Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

2018-05-21 Thread kadafax

Le 21/05/2018 à 12:38, Aki Tuomi a écrit :
> can you try turning on pluign { fts_enforced = yes } and repeat your test?

Same (wrong) result:
1. Send an email with "too6Ouka" in the body

2. Search against "too6Ouka":
# doveadm search -u username mailbox INBOX body too6Ouka
--> No result

3. Force re-index:
# doveadm fts rescan -u username

4. Search again against "too6Ouka":
# doveadm search -u username mailbox INBOX body too6Ouka
--> e09cce0283e8695ab76002deed92 29055

Don't know if relevant, but on a side note, if I send a second message
with "too6Ouka" in the body, followed by:
# doveadm -v index -u username Inbox
--> doveadm(username): Info: INBOX: Cache is already up to date
And a search against the pattern immediately return only one result
instead of two.

# doveconf -n
# 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.22 (22940fb7)
# OS: Linux 3.10.0-862.2.3.el7.x86_64 x86_64 CentOS Linux release
7.5.1804 (Core)  ext4
# Hostname: mailserver.domain
auth_master_user_separator = *
auth_socket_path = /var/run/dovecot/auth-userdb
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@*
auth_verbose = yes
base_dir = /var/run/dovecot/
debug_log_path = /var/log/dovecot-debug.log
default_client_limit = 5120
default_process_limit = 1024
default_vsz_limit = 2 G
hostname = domain
info_log_path = /var/log/dovecot.log
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *
log_path = /var/log/dovecot-error.log
login_greeting = Welcome to dovecot on mydomain
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_gid = vmail
mail_location = maildir:/var/vmail/%Ln
mail_plugins = " quota stats fts fts_solr"
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace {
  inbox = yes
  location = maildir:/var/vmail/%Ln
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix =
  type = private
}
passdb {
  args = /usr/local/etc/dovecot/deny-users
  deny = yes
  driver = passwd-file
}
passdb {
  args = /usr/local/etc/dovecot/master-users
  driver = passwd-file
  master = yes
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  fts = solr
  fts_autoindex = yes
  fts_enforced = yes
  fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search
  quota = maildir:User quota
  quota_rule = *:storage=20GB
  quota_rule2 = Trash:storage=+200M
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = /var/sieve-scripts/%Ln.sieve
  sieve_dir = /var/vmail/%Ln/home/sieve
  stats_command_min_time = 1 mins
  stats_domain_min_time = 12 hours
  stats_ip_min_time = 12 hours
  stats_memory_limit = 16 M
  stats_refresh = 30 secs
  stats_session_min_time = 15 mins
  stats_track_cmds = yes
  stats_user_min_time = 1 hours
}
postmaster_address = postmaster@domain
protocols = imap pop3 sieve sieve
service auth-worker {
  user = root
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0777
    user = vmail
  }
}
service imap-login {
  inet_listener imaps {
    port = 993
    ssl = yes
  }
  process_min_avail = 10
}
service imap {
  process_limit = 2128
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}
service pop3-login {
  inet_listener pop3s {
    port = 995
    ssl = yes
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  user = nobody
}
service stats {
  fifo_listener stats-mail {
    mode = 0600
    user = vmail
  }
}
ssl_cert = 
> Aki
>
>
> On 21.05.2018 13:31, kada...@gmail.com wrote:
>> Le 21/05/2018 à 11:46, Aki Tuomi a écrit :
>>> That message is coming from the temporary raw user, not from the actual
>>> user.
>>>
>>> How did you verify that the FTS indexes are not created or updated
>>> automatically?
>> I send an email containing a specific strings ('thahB8ea' in that case)
>> and search agains it:
>> # doveadm search -u username mailbox INBOX body thahB8ea
>> --> No result
>> (same with integrated search on roundcube)
>>
>> Then I force a rescan, followed by the same search:
>> # doveadm  fts rescan -u username
>> # doveadm search -u username mailbox INBOX body thahB8ea
>> (Long waiting time)
>> --> e09cce0283e8695ab76002deed92 29048
>>
>> So I conclude that autoindexing is not working.
>>
>> Thanks,
>> kfx
>>

Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

2018-05-21 Thread Aki Tuomi

can you try turning on pluign { fts_enforced = yes } and repeat your test?

Aki


On 21.05.2018 13:31, kada...@gmail.com wrote:
> Le 21/05/2018 à 11:46, Aki Tuomi a écrit :
>> That message is coming from the temporary raw user, not from the actual
>> user.
>>
>> How did you verify that the FTS indexes are not created or updated
>> automatically?
> I send an email containing a specific strings ('thahB8ea' in that case)
> and search agains it:
> # doveadm search -u username mailbox INBOX body thahB8ea
> --> No result
> (same with integrated search on roundcube)
>
> Then I force a rescan, followed by the same search:
> # doveadm  fts rescan -u username
> # doveadm search -u username mailbox INBOX body thahB8ea
> (Long waiting time)
> --> e09cce0283e8695ab76002deed92 29048
>
> So I conclude that autoindexing is not working.
>
> Thanks,
> kfx
>
>> Aki
>>
>>
>> On 04.05.2018 12:47, kada...@gmail.com wrote:
>>>  Bump. Can someone from dovecot dev team please respond something to my
>>> problem ? I've provided all the possible information, and if not please
>>> tell me.
>>> Or do you only provide support for your own FTS solution bundled with
>>> the paid version of dovecot-pro ? If so, you should clearly state it in
>>> the wiki.
>>>
>>> Thanks
>>>
>>> Le 19/04/2018 à 10:30, kada...@gmail.com a écrit :
 I think I'm progressing: if I create a brand new user, send her an
 email, it will not be indexed right away. I need to doveadm rescan /
 index the Inbox before.
 At the end of this email, a dovecot's debug log snipset (grep'ed on this
 user). In it I can read: "Debug: fts: Indexes disabled for namespace ''
 Can this be a problem ?

 Thanks

 doveconf -n output:
 # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf
 # Pigeonhole version 0.4.22 (22940fb7)
 # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux release
 7.4.1708 (Core)  ext4
 # Hostname: mailServer.myDomain
 auth_master_user_separator = *
 auth_socket_path = /var/run/dovecot/auth-userdb
 auth_username_chars =
 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@*
 auth_verbose = yes
 base_dir = /var/run/dovecot/
 debug_log_path = /var/log/dovecot-debug.log
 default_client_limit = 5120
 default_process_limit = 1024
 default_vsz_limit = 2 G
 hostname = myDomain
 info_log_path = /var/log/dovecot.log
 lda_mailbox_autocreate = yes
 lda_mailbox_autosubscribe = yes
 listen = *
 log_path = /var/log/dovecot-error.log
 login_greeting = Welcome
 login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
 mail_gid = vmail
 mail_location = maildir:/var/vmail/%Ln
 mail_plugins = " quota stats fts fts_solr"
 mail_uid = vmail
 managesieve_notify_capability = mailto
 managesieve_sieve_capability = fileinto reject envelope
 encoded-character vacation subaddress comparator-i;ascii-numeric
 relational regex imap4flags copy include variables body enotify
 environment mailbox date index ihave duplicate mime foreverypart 
 extracttext
 namespace {
   inbox = yes
   location = maildir:/var/vmail/%Ln
   mailbox Drafts {
     auto = subscribe
     special_use = \Drafts
   }
   mailbox Sent {
     auto = subscribe
     special_use = \Sent
   }
   mailbox "Sent Messages" {
     special_use = \Sent
   }
   mailbox Trash {
     auto = subscribe
     special_use = \Trash
   }
   prefix =
   type = private
 }
 passdb {
   args = /usr/local/etc/dovecot/deny-users
   deny = yes
   driver = passwd-file
 }
 passdb {
   args = /usr/local/etc/dovecot/master-users
   driver = passwd-file
   master = yes
 }
 passdb {
   args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
 }
 plugin {
   fts = solr
   fts_autoindex = yes
   fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search
   quota = maildir:User quota
   quota_rule = *:storage=20GB
   quota_rule2 = Trash:storage=+200M
   quota_warning = storage=95%% quota-warning 95 %u
   quota_warning2 = storage=80%% quota-warning 80 %u
   sieve = /var/sieve-scripts/%Ln.sieve
   sieve_dir = /var/vmail/%Ln/home/sieve
   stats_command_min_time = 1 mins
   stats_domain_min_time = 12 hours
   stats_ip_min_time = 12 hours
   stats_memory_limit = 16 M
   stats_refresh = 30 secs
   stats_session_min_time = 15 mins
   stats_track_cmds = yes
   stats_user_min_time = 1 hours
 }
 postmaster_address = postmaster@myDomain
 protocols = imap pop3 sieve sieve
 service auth-worker {
   user = root
 }
 service auth {
   unix_listener /var/spool/postfix/private/auth {
     mode = 0777
     user = vmail
   }
 }
 service imap-login {
   inet_listener imaps {

Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

2018-05-21 Thread kadafax

Le 21/05/2018 à 11:46, Aki Tuomi a écrit :
> That message is coming from the temporary raw user, not from the actual
> user.
>
> How did you verify that the FTS indexes are not created or updated
> automatically?

I send an email containing a specific strings ('thahB8ea' in that case)
and search agains it:
# doveadm search -u username mailbox INBOX body thahB8ea
--> No result
(same with integrated search on roundcube)

Then I force a rescan, followed by the same search:
# doveadm  fts rescan -u username
# doveadm search -u username mailbox INBOX body thahB8ea
(Long waiting time)
--> e09cce0283e8695ab76002deed92 29048

So I conclude that autoindexing is not working.

Thanks,
kfx

>
> Aki
>
>
> On 04.05.2018 12:47, kada...@gmail.com wrote:
>>  Bump. Can someone from dovecot dev team please respond something to my
>> problem ? I've provided all the possible information, and if not please
>> tell me.
>> Or do you only provide support for your own FTS solution bundled with
>> the paid version of dovecot-pro ? If so, you should clearly state it in
>> the wiki.
>>
>> Thanks
>>
>> Le 19/04/2018 à 10:30, kada...@gmail.com a écrit :
>>> I think I'm progressing: if I create a brand new user, send her an
>>> email, it will not be indexed right away. I need to doveadm rescan /
>>> index the Inbox before.
>>> At the end of this email, a dovecot's debug log snipset (grep'ed on this
>>> user). In it I can read: "Debug: fts: Indexes disabled for namespace ''
>>> Can this be a problem ?
>>>
>>> Thanks
>>>
>>> doveconf -n output:
>>> # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf
>>> # Pigeonhole version 0.4.22 (22940fb7)
>>> # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux release
>>> 7.4.1708 (Core)  ext4
>>> # Hostname: mailServer.myDomain
>>> auth_master_user_separator = *
>>> auth_socket_path = /var/run/dovecot/auth-userdb
>>> auth_username_chars =
>>> abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@*
>>> auth_verbose = yes
>>> base_dir = /var/run/dovecot/
>>> debug_log_path = /var/log/dovecot-debug.log
>>> default_client_limit = 5120
>>> default_process_limit = 1024
>>> default_vsz_limit = 2 G
>>> hostname = myDomain
>>> info_log_path = /var/log/dovecot.log
>>> lda_mailbox_autocreate = yes
>>> lda_mailbox_autosubscribe = yes
>>> listen = *
>>> log_path = /var/log/dovecot-error.log
>>> login_greeting = Welcome
>>> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
>>> mail_gid = vmail
>>> mail_location = maildir:/var/vmail/%Ln
>>> mail_plugins = " quota stats fts fts_solr"
>>> mail_uid = vmail
>>> managesieve_notify_capability = mailto
>>> managesieve_sieve_capability = fileinto reject envelope
>>> encoded-character vacation subaddress comparator-i;ascii-numeric
>>> relational regex imap4flags copy include variables body enotify
>>> environment mailbox date index ihave duplicate mime foreverypart extracttext
>>> namespace {
>>>   inbox = yes
>>>   location = maildir:/var/vmail/%Ln
>>>   mailbox Drafts {
>>>     auto = subscribe
>>>     special_use = \Drafts
>>>   }
>>>   mailbox Sent {
>>>     auto = subscribe
>>>     special_use = \Sent
>>>   }
>>>   mailbox "Sent Messages" {
>>>     special_use = \Sent
>>>   }
>>>   mailbox Trash {
>>>     auto = subscribe
>>>     special_use = \Trash
>>>   }
>>>   prefix =
>>>   type = private
>>> }
>>> passdb {
>>>   args = /usr/local/etc/dovecot/deny-users
>>>   deny = yes
>>>   driver = passwd-file
>>> }
>>> passdb {
>>>   args = /usr/local/etc/dovecot/master-users
>>>   driver = passwd-file
>>>   master = yes
>>> }
>>> passdb {
>>>   args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
>>>   driver = ldap
>>> }
>>> plugin {
>>>   fts = solr
>>>   fts_autoindex = yes
>>>   fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search
>>>   quota = maildir:User quota
>>>   quota_rule = *:storage=20GB
>>>   quota_rule2 = Trash:storage=+200M
>>>   quota_warning = storage=95%% quota-warning 95 %u
>>>   quota_warning2 = storage=80%% quota-warning 80 %u
>>>   sieve = /var/sieve-scripts/%Ln.sieve
>>>   sieve_dir = /var/vmail/%Ln/home/sieve
>>>   stats_command_min_time = 1 mins
>>>   stats_domain_min_time = 12 hours
>>>   stats_ip_min_time = 12 hours
>>>   stats_memory_limit = 16 M
>>>   stats_refresh = 30 secs
>>>   stats_session_min_time = 15 mins
>>>   stats_track_cmds = yes
>>>   stats_user_min_time = 1 hours
>>> }
>>> postmaster_address = postmaster@myDomain
>>> protocols = imap pop3 sieve sieve
>>> service auth-worker {
>>>   user = root
>>> }
>>> service auth {
>>>   unix_listener /var/spool/postfix/private/auth {
>>>     mode = 0777
>>>     user = vmail
>>>   }
>>> }
>>> service imap-login {
>>>   inet_listener imaps {
>>>     port = 993
>>>     ssl = yes
>>>   }
>>>   process_min_avail = 10
>>> }
>>> service imap {
>>>   process_limit = 2128
>>> }
>>> service managesieve-login {
>>>   inet_listener sieve {
>>>     port = 4190
>>>   }
>>> }
>>> service pop3-login {
>>>   inet_listener pop3s {
>>>     port = 995

Re: install dovecot 2.2.35 debian jessie

2018-05-21 Thread Aki Tuomi



On 27.04.2018 16:51, Webert de Souza Lima wrote:
> Hey Aki Tuomi, how are you doing?
>
> I have tried many ways for getting 2.2.35 pre-built installed via
> 'apt-get install' in Debian Jessie and Stretch using the official repos.
> The reason I prefer to install pre-built instead of compiling it is
> because the I run it on dockers, so it's a lot easier and automated to
> just apt-get install it.
>
> I was using 2.2.31 devel in Debian Jessie, I wanted to move to stable,
> but it will only install 2.2.27 or  2.2.36 alpha from the repos.
>
> 2.2.34 is the default stable using stretch + stretch backport repos,
> it seems OK to me. I have upgraded all my instances.
>
> I will take the opportunity to ask you a question: would you recommend
> me to update director instances too? They run on separate containers.
> I haven't upgraded them and it all seems to be working just fine in
> production.
>

We of course recommend running same version, but I don't know if there
is any pressing need to do so.

Aki

>
> Regards,
>
> Webert Lima
> DevOps Engineer at MAV Tecnologia
> *Belo Horizonte - Brasil*
> *IRC NICK - WebertRLZ*
>
> On Fri, Apr 27, 2018 at 3:10 AM, Aki Tuomi  > wrote:
>
> Hi!
>
> 2.2.35 is not unstable, but apparently it is in debian
> distribution, which is called 'unstable'.
>
> Aki
>
>
> On 27.04.2018 06:16, Webert de Souza Lima wrote:
>> Got 2.2.34 running using debian strech image + strech-backports
>> repos!
>>
>>
>> Regards,
>>
>> Webert Lima
>> DevOps Engineer at MAV Tecnologia
>> *Belo Horizonte - Brasil*
>> *IRC NICK - WebertRLZ*
>>
>> On Thu, Apr 26, 2018 at 9:37 PM, Webert de Souza Lima
>> > wrote:
>>
>> Oh thank you Cedric, I hadn't check that. So 2.2.35 is
>> unstable, huh?
>> I'll deploy 2.2.34 instead. 
>>
>> Thank you!
>>
>>
>> Regards,
>>
>> Webert Lima
>> DevOps Engineer at MAV Tecnologia
>> *Belo Horizonte - Brasil*
>> *IRC NICK - WebertRLZ*
>>
>> On Thu, Apr 26, 2018 at 7:51 PM, Cedric M
>> >
>> wrote:
>>
>> Hi,
>> did you check in unstable ? 
>> https://tracker.debian.org/pkg/dovecot
>> 
>>
>> 2018-04-26 16:43 GMT-04:00 Webert de Souza Lima
>> >:
>>
>> hmm I think I should use stretch instead of jessie,
>> OR I should use a stretch-backport repos, right?
>>
>>
>> Regards,
>>
>> Webert Lima
>> DevOps Engineer at MAV Tecnologia
>> *Belo Horizonte - Brasil*
>> *IRC NICK - WebertRLZ*
>>
>> On Thu, Apr 26, 2018 at 5:39 PM, Webert de Souza Lima
>> > > wrote:
>>
>> Hi, I can't figure how to install latest stable
>> dovecot version 2.2.35 in Debian Jessie
>>
>> If I follow this guide
>> , it ends up
>> installing 2.3
>> If I follow this guide
>> ,
>> it ends up installing either 2.2.13 if I use
>> "stable" or 2.2.36 alpha if I use "jessie"
>>
>> I see that 2.2.35 seems to be missing here
>> too: 
>> http://xi.dovecot.fi/debian/pool/jessie-auto/dovecot-2.2/
>> 
>> 
>>
>> Thanks.
>>
>> Regards,
>>
>> Webert Lima
>> DevOps Engineer at MAV Tecnologia
>> *Belo Horizonte - Brasil*
>> *IRC NICK - WebertRLZ*
>>
>>
>>
>>
>>
>
>

Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

2018-05-21 Thread Aki Tuomi

That message is coming from the temporary raw user, not from the actual
user.

How did you verify that the FTS indexes are not created or updated
automatically?

Aki


On 04.05.2018 12:47, kada...@gmail.com wrote:
>  Bump. Can someone from dovecot dev team please respond something to my
> problem ? I've provided all the possible information, and if not please
> tell me.
> Or do you only provide support for your own FTS solution bundled with
> the paid version of dovecot-pro ? If so, you should clearly state it in
> the wiki.
>
> Thanks
>
> Le 19/04/2018 à 10:30, kada...@gmail.com a écrit :
>> I think I'm progressing: if I create a brand new user, send her an
>> email, it will not be indexed right away. I need to doveadm rescan /
>> index the Inbox before.
>> At the end of this email, a dovecot's debug log snipset (grep'ed on this
>> user). In it I can read: "Debug: fts: Indexes disabled for namespace ''
>> Can this be a problem ?
>>
>> Thanks
>>
>> doveconf -n output:
>> # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf
>> # Pigeonhole version 0.4.22 (22940fb7)
>> # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux release
>> 7.4.1708 (Core)  ext4
>> # Hostname: mailServer.myDomain
>> auth_master_user_separator = *
>> auth_socket_path = /var/run/dovecot/auth-userdb
>> auth_username_chars =
>> abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@*
>> auth_verbose = yes
>> base_dir = /var/run/dovecot/
>> debug_log_path = /var/log/dovecot-debug.log
>> default_client_limit = 5120
>> default_process_limit = 1024
>> default_vsz_limit = 2 G
>> hostname = myDomain
>> info_log_path = /var/log/dovecot.log
>> lda_mailbox_autocreate = yes
>> lda_mailbox_autosubscribe = yes
>> listen = *
>> log_path = /var/log/dovecot-error.log
>> login_greeting = Welcome
>> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
>> mail_gid = vmail
>> mail_location = maildir:/var/vmail/%Ln
>> mail_plugins = " quota stats fts fts_solr"
>> mail_uid = vmail
>> managesieve_notify_capability = mailto
>> managesieve_sieve_capability = fileinto reject envelope
>> encoded-character vacation subaddress comparator-i;ascii-numeric
>> relational regex imap4flags copy include variables body enotify
>> environment mailbox date index ihave duplicate mime foreverypart extracttext
>> namespace {
>>   inbox = yes
>>   location = maildir:/var/vmail/%Ln
>>   mailbox Drafts {
>>     auto = subscribe
>>     special_use = \Drafts
>>   }
>>   mailbox Sent {
>>     auto = subscribe
>>     special_use = \Sent
>>   }
>>   mailbox "Sent Messages" {
>>     special_use = \Sent
>>   }
>>   mailbox Trash {
>>     auto = subscribe
>>     special_use = \Trash
>>   }
>>   prefix =
>>   type = private
>> }
>> passdb {
>>   args = /usr/local/etc/dovecot/deny-users
>>   deny = yes
>>   driver = passwd-file
>> }
>> passdb {
>>   args = /usr/local/etc/dovecot/master-users
>>   driver = passwd-file
>>   master = yes
>> }
>> passdb {
>>   args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
>>   driver = ldap
>> }
>> plugin {
>>   fts = solr
>>   fts_autoindex = yes
>>   fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search
>>   quota = maildir:User quota
>>   quota_rule = *:storage=20GB
>>   quota_rule2 = Trash:storage=+200M
>>   quota_warning = storage=95%% quota-warning 95 %u
>>   quota_warning2 = storage=80%% quota-warning 80 %u
>>   sieve = /var/sieve-scripts/%Ln.sieve
>>   sieve_dir = /var/vmail/%Ln/home/sieve
>>   stats_command_min_time = 1 mins
>>   stats_domain_min_time = 12 hours
>>   stats_ip_min_time = 12 hours
>>   stats_memory_limit = 16 M
>>   stats_refresh = 30 secs
>>   stats_session_min_time = 15 mins
>>   stats_track_cmds = yes
>>   stats_user_min_time = 1 hours
>> }
>> postmaster_address = postmaster@myDomain
>> protocols = imap pop3 sieve sieve
>> service auth-worker {
>>   user = root
>> }
>> service auth {
>>   unix_listener /var/spool/postfix/private/auth {
>>     mode = 0777
>>     user = vmail
>>   }
>> }
>> service imap-login {
>>   inet_listener imaps {
>>     port = 993
>>     ssl = yes
>>   }
>>   process_min_avail = 10
>> }
>> service imap {
>>   process_limit = 2128
>> }
>> service managesieve-login {
>>   inet_listener sieve {
>>     port = 4190
>>   }
>> }
>> service pop3-login {
>>   inet_listener pop3s {
>>     port = 995
>>     ssl = yes
>>   }
>> }
>> service quota-warning {
>>   executable = script /usr/local/bin/quota-warning.sh
>>   user = nobody
>> }
>> service stats {
>>   fifo_listener stats-mail {
>>     mode = 0600
>>     user = vmail
>>   }
>> }
>> ssl_cert = > ssl_key =  # hidden, use -P to show it
>> userdb {
>>   args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext
>>   driver = ldap
>>   override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home
>> }
>> verbose_proctitle = yes
>> protocol lda {
>>   mail_plugins = " quota stats fts fts_solr sieve virtual"
>> }
>> protocol imap {
>>   mail_max_userip_connections = 20
>>   mail_plugins = " quota stats fts

Re: Dovecot fts script with solr

2018-05-21 Thread Aki Tuomi



On 06.05.2018 13:13, André Rodier wrote:
> Hello again,
>
> I have created a parser script, a little bit more advanced than the
> one provided with Dovecot. The main feature is probably to index
> documents inside zip/rar/tgz archives...
>
> I am using Ansible, swaks and doveadm to run automatic tests for each
> supported content. For specific reasons, I am not yet able to add
> Apache Tika to the distribution. However, I already made some tests
> with it. For now, I want to talk about the indexing script.
>
> I also have noticed a few weird behaviours. I will mention them at the
> end, albeit I am not 100% sure where they are coming from. I realised
> last week that using QEMU snapshots was not working as expected, so I
> am now more careful with this feature.
>
> For the developers or users who would be interested and the Dovecot
> team members to understand my questions, here how the tests are working:
>
> To run my tests, I have a set of files in various formats, with a UUID
> inside. They are office files, text files, or even archives with a
> text file inside...
>
> The first test I am running is the script alone. I check that the
> script can convert the file to text, and then I use grep to check the
> UUID is present. This works *perfectly* for all the content, except
> ppt, but it's minor.
>
> The second test is full:
> - I use swaks to send the email with an attachment and the appropriate
> mime type.
> - I then refresh the index using doveadm rescan.
> - I check that fts search returns a line, with doveadm fts search.
> - I then expunge the mailbox to be sure that the next test is valid.
>
> For the second test, it works almost all the time, except in the
> following situations:
> - When the attachment is an email (mime type message/rfc822)
> - RTF (could be a bug in my script)
> - Text file in UTF16 (Even if this file is converted to UTF8)
>
> *Questions:*
> 1 - Is there any limitation or special case for the mime message/rfc822

Not that I can see in decoder.

> 2 - Is the mime type received coming from the email headers?

Mime type received comes from mail header, unless it's
"application/octet-stream", in which case autodetection is attempted
based on file suffix.

> 3 - When the script is called without arguments, what is the purpose
> of the extension at the end of each supported mime types?

The idea is to provide mappings for decoder, so that if the content type
is "application/octet-stream", autodetection can be performed.

> 4 - Can I return a wildcard in the supported mime types, for instance
> "text/* *" ?

Content type matching is done with strcmp, which is probably bit
suboptimal. Have to take a note of this.

> 5 - I would like to handle attachments of types
> application/octet-stream. I have added "application/octet-stream *",
> but I am not sure if dovecot will pass the attachments with these mime
> type or not.
>

application/octet-stream is already handled in code.

> *Notes:*
> 1 - I used netcat to monitor the solr server. I realise that
> sometimes, the data sent to the solr server only contains the headers
> of the email, not the text returned by the parser. Especially with
> rfc822 messages. I will do more tests.
> 2 - I just finished to write the script, it's not yet refactored, but
> at list it is well documented. I will do a full security audit later.
> I am actually testing an associated AppArmor profile.
> 3 - I will do more intensive test on the script on bigger mail boxes
> with more attachments.
> 4 - I may rewrite the script in Python
> 5 - Suggestions welcome.
>
> I initially attached the current version of the script, but the email
> is probably pending for review...In this case, the last development
> version is on Github:
> https://github.com/progmaticltd/homebox/blob/dev/install/playbooks/roles/dovecot/files/fts/decode2text
> The configuration of supported mime types is a simple file, accessible
> on github as well:
> https://github.com/progmaticltd/homebox/blob/dev/install/playbooks/roles/dovecot/templates/fts/mime-supported.conf
>
> Thanks for your advices or suggestions.

Aki Tuomi
Dovecot oy

Re: Cannot delete folder

2018-05-21 Thread Aki Tuomi

I guess you have had different version(s) or setting(s) in the past, and
you can probably fix this by setting perms to same as the new folders.

Aki


On 21.05.2018 11:46, Yves Goergen wrote:
> I couldn't find anything relevant between all the debug log lines. Is
> that in /var/log/mail.log?
>
> Here's the lines I think are related to creating and deleting the folder:
>
>> May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace :
>> /var/mail/virtual/.de/yg/.Test doesn't exist yet, using default
>> permissions
>> May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace :
>> Using permissions from /var/mail/virtual/.de/yg: mode=0700
>> gid=default
>> May 21 10:36:23 mond2 dovecot: imap(yg@.de): Debug: Namespace :
>> /var/mail/virtual/.de/yg/.Trash.Test doesn't exist yet, using
>> default permissions
>
> That's it.
>
> Thunderbird shows an error message with: [CANNOT] Renaming not
> supported across conflicting directory permissions (0.000 + 0.000 secs).
>
> -Yves
>
> 
> Von: Aki Tuomi
> Gesendet: So, 2018-05-20 16:00 +0200
>
> Assuming '8' = mail, i cannot see anything wrong here...
>
> Can you turn on 'mail_debug=yes' and attempt folder deletion, and
> check logs after that?
>
> Aki
>
>> On 20 May 2018 at 16:19 Yves Goergen 
>> wrote:
>>
>>
>> Here's the output of 'doveconf -n':
>>
>> # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
>> # Pigeonhole version 0.4.13 (7b14904)
>> doveconf: Warning: NOTE: You can get a new clean config file with:
>> doveconf -n > dovecot-new.conf
>> doveconf: Warning: Obsolete setting in /etc/dovecot/local.conf:21:
>> ssl_parameters_regenerate should have 'hours' suffix
>> # OS: Linux 4.4.0-124-generic x86_64 Ubuntu 16.04.3 LTS
>> auth_mechanisms = plain login
>> disable_plaintext_auth = no
>> first_valid_uid = 8
>> imap_idle_notify_interval = 5 mins
>> last_valid_uid = 8
>> listen = 5.9.48.194, 2a01:4f8:161:40c9::2, 127.0.0.1, ::1
>> log_timestamp = "%Y-%m-%d %H:%M:%S "
>> mail_gid = 8
>> mail_plugins = quota
>> mail_privileged_group = mail
>> mail_uid = 8
>> namespace inbox {
>>     inbox = yes
>>     location =
>>     mailbox Drafts {
>>   special_use = \Drafts
>>     }
>>     mailbox Junk {
>>   special_use = \Junk
>>     }
>>     mailbox Sent {
>>   special_use = \Sent
>>     }
>>     mailbox "Sent Messages" {
>>   special_use = \Sent
>>     }
>>     mailbox Trash {
>>   special_use = \Trash
>>     }
>>     prefix =
>> }
>> passdb {
>>     args = /etc/dovecot/dovecot-sql.conf.ext
>>     driver = sql
>> }
>> plugin {
>>     quota = maildir
>> }
>> protocols = imap pop3
>> service imap-login {
>>     process_limit = 300
>> }
>> ssl_cert = > ssl_cipher_list =
>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
>>
>> ssl_dh_parameters_length = 2048
>> ssl_key = > ssl_parameters_regenerate = 1 hours
>> ssl_prefer_server_ciphers = yes
>> ssl_protocols = !SSLv2 !SSLv3
>> userdb {
>>     driver = prefetch
>> }
>> protocol imap {
>>     mail_max_userip_connections = 100
>>     mail_plugins = quota imap_quota
>> }
>>
>> And this is the 'namei' output:
>>
>> f: /var/mail/virtual/.de/yg/
>> drwxr-xr-x root root /
>> drwxr-xr-x root root var
>> drwxrwsr-x root mail mail
>> drwx--S--- mail mail virtual
>> drwx--S--- mail mail .de
>> drwx--S--- mail mail yg
>>
>> Here's also the 'ls -al' output of that mailbox directory:
>>
>> insgesamt 136
>> drwx--S--- 12 mail mail  4096 Mai 19 13:52 .
>> drwx--S---  4 mail mail  4096 Mär  4 12:23 ..
>> drwx--S---  5 mail mail  4096 Mai 18 23:16 .Archives
>> drwx--S---  5 mail mail  4096 Mai 17 16:22 .Archives.TestA
>> drwx--S---  5 mail mail  4096 Mai 18 14:09 .Archives.TestB
>> drwx--S---  2 mail mail  4096 Mai 19 13:32 cur
>> -rw---  1 mail mail   736 Mai 16 15:23 dovecot.index
>> -rw---  1 mail mail 22096 Mai 19 13:32 dovecot.index.cache
>> -rw---  1 mail mail 27260 Mai 19 13:34 dovecot.index.log
>> -rw---  1 mail mail    28 Mai 16 14:33 dovecot-keywords
>> -rw---  1 mail mail   168 Mai 16 17:41 dovecot.mailbox.log
>> -rw---  1 mail mail   406 Mai 19 13:32 dovecot-uidlist
>> -rw---  1 mail mail 8 Mai 16 17:41 dovecot-uidvalidity
>> -r--r--r--  1 mail mail 0 Feb 28 19:32 dovecot-uidvalidity.5a96f5de
>> drwxr-sr-x  5 mail mail  4096

Re: Cannot delete folder

2018-05-21 Thread Yves Goergen

I couldn't find anything relevant between all the debug log lines. Is 
that in /var/log/mail.log?


Here's the lines I think are related to creating and deleting the folder:


May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace : 
/var/mail/virtual/.de/yg/.Test doesn't exist yet, using default permissions
May 21 10:36:14 mond2 dovecot: imap(yg@.de): Debug: Namespace : Using 
permissions from /var/mail/virtual/.de/yg: mode=0700 gid=default
May 21 10:36:23 mond2 dovecot: imap(yg@.de): Debug: Namespace : 
/var/mail/virtual/.de/yg/.Trash.Test doesn't exist yet, using default 
permissions


That's it.

Thunderbird shows an error message with: [CANNOT] Renaming not supported 
across conflicting directory permissions (0.000 + 0.000 secs).


-Yves


Von: Aki Tuomi
Gesendet: So, 2018-05-20 16:00 +0200

Assuming '8' = mail, i cannot see anything wrong here...

Can you turn on 'mail_debug=yes' and attempt folder deletion, and check 
logs after that?


Aki


On 20 May 2018 at 16:19 Yves Goergen  wrote:


Here's the output of 'doveconf -n':

# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
doveconf: Warning: NOTE: You can get a new clean config file with:
doveconf -n > dovecot-new.conf
doveconf: Warning: Obsolete setting in /etc/dovecot/local.conf:21:
ssl_parameters_regenerate should have 'hours' suffix
# OS: Linux 4.4.0-124-generic x86_64 Ubuntu 16.04.3 LTS
auth_mechanisms = plain login
disable_plaintext_auth = no
first_valid_uid = 8
imap_idle_notify_interval = 5 mins
last_valid_uid = 8
listen = 5.9.48.194, 2a01:4f8:161:40c9::2, 127.0.0.1, ::1
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_gid = 8
mail_plugins = quota
mail_privileged_group = mail
mail_uid = 8
namespace inbox {
inbox = yes
location =
mailbox Drafts {
  special_use = \Drafts
}
mailbox Junk {
  special_use = \Junk
}
mailbox Sent {
  special_use = \Sent
}
mailbox "Sent Messages" {
  special_use = \Sent
}
mailbox Trash {
  special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
quota = maildir
}
protocols = imap pop3
service imap-login {
process_limit = 300
}
ssl_cert = 
Date: 19/05/2018 18:51 (GMT+02:00)
To: Dovecot Mailing List 
Subject: Re: Cannot delete folder

The issue still exists. Can anybody explain to me why dovecot creates
IMAP folders with the wrong filesystem permissions? Every new folder
that I create in the mailbox through Thunderbird (IMAP) has the wrong
permission and cannot be deleted anymore.

This looks like a dovecot bug to me.

-Yves



Von: Yves Goergen
Gesendet: Di, 2018-03-06 20:55 +0100

Hi there,

I cannot delete IMAP folders in new mailboxes. When trying to delete a
folder in Thunderbird I get the error message "renaming not supported
across conflicting directory permissions". Looking up the error on the
web led me to checking the mailbox directory's permissions. They're
inconsistent in new mailboxes. While every single directory in an old
mailbox has "drwx--S---", some of the directories in a new mailbox have
"drwxr-sr-x" instead. I'm not sure what that means. The folders with the
later permissions are "Drafts", "Sent" and "Trash". Folders with the
first permissions are "Junk", "Archives" and "Archives.2018". All
folders were created by Thunderbird automatically. Now I wanted to
delete "Archives.2018".

What's the issue here? Why do folders created by the same MUA in the
same event on the same mail server have different permissions and why
can't I delete one of them (which would probably move it into the trash
folder first)?

Dovecot version is 2.2.22 on Ubuntu 16.04, Thunderbird is 52.6.0 on
Windows 10.

-Yves

Re: Cannot delete folder

Re: Cannot delete folder

Re: Cannot delete folder

Re: Cannot delete folder

[no subject]

Re: Cannot delete folder

Re: Dovecot blacklist?

Re: Dovecot blacklist?

Re: SSL error after upgrading to 2.31

Re: SSL error after upgrading to 2.31

Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

Re: install dovecot 2.2.35 debian jessie

Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

Re: Dovecot fts script with solr

Re: Cannot delete folder

Re: Cannot delete folder

18 matches

Site Navigation

Mail list logo

Footer information