lmtp with sieve editheader Panic: file edit-mail.c: line 1809 (merge_from_parent): assertion failed

[Matthias Lay]

Hi,

I have experienced a crash in lmtp delivery on some mails when sieve
editheader plugin is used. The first one was an "analytics mail" from
azure exchange. I stripped down the mail to the relevant parts that
trigger this error.

log message:

lmtp(johnd)<27019>: Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42)
[0x7f3048ca0b72] ->
/usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f3048ca0c8e]
-> /usr/lib/dovecot/libdovecot.so.0(+0xff5eb) [0x7f3048cad5eb] ->
/usr/lib/dovecot/libdovecot.so.0(+0xff681) [0x7f3048cad681] ->
/usr/lib/dovecot/libdovecot.so.0(+0x542c4) [0x7f3048c022c4] ->
/usr/lib/dovecot/libdovecot-sieve.so.0(+0x2e11d) [0x7f304841811d] ->
/usr/lib/dovecot/libdovecot.so.0(i_stream_read_memarea+0x74)
[0x7f3048cb92d4] -> /usr/lib/dovecot/libdovecot.so.0(+0x10eb05)
[0x7f3048cbcb05] -> /usr/lib/dovecot/libdovecot.so.0(+0x10f0a9)
[0x7f3048cbd0a9] ->
/usr/lib/dovecot/libdovecot.so.0(i_stream_read_memarea+0x74)
[0x7f3048cb92d4] ->
/usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x3a) [0x7f3048cb94ba]
-> /usr/lib/dovecot/libdovecot.so.0(+0x112f51) [0x7f3048cc0f51] ->
/usr/lib/dovecot/libdovecot.so.0(i_stream_read_memarea+0x74)
[0x7f3048cb92d4] ->
/usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x3a) [0x7f3048cb94ba]
-> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d)
[0x7f3048cb9ddd] ->
/usr/lib/dovecot/libdovecot.so.0(io_stream_copy+0x74) [0x7f3048cd28f4]
-> /usr/lib/dovecot/libdovecot.so.0(o_stream_send_istream+0x4f)
[0x7f3048cd25ef] ->
/usr/lib/dovecot/libdovecot-storage.so.0(index_storage_save_continue+0x29)
[0x7f3048e5dd39] ->
/usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_continue+0x20)
[0x7f3048e07930] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_continue+0x3a)
[0x7f3048ddff3a] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xb1)
[0x7f3048dd0b71] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x475b)
[0x7f3048eed75b] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x6439f)
[0x7f3048de039f] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x52ff4)
[0x7f304843cff4] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x473e9)
[0x7f30484313e9] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x482a0)
[0x7f30484322a0] ->
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x1af)
[0x7f3048432fbf] ->
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_multiscript_finish+0x218)
[0x7f3048446f38] -> /usr/lib/dovecot/lib90_sieve_plugin.so(+0x4030)
[0x7f3048ba9030] ->
/usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x16f)
[0x7f3048eee8ef]
lmtp(johnd)<27019>: Fatal: master:
service(lmtp): child 27019 killed with signal 6 (core dumps disabled -
https://dovecot.org/bugreport.html#coredumps)


the sieve script only has one addheader line

require ["editheader","fileinto","body", "include"];
addheader :last "X-Long-Duration" "XXX";



I will attach doveconf and the lmtp communication. inserted to lmtp
with:

socat - UNIX-CONNECT:$lmtp_socket <<<"$(<$file)"

greetz Matze


info.tar.gz
Description: application/gzip

Re: A user's last access time

[Matthias Lay]

Am Tue, 20 Oct 2020 09:31:16 +0700
schrieb Victor Sudakov :


> Hello Sami,
> 
> I have seen this but I do not want this information in a database. If
> you know how to make the lastlogin_plugin write to a local file, that
> would be very helpful. Even a local sqlite database would do.
> 
> The documentation for the plugin seems very scarce.
> 
> 

you could touch a file in the postlogin script


pgpWpDkrJRnrj.pgp
Description: Digitale Signatur von OpenPGP

sieve size match with multiple of 4GB not matching (BUG?)

[Matthias Lay]

Hi,

I have noticed a strange behaviour when using size matches with GB in
sieve.
If I use filter that matches mails smaller then 4, or 8GB it does not
match in any case. 
same filter matches with 3,5,6,7 GB 

example filter:

require "vnd.dovecot.debug";
if allof (size :under 4G)
{
debug_log "MATCH";
}
else
{
debug_log "NO MATCH";

}

# sieve-filter -u johnd filter.sieve INBOX
>> Filtering message:

  ID:  <5167DFC8.8040207@XX>
  Date:Fri, 12 Apr 2013 12:19:52 +0200
  Size:1820898 bytes
  Subject: Fwd: Multi-media mail demonstration
info: DEBUG: NO MATCH.

**


same mail with this sieve rule:

if allof (size :under 5G)
{
debug_log "MATCH";
}
else
{
debug_log "NO MATCH";

}


# sieve-filter -u johnd filter.sieve INBOX
>> Filtering message:

  ID:  <5167DFC8.8040207@XX>
  Date:Fri, 12 Apr 2013 12:19:52 +0200
  Size:1820898 bytes
  Subject: Fwd: Multi-media mail demonstration
info: DEBUG: MATCH.



same for M. 
4096M => NO MATCH
4097M => MATCH


Greetz Matze

Re: PATCH for Stephan: Re: pigeonhole/lda accessing -m folder

[Matthias Lay via dovecot]

I just noticed you implemented the Dovecot Environment Extension. 
Thanks!

maybe it helps somebody to get this referenced in this thread

https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt

example:
default folder is accessible with:
 
  ${env.vnd.dovecot.default_mailbox}


Greetz Matze





Am Mon, 10 Aug 2015 17:05:58 +0200
schrieb Stephan Bosch :

> Op 8/10/2015 om 4:29 PM schreef Steffen Kaiser:
> > On Mon, 10 Aug 2015, matthias lay wrote:
> >
> > Dear Stephan,
> >
> > If you find this code OK, please merge it into Pigeonhole.  
> 
> Non-standard entries like this require a vnd.dovecot prefix.
> 
> > I find this information useful, too. The +detail and the default
> > mailbox are not necessarily the same. Maybe, there is a namespace to
> > expose this information easily?  
> 
> The Pigeonhole internals support custom variable namespaces. The
> extdata plugin uses this too, so adding something like that is
> architecturally not problematic.
> 
> I would make this dependent on an extension like
> "vnd.dovecot.environment", so that non-standard features are only
> added when this is in the require line. Currently, the "auth"
> envelope field (CMUSieve) is the only exception and I like to keep it
> that way.
> 
> I'll give this a look. Also, are there any other environment items
> that may be useful?
> 
> Regards,
> 
> Stephan.

Re: Dovecot and Oauth

[Matthias Lay via dovecot]

Am Tue, 20 Aug 2019 11:07:13 +1200
schrieb Michael Hallager via dovecot :

> Hi all,
> 
> I am considering expanding a mail server to support Oauth with
> Google. I have read through the following:
> https://wiki.dovecot.org/PasswordDatabase/oauth2
> however, it doesn't work and appears to me to be missing important 
> information, the least of which is API authorisation. Searching with 
> Google, I can't find any more information beyond what amounts to
> C'ing of the above link.
> 
> Has anyone successfully deployed Oauth with Google and could they
> please send me their config files?
> 
> Thanks,
> 
> Michael


Hi,

you should try the

## HTTP request raw log directory
# rawlog_dir = /tmp/oauth2

option. this dumps the http requests in this dir and you can see the
requests and the response from the server.

maybe its the missing 
scope=
option as mentioned in this thread [[ Dovecot - Microsoft Azure AD ]]
if its the case try building dovecot from master branch.


Greetz Matze

Re: lmtp with non ASCII Parameters changed in dovecot 2.3

[Matthias Lay via dovecot]

 
> mail from:
> 500 5.5.2 Invalid command syntax

this was a bad example. the problem is more common on rcpt to: if you
deliver to usernames or special folders

rcpt to:<üser>
or
rcpt to:

lmtp with non ASCII Parameters changed in dovecot 2.3

[Matthias Lay via dovecot]

Hi,

I updated from 2.2 to 2.3 and noticed lmtp rejects parameters
containing non ASCII now. this worked with dovecot-2.2 lmtp.

the string is already denied after reading in
"smtp_command_parse_parameters()"

mail from:
500 5.5.2 Invalid command syntax

is there a plan to change this behaviour, or to implement the SMTPUTF8
extension in lib-smtp?




I guess thie would fix Michals "Dovecot LMTP rejecting mail from address
with apostrophe" Problem too.

Greetz Matze

Re: Variable expansion with variables containing '%' (ldap with 2.3.6)

[Matthias Lay via dovecot]

Am Tue, 2 Jul 2019 08:51:24 +0300
schrieb Aki Tuomi :

> On 2.7.2019 8.24, Aki Tuomi via dovecot wrote:
> > On 1.7.2019 13.37, Matthias Lay via dovecot wrote:  
> >> Am Mon, 1 Jul 2019 13:08:46 +0300 (EEST)
> >> schrieb Aki Tuomi :
> >>  
> >>> Hi,
> >>>
> >>> there seems to be a problem when expanding variables containing a
> >>> single '%' in value in dovecot V2.3.6
> >>>
> >>> having a user defined Variable in user_attrs like
> >>>
> >>> user_attrs = name=home=/maildir/%Ln,
> >>> =myvar=path/%L{ldap:sAMAccountName}
> >>>
> >>>
> >>> and sAMAccountName conains a '%', in my example "sonder%zeichen"
> >>> leads to:
> >>>
> >>> lda(sonder%zeichen)<5723>: Fatal: Failed
> >>> to expand plugin setting myvar = 'path/sonder%zeichen': Unknown
> >>> variable '%z'
> >>>
> >>> same setup works with dovecot 2.2.29
> >>>
> >>> Any Feedback appreciated.
> >>> Thanks.
> >>> Matze
> >>>
> >>> You can use %% to escape a %
> >>> ---
> >>> Aki Tuomi
> >>>  
> >> Hi Aki,
> >>
> >> nope this doesnt work. if I use 'sonder%%zeichen', the ldap lookup
> >> searches for the User/Value "sonder%%zeichen" in ldap/AD. what
> >> fails, as this user doesnt exist.
> >>
> >> and I cant escape the value in all cases anyway, as its an remote
> >> value, coming from the ldap server.
> >>
> >> seems to me the sequence of intepreting variables and modifiers
> >> changed between 2.2 ans 2.3
> >> now it gets the variable value from remote in the first place, and
> >> interprets the value itself for more variables or modifiers, which
> >> might not be what you want.
> >>
> >> like in a subquery using 
> >>
> >> @mail=%{ldap:mailDN}
> >>
> >> but I dont use subqueries. just a simple expansion
> >>
> >> =myvar=%{ldap:mailDN}
> >>
> >> any more ideas?
> >>
> >>
> >>
> >>
> >>  
> > I have to investigate this a bit.
> >
> >
> > Aki
> >  
> 
> Seems to be a bug of a kind. I'll open it in our issue tracker.
> 
> Aki
> 

thx for your effort Aki. 
If I can help out testing a patch, just let me
know

Matze

Re: Variable expansion with variables containing '%' (ldap with 2.3.6)

[Matthias Lay via dovecot]

Am Mon, 1 Jul 2019 13:08:46 +0300 (EEST)
schrieb Aki Tuomi :

> Hi,
> 
> there seems to be a problem when expanding variables containing a
> single '%' in value in dovecot V2.3.6
> 
> having a user defined Variable in user_attrs like
> 
> user_attrs = name=home=/maildir/%Ln,
> =myvar=path/%L{ldap:sAMAccountName}
> 
> 
> and sAMAccountName conains a '%', in my example "sonder%zeichen"
> leads to:
> 
> lda(sonder%zeichen)<5723>: Fatal: Failed to
> expand plugin setting myvar = 'path/sonder%zeichen': Unknown
> variable '%z'
> 
> same setup works with dovecot 2.2.29
> 
> Any Feedback appreciated.
> Thanks.
> Matze
> 
> You can use %% to escape a %
> ---
> Aki Tuomi
> 

Hi Aki,

nope this doesnt work. if I use 'sonder%%zeichen', the ldap lookup
searches for the User/Value "sonder%%zeichen" in ldap/AD. what fails, as
this user doesnt exist.

and I cant escape the value in all cases anyway, as its an remote
value, coming from the ldap server.

seems to me the sequence of intepreting variables and modifiers changed
between 2.2 ans 2.3
now it gets the variable value from remote in the first place, and
interprets the value itself for more variables or modifiers, which
might not be what you want.

like in a subquery using 

@mail=%{ldap:mailDN}

but I dont use subqueries. just a simple expansion

=myvar=%{ldap:mailDN}

any more ideas?

Variable expansion with variables containing '%' (ldap with 2.3.6)

[Matthias Lay via dovecot]

Hi,

there seems to be a problem when expanding variables containing a
single '%' in value in dovecot V2.3.6

having a user defined Variable in user_attrs like

user_attrs = name=home=/maildir/%Ln,
=myvar=path/%L{ldap:sAMAccountName}


and sAMAccountName conains a '%', in my example "sonder%zeichen"
 leads to:

lda(sonder%zeichen)<5723>: Fatal: Failed to
expand plugin setting myvar = 'path/sonder%zeichen': Unknown
variable '%z'

same setup works with dovecot 2.2.29

Any Feedback appreciated.
Thanks.
Matze

Variable expansion with variables containing '%' (ldap with 2.3.6)

[Matthias Lay via dovecot]

Hi,

there seems to be a problem when expanding variables containing a
single '%' in value in dovecot V2.3.6

having a user defined Variable in user_attrs like

user_attrs = name=home=/maildir/%Ln,
=myvar=path/%L{ldap:sAMAccountName}


and sAMAccountName conains a '%', in my example "sonder%zeichen"
 leads to:

lda(sonder%zeichen)<5723>: Fatal: Failed to
expand plugin setting myvar = 'path/sonder%zeichen': Unknown
variable '%z'

same setup works with dovecot 2.2.29

Any Feedback appreciated.
Thanks.
Matze

Re: [BUG] auth_bind with "()#<>"\:," in username not working

[Matthias Lay]

sorry forgot

password for all test users is "insecure"

and you´ll need the function in the header too



diff --git a/src/auth/db-ldap.h b/src/auth/db-ldap.h
index 8a51081..82ed1b3 100644
--- a/src/auth/db-ldap.h
+++ b/src/auth/db-ldap.h
@@ -197,6 +197,8 @@ void db_ldap_enable_input(struct ldap_connection
*conn, bool enable); 
 const char *ldap_escape(const char *str,
const struct auth_request *auth_request);
+const char *ldapdn_escape(const char *str,
+   const struct auth_request *auth_request);
 const char *ldap_get_error(struct ldap_connection *conn);
 
 struct db_ldap_result_iterate_context *










On Tue, 2 Aug 2016 14:32:48 +0200
Matthias Lay <matthias@securepoint.de> wrote:

> Hi once again, replying to myself
> 
> 
> I think I tracked down the problem with a local openldap server.
> 
> IMO the point is, you are using a ldap search escaping for a DN
> Request which needs another kind of escaping.
> the '(' worked well with my NULL-Patch because '(' is a char that
> needs escaping for a search filter but not for DN.
> 
> I experienced some more problems with users containing a '+', '<' for
> example. so I googled a bit and found this one.
> 
> http://www.openldap.org/lists/openldap-software/200407/msg00722.html
> 
> So you might be missing (or I didnt find it) a special DN escaping
> function. I added one in the following patch and all the special chars
> seems to work find in the bind AND search requests.
> 
> 
> 
> diff --git a/src/auth/db-ldap.c b/src/auth/db-ldap.c
> index 1476fa9..e9218ca 100644
> --- a/src/auth/db-ldap.c
> +++ b/src/auth/db-ldap.c
> @@ -1423,6 +1422,35 @@ db_ldap_value_get_var_expand_table(struct
> auth_request *auth_request, return table;
>  }
>  
> +
> +#define IS_LDAPDN_ESCAPED_CHAR(c) \
> +   ((c) == '"' || (c) == '+' || (c) == ',' || (c) == '\\' || (c)
> == '<' || (c) == '>' || (c) == ';') +
> +const char *ldapdn_escape(const char *str,
> +   const struct auth_request *auth_request
> ATTR_UNUSED) +{
> +   const char *p;
> +   string_t *ret;
> +
> +   for (p = str; *p != '\0'; p++) {
> +   if (IS_LDAPDN_ESCAPED_CHAR(*p))
> +   break;
> +   }
> +
> +   if (*p == '\0')
> +   return str;
> +
> +   ret = t_str_new((size_t) (p - str) + 64);
> +   str_append_n(ret, str, (size_t) (p - str));
> +
> +   for (; *p != '\0'; p++) {
> +   if (IS_LDAPDN_ESCAPED_CHAR(*p))
> +   str_append_c(ret, '\\');
> +   str_append_c(ret, *p);
> +   }
> +   return str_c(ret);
> +}
> +
>  #define IS_LDAP_ESCAPED_CHAR(c) \
> ((c) == '*' || (c) == '(' || (c) == ')' || (c) == '\\')
>  
> 
> 
> 
> 
> diff --git a/src/auth/passdb-ldap.c b/src/auth/passdb-ldap.c
> index c1c2544..5629d85 100644
> --- a/src/auth/passdb-ldap.c
> +++ b/src/auth/passdb-ldap.c
> @@ -367,7 +374,7 @@ ldap_verify_plain_auth_bind_userdn(struct
> auth_request *auth_request, 
> brequest->request.type = LDAP_REQUEST_TYPE_BIND;
>  
> -   vars = auth_request_get_var_expand_table(auth_request,
> ldap_escape);
> +   vars = auth_request_get_var_expand_table(auth_request,
> ldapdn_escape); 
>   dn = t_str_new(512);
> var_expand(dn, conn->set.auth_bind_userdn, vars);
>  
> 
> 
> 
> 
> an ldif file for testing.
> add them with 
> # slapadd -l filename
> 
> 
> # cat user.ldif
> dn: dc=uma,dc=local
> dc: uma
> objectClass: dcObject
> objectClass: domain
> structuralObjectClass: domain
> entryUUID: 5cdda309-7ad5-4b03-b981-784c1b7ec27e
> creatorsName: cn=admin,dc=uma,dc=local
> createTimestamp: 20160729231019Z
> entryCSN: 20160729231019.057480Z#00#000#00
> modifiersName: cn=admin,dc=uma,dc=local
> modifyTimestamp: 20160729231019Z
> 
> dn: ou=users,dc=uma,dc=local
> ou: users
> objectClass: organizationalUnit
> structuralObjectClass: organizationalUnit
> entryUUID: cc56753d-09aa-404a-8446-5d0bf75531a3
> creatorsName: cn=admin,dc=uma,dc=local
> createTimestamp: 20160729231019Z
> entryCSN: 20160729231019.147739Z#00#000#00
> modifiersName: cn=admin,dc=uma,dc=local
> modifyTimestamp: 20160729231019Z
> 
> dn: uid=s\+schmidt,ou=users,dc=uma,dc=local
> givenName: Stefan
> uid: s+schmidt
> sn: Schmidt
> mail:: cy5zY2htaWR0QHR0dC1wb2ludC5sb2NhbA0=
> cn: Stefan Schmidt
> objectClass: person
> objectClass: inetOrgPerson
> userPassword:: aW5zZWN1cmU=
> structuralObjectClass: inetOrgPerson
> entryUUID: fffad6fe-d083-4ab9-b6c2-da82067d510b
> creatorsName: cn=admin,dc=uma,dc=local
&g

[BUG] auth_bind with "()#<>"\:," in username not working

[Matthias Lay]

028-444c-8109-cfe9dd182b0b
creatorsName: cn=admin,dc=uma,dc=local
createTimestamp: 20160729231039Z
entryCSN: 20160729231039.364040Z#00#000#00
modifiersName: cn=admin,dc=uma,dc=local
modifyTimestamp: 20160729231039Z

dn: uid=mmeier,ou=users,dc=uma,dc=local
givenName: Manfred
uid: mmeier
sn: Meier
mail:: bS5tZWllckB0dHQtcG9pbnQubG9jYWwN
cn: Manfred Meier
objectClass: person
objectClass: inetOrgPerson
userPassword:: aW5zZWN1cmU=
structuralObjectClass: inetOrgPerson
entryUUID: 16ef0511-25ed-4001-a1bd-1ad72abbfc02
creatorsName: cn=admin,dc=uma,dc=local
createTimestamp: 20160729231039Z
entryCSN: 20160729231039.369003Z#00#000#00
modifiersName: cn=admin,dc=uma,dc=local
modifyTimestamp: 20160729231039Z




Greetz




On Tue, 26 Jul 2016 13:07:24 +0200
Matthias Lay <matthias@securepoint.de> wrote:

> Hi guys,
> 
> 
> I had a look in the sources about this problem.
> 
> the problem seems to be the ldap_escape function that is called from
> 
> ldap_verify_plain_auth_bind_userdn(..)
> 
> I dont really know if this escaping is needed at this point, but with
> this change it works for me. No other problems discovered so far.
> 
> could somebody, who is deeper in the sources give me a hint if
> this will make some troubles?
> 
> 
> Patch for 2.2.16:
> 
> diff --git a/src/auth/passdb-ldap.c b/src/auth/passdb-ldap.c
> index c1c2544..10bfe20 100644
> --- a/src/auth/passdb-ldap.c
> +++ b/src/auth/passdb-ldap.c
> @@ -367,7 +367,7 @@ ldap_verify_plain_auth_bind_userdn(struct
> auth_request *auth_request, 
> brequest->request.type = LDAP_REQUEST_TYPE_BIND;
>  
> -   vars = auth_request_get_var_expand_table(auth_request,
> ldap_escape);
> +   vars = auth_request_get_var_expand_table(auth_request, NULL);
> dn = t_str_new(512);
> var_expand(dn, conn->set.auth_bind_userdn, vars);
>  

[BUG] Re: auth_bind with "()" in username not working

[Matthias Lay]

Hi guys,


I had a look in the sources about this problem.

the problem seems to be the ldap_escape function that is called from

ldap_verify_plain_auth_bind_userdn(..)

I dont really know if this escaping is needed at this point, but with
this change it works for me. No other problems discovered so far.

could somebody, who is deeper in the sources give me a hint if
this will make some troubles?


Patch for 2.2.16:

diff --git a/src/auth/passdb-ldap.c b/src/auth/passdb-ldap.c
index c1c2544..10bfe20 100644
--- a/src/auth/passdb-ldap.c
+++ b/src/auth/passdb-ldap.c
@@ -367,7 +367,7 @@ ldap_verify_plain_auth_bind_userdn(struct
auth_request *auth_request, 
brequest->request.type = LDAP_REQUEST_TYPE_BIND;
 
-   vars = auth_request_get_var_expand_table(auth_request,
ldap_escape);
+   vars = auth_request_get_var_expand_table(auth_request, NULL);
dn = t_str_new(512);
var_expand(dn, conn->set.auth_bind_userdn, vars);
 

Re: auth_bind with "()" in username not working

[Matthias Lay]

Hi again,

did some more tseting on this.

I think the problem is the ldap userlookup, where "("s are evil and
have to be quoted, but these quotes should be removed for the bind
request.

I get my usernames from ldap with a filter like this

user_filter = (sAMAccountName=%Ln)

so I think in between this to steps is the problem.

For testing I hard coded the username for auth_bind and compared strace
output from the auth process


auth_bind_userdn = "spdev\\claasc (test)"


this works fine. strace output from imap login


write(26, "0+\2\1\2`&\2\1\3\4\23spdev\\claasc
(test)\200\fHubertHans99", 45) 


compared to

auth_bind_userdn = "spdev\\%Ln"

which gives

write(26, "0-\2\1\2`(\2\1\3\4\25spdev\\claasc
\\(test\\)\200\fHubertHans99", 47)

and wrong credentials


nobody else encountering similar problems? maybe the "()" are the only
chars making problems at this point


Greetz Matze

auth_bind with "()" in username not working

[Matthias Lay]

Hi all,


I have an AD testsetup with auth_bind setting


auth_bind_userdn = "spdev\\%Ln" 


I created a testuser "claasc (test)" which works fine in all ldapfilters
but not for the auth_bind.

the log shows everything correct just "invalid credentials"


mail.debug: Jun  9 14:12:31 dovecot: auth: Debug: auth client connected
(pid=12202) mail.debug: Jun  9 14:12:31 dovecot: auth: Debug: client
in: AUTH1   PLAIN   service=imap
secured session=T6knVtc0wQB/AAABlip=127.0.0.1
rip=127.0.0.1   lport=143   rport=39873 mail.debug: Jun  9
14:12:31 dovecot: auth: Debug: client passdb out: CONT  1
mail.debug: Jun  9 14:12:31 dovecot: auth: Debug: client in:
CONT mail.debug: Jun  9 14:12:31 dovecot: auth: Debug:
passwd-file(claasc (test),127.0.0.1,): cache miss
mail.debug: Jun  9 14:12:31 dovecot: auth: Debug: passwd-file(claasc
(test),127.0.0.1,): lookup: user=claasc (test)
file=/etc/dovecot/passwd.postmaster mail.info: Jun  9 14:12:31 dovecot:
auth: passwd-file(claasc (test),127.0.0.1,): unknown
user (given password: HubertHans99) mail.debug: Jun  9 14:12:31
dovecot: auth: Debug: ldap(claasc (test),127.0.0.1,):
cache miss mail.info: Jun  9 14:12:31 dovecot: auth: ldap(claasc
(test),127.0.0.1,): invalid credentials mail.debug:
Jun  9 14:12:33 dovecot: auth: Debug: client passdb out: FAIL
1   user=claasc (test) mail.info: Jun  9 14:12:33 dovecot:
imap-login: Aborted login (auth failed, 1 attempts in 2 secs):
user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1,
secured, session=


So I guess its just a bind problem. 

strace output from auth process of imap login:

write(26, "0-\2\1\4`(\2\1\3\4\25spdev\\claasc
\\(test\\)\200\fHubertHans99", 47) = 47


the additional \\ in front of the brackets look strange to me and might
be the reason.

tested Version is 2.2.18


Greetz Matze

Re: A few new Pigeonhole features

[Matthias Lay]

On Thu, 14 Apr 2016 12:22:22 +0200
Stephan Bosch  wrote:


> 
> I am not sure what you mean exactly. Can you clarify?
>


sure,

at the moment, if I am using a script like this

fileinto :copy "myspecialplace"

in lda sieve, the hardlinking is not working the mail gets duplicated on
the FS, as far as I can see.

So if the sieve is acting on an already saved mail, I guess it would be
hard linked?



Regards Matze

Re: A few new Pigeonhole features

[Matthias Lay]

On Thu, 7 Apr 2016 15:44:04 +0200
Stephan Bosch  wrote:


> 
> - Recently, I implemented support for the "imapsieve" extension in
> Sieve and its IMAP counterpart (https://tools.ietf.org/html/rfc6785).
> This is now in the Pigeonhole master branch. This allows running
> Sieve scripts at IMAP activity, rather than at delivery. There are
> also facilities for the familiar sieve_before/sieve_after
> administrator scripts. The documentation is currently only in th


this sounds great! 
I guess this will work with dovecot hardlinks on sieve copying in
contrast to lda sieve?


Greetz Matze



> sources and not on the wiki:
> 
> https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/plugins/imapsieve.txt
> 
> - Yesterday, I implemented online trace debugging for Sieve scripts: 
> i.e., directly during delivery rather than only using the sieve-test 
> tool. This is also supported for the IMAPSIEVE feature discussed
> above. This is documented in the INSTALL file:
> 
> https://github.com/dovecot/pigeonhole/blob/master/INSTALL#L522
> 
> 
> 
> In other news, I started a specification for a new Sieve feature that 
> allows using the IMAP SPECIAL-USE attributes from Sieve scripts:
> 
> https://tools.ietf.org/html/draft-bosch-sieve-special-use-00
> 
> There is no implementation yet.
> 
> 
> Any comments, suggestions or bug reports are welcome.
> 
> Regards,
> 
> Stephan.

Re: subscribe users mailbox with non-valid mutf7 chars

[Matthias Lay]

did some more testing on this and encoutnered its even not working when
using filesystem links AND encoding the username in mutf7.

it ends up in the same error than doveadm subscribe.



what i tried:

* renamed public folder on FS to mutf7:

"a & p products group" => "a &- p products group"

* created link in the users Maildir:

private/franzf/Maildir# ll public/a\ \&-\ p\ products\ group/
total 8K
drwxr-xr-x 2 vmail vmail 4096 Mar 16 15:09 .
drwxr-xr-x 5 vmail vmail 4096 Mar 16 15:06 ..
lrwxrwxrwx 1 vmail vmail   56 Mar 16 15:09 LTA
-> /var/data/vmail/public/a &- p products group/Maildir/LTA


* trying to select the folder in imap:



4 select "public/a &- p products group/LTA/forever"
imap(franzf): Debug: fs: root=/var/data/vmail/public/a &- p products
group/Maildir, index=, indexpvt=, control=,
inbox=/var/data/vmail/public/a &- p products group/Maildir/INBOX, alt=
imap(franzf): Debug: acl: initializing backend with data:
vfile:/etc/dovecot/acls/acl-defaults imap(franzf): Debug: acl: acl
username = a &- p products group imap(franzf): Debug: acl: owner = 1
imap(franzf): Debug: acl vfile: Global ACL
file: /etc/dovecot/acls/acl-defaults imap(franzf): Debug: fs:
root=/var/data/vmail/public/a &- p products group/Maildir, index=,
indexpvt=, control=, inbox=/var/data/vmail/public/a &- p products
group/Maildir/INBOX, alt= imap(franzf): Debug: acl: initializing
backend with data: vfile:/etc/dovecot/acls/acl-defaults imap(franzf):
Debug: acl: acl username = franzf imap(franzf): Debug: acl: owner = 0
imap(franzf): Debug: acl vfile: Global ACL
file: /etc/dovecot/acls/acl-defaults 4 NO [CANNOT] Invalid mailbox name
'public/a & p products group/LTA/forever': Missing namespace prefix
'public/a &- p products group/' (0.000 secs).




the same procedure works great with every Ascii Username.



On Tue, 9 Feb 2016 12:31:31 +0100
Matthias Lay <matthias@securepoint.de> wrote:

> 
> Hi all,
> 
> I got a question if its possible to subscribe a mailbox from another
> user, who contains non mutf7 valid chars in his mailbox path.
> 
> 
> I have a private namespace which can subscribe to mailboxes of a
> shared namespace with public/ prefix
> 
> in this example
> username in private namespace is johnd
> public username is "a & p products group"
> 
> 
> when I try the subscription I get this one:
> 
> 
> # doveadm mailbox subscribe -u johnd "public/a & p products
> group/INBOX" doveadm(johnd): Error: Can't public/a & p products
> group/INBOX mailbox subscribe to: Invalid mailbox name 'public/a & p
> products group/INBOX': Missing namespace prefix 'public/a &- p
> products group/' 
> 
> 
> is there a way to work around this? (except for not to use usernames
> like this)
> 
> 
> 
> greetz matze

subscribe users mailbox with non-valid mutf7 chars

[Matthias Lay]

Hi all,

I got a question if its possible to subscribe a mailbox from another
user, who contains non mutf7 valid chars in his mailbox path.


I have a private namespace which can subscribe to mailboxes of a
shared namespace with public/ prefix

in this example
username in private namespace is johnd
public username is "a & p products group"


when I try the subscription I get this one:


# doveadm mailbox subscribe -u johnd "public/a & p products group/INBOX"
doveadm(johnd): Error: Can't public/a & p products group/INBOX mailbox
subscribe to: Invalid mailbox name 'public/a & p products group/INBOX':
Missing namespace prefix 'public/a &- p products group/'



is there a way to work around this? (except for not to use usernames
like this)



greetz matze

Re: My dovecot works fine against Active Directory 2003, but not against AD2008

[Matthias Lay]

Hi Fran,


this is not a dovecot problem, thats a pure dns problem and can only
be fixed in your dns environment.


referrals are propagated in a "special" dns design in SRV records.
so the ldap client performs a dns lookup for this names and this is the
point of hanging (as in most "hanging cases", its dns).

see:
https://technet.microsoft.com/en-us/library/cc978014.aspx
https://technet.microsoft.com/en-us/library/cc961719.aspx
http://www.mail-archive.com/cas@tp.its.yale.edu/msg00797.html

for information.


Greetz Matze




On Thu, 10 Sep 2015 13:10:57 +0200
Fran <cumc-436...@chguadalquivir.es> wrote:

> Hi Matthias,
> 
> thank you very much! that fixed the problem.
> 
> I had workaround the problem by using "base = ou=, dc=dom",
> instead of "base = dc=dom" in the dovecot-ldap.conf.ext file, because
> that also worked (I don't know why, but the problem happen if you use
> as base just the domain, but not if you add a second level). But that
> forced to me to use several userdb/passdb blocks definitions, one for
> each OU in which I have users, so I think that your fix is better.
> 
> I'm not able to understand the actual reason behind all this though...
> 
> What's the technical explanation behind this behaviour?? I mean, it
> seems to be that the problem is that the Domain controller (DC) was
> sending a "referrals" answer and dovecot auth made a connection to
> these others DC but something wrong happened (dovecot can't deal
> correctly with that kind of answers?? I don't know).
> 
> Anyways, as far as I know:
> 
> 1) A referral answer should be done by a DC when it can't provide the
> object that the client are requesting
> 2) REFERRALS off in ldap.conf means that the client should not follow
> referrals returned by the DC
> 
> So, if a referral answer is given from my DC, I think that is because
> such DC can't provide the object which the client is looking for, so,
> why works fine just by telling dovecot: "Don't follow referrals"?
> 
> Regards
> 
> 
> 
> El 09/09/2015 a las 17:22, Matthias Lay escribió:
> > hi,
> >
> > check your 
> >
> > /etc/openldap/ldap.conf
> >
> > for
> >
> > REFERRALS off
> >
> > I had this errors with "referrals on" in misconfigured dns
> > environments.
> >
> >
> > you can debug the dns packets by strace-ing the auth process
> >
> >
> >
> >
> > On Tue, 8 Sep 2015 11:00:37 +0200

Re: My dovecot works fine against Active Directory 2003, but not against AD2008

[Matthias Lay]

hi,

check your 

/etc/openldap/ldap.conf

for

REFERRALS off

I had this errors with "referrals on" in misconfigured dns environments.


you can debug the dns packets by strace-ing the auth process




On Tue, 8 Sep 2015 11:00:37 +0200
Fran  wrote:

> Hello,
> 
> my dovecot installation has been working fine against AD till we
> upgrade from AD 2003 to AD 2008. As
> http://wiki2.dovecot.org/AuthDatabase/LDAP said, now I'm not able to
> connect AD through 389 port. The port 3268 works fine though.
> 
> (...)
> Sep  7 19:02:05  dovecot: imap-login: Error:
> master(imap): Auth request timed out (received 0/12 bytes)
> Sep  7 19:02:05  dovecot: imap-login: Internal login
> failure (pid=4846 id=1) (internal failure, 1 successful auths):
> user=<>, method=PLAIN, rip=,
> lip=, TLS, session=
> (...)
> Sep  7 19:02:06  dovecot: auth: Error:
> ldap(,,): Connection appears
> to be hanging, reconnecting
> Sep  7 19:02:06  dovecot: auth: Error:
> ldap(,,): LDAP search
> returned multiple entries
> (...)
> 
> Is there a technical reason for this problem? Does it exist any
> workaround?
> 
> The use of Global Catalog (port 3268) is not a solution for me, since
> it misses many attributes. (ex. I use the field "initials" to set the
> quota and this field is not available through port 3268).
> 
> I also noticed that, now, it uses any DC available in the domain, it
> doesn't care what I configured in "hosts = " parameter.
> 
> This is using "hosts = dc03.domain:389":
> ---
> 
> [root@ ~]# netstat -anp | grep dovecot | grep auth
> tcp   22  0 :55217 
> :389  ESTABLISHED 4872/dovecot/auth
> tcp   22  0 :57645 
> :389ESTABLISHED 4872/dovecot/auth
> tcp0  0 :55216 
> :389  ESTABLISHED 4872/dovecot/auth
> 
> It looks like it does a look up for other domains controller (I don't
> know how nor why) and it connect aleatory to any DC in my domain (in
> this case dc06.domain, but it changes any time), additionally to the
> configured one (dc03.domain).
> 
> This is using "hosts = dc03.domain:3268":
> 
> [root@ ~]# netstat -anp | grep dovecot | grep auth
> tcp0  0 :58485 
> :3268 ESTABLISHED 4982/dovecot/auth
> 
> In this case, only the configured server in host parameter is used (I
> think this is the right behaviour)
> 
> 
> Aditional info:
> ---
> CentOS Linux release 7.0.1406 (Core)
> 
> dovecot 2.2.10
> 
> Build options: ioloop=epoll notify=inotify ipv6 openssl
> io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox
> cydir imapc pop3c raw fail SQL driver plugins: mysql postgresql sqlite
> Passdb: checkpassword ldap pam passwd passwd-file shadow sql
> Userdb: checkpassword ldap(plugin) nss passwd prefetch passwd-file sql
> 
> 
> My /etc/dovecot/dovecot-ldap.conf.ext
> --
> #hosts = dc03.domain:3268
> hosts = dc03.domain:389
> #uris = ldap://dc03.domain
> base = DC=domain
> #tls = yes
> tls = no
> ldap_version = 3
> auth_bind = yes
> auth_bind_userdn = %u@domain
> #auth_bind_userdn = DOMAIN\%u
> dn = cn=,cn=Users,dc=domain
> dnpass = 
> 
> #scope   = subtree
> #deref   = never
> 
> user_filter =
> (&(userPrincipalName=%u@domain)(objectClass=person)(|(mail=%u@)(othermailbox=%u@)))
> pass_filter =
> (&(userPrincipalName=%u@domain)(objectClass=person)(|(mail=%u@)(othermailbox=%u@)))
> pass_attrs  = userPassword=password
> user_attrs  = Initials=quota_rule=*:storage=%$MB
> ---
> 
> 
> --
> Log trace using PORT 389:
> --
> Sep  7 19:00:35  dovecot: imap-login: Debug: SSL:
> elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
> Sep  7 19:00:35  dovecot: imap-login: Debug: SSL:
> elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
> Sep  7 19:00:35  dovecot: imap-login: Debug: SSL:
> where=0x10, ret=1: before/accept initialization []
> Sep  7 19:00:35  dovecot: imap-login: Debug: SSL:
> where=0x2001, ret=1: before/accept initialization []
> Sep  7 19:00:35  dovecot: imap-login: Debug: SSL:
> where=0x2001, ret=1: SSLv3 read client hello A []
> Sep  7 19:00:35  dovecot: imap-login: Debug: SSL:
> where=0x2001, ret=1: SSLv3 write server hello A []
> Sep  7 19:00:35  dovecot: imap-login: Debug: SSL:
> where=0x2001, ret=1: SSLv3 write certificate A []
> Sep  7 19:00:35  dovecot: auth: Debug: Loading modules
> from directory: /usr/lib64/dovecot/auth
> Sep  7 19:00:35  dovecot: auth: Debug: Module loaded:
> /usr/lib64/dovecot/auth/libdriver_sqlite.so
> Sep  7 19:00:35  dovecot: auth: Debug: Loading modules
> from directory: /usr/lib64/dovecot/auth
> Sep  7 19:00:35  dovecot: auth: Debug: Module loaded:
> /usr/lib64/dovecot/auth/libauthdb_ldap.so
> Sep  7 19:00:35  dovecot: auth: Debug: Read auth token
> secret 

Re: question on autch cache parameters[Bug]

[Matthias Lay]

hi again,

On Thu, 27 Aug 2015 14:37:59 +0300
Teemu Huovila  wrote:


> 
> However, I am unable to reproduce this. Could you post your doveconf
> -n please? Im especially interested in your passdb and userdb
> configurations and auth-cache settings.
> 

just reproduced the bug with a fresh clean 2.2.18 install

ldap userdb an 2 masterusers with the ACL_GROUP attribut in passwd file


env output in imap-postlogin

first login:
AUTH_TOKEN=4adba75022f765fc3215ac5243337fd99adfdbf5
MASTER_USER=master2
SPUSER=private/johnd
LOCAL_IP=127.0.0.1
USER=johnd
AUTH_USER=master2
PWD=/run/dovecot
USERDB_KEYS=ACL_GROUPS HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER 
SHLVL=1
HOME=/home/vmail/private/johnd
ACL_GROUPS=umareadmaster
IP=127.0.0.1
_=/usr/bin/env


logout and next login:

AUTH_TOKEN=83d7ede27b4fbc4de2abad58e84e65ac1073e4ec
MASTER_USER=master2
SPUSER=private/johnd
LOCAL_IP=127.0.0.1
USER=johnd
AUTH_USER=master2
PWD=/run/dovecot
USERDB_KEYS=HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER 
SHLVL=1
HOME=/home/vmail/private/johnd
IP=127.0.0.1
_=/usr/bin/env


##
% doveconf -n:


# 2.2.18: /etc/dovecot/dovecot.conf
# OS: Linux 3.12.44-gentoo x86_64 Gentoo Base System release 2.2 
auth_cache_negative_ttl = 30 mins
auth_cache_size = 10 k
auth_master_user_separator = *
auth_use_winbind = yes
auth_username_chars = 
auth_verbose = yes
log_path = /var/log/dovecot.log
mail_gid = vmail
mail_home = /home/vmail/private/%u
mail_location = maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX
mail_uid = vmail
namespace {
  inbox = yes
  location = 
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  prefix = 
  separator = /
  subscriptions = yes
  type = private
}
namespace {
  hidden = no
  inbox = no
  list = children
  location =
maildir:/home/vmail/public/%%Lu/Maildir:LAYOUT=fs:INBOX=/home/vmail/public/%%Lu/Maildir/INBOX
prefix = public/%%u/ separator = /
  subscriptions = no
  type = shared
}
passdb {
  args = /etc/dovecot/master-users1
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/master-users2
  driver = passwd-file
  master = yes
}
service auth {
  unix_listener auth-client {
group = 
mode = 0600
user = $default_internal_user
  }
  unix_listener auth-login {
group = 
mode = 0600
user = $default_internal_user
  }
  unix_listener auth-master {
group = 
mode = 0600
user = $default_internal_user
  }
  unix_listener auth-userdb {
group = vmail
mode = 0660
user = $default_internal_user
  }
  unix_listener login/login {
group = 
mode = 0666
user = $default_internal_user
  }
  user = $default_internal_user
}
service imap-login {
  inet_listener imap {
port = 143
  }
}
service imap-postlogin {
  executable = script-login /usr/libexec/dovecot/imap-postlogin
  user = vmail
}
service imap {
  executable = imap imap-postlogin
}
ssl_cert = 

auth_master_user_separator = *

# Example master user passdb using passwd-file. You can use any passdb
though. passdb {
  driver = passwd-file
  master = yes
  args = /etc/dovecot/master-users1

  # Unless you're using PAM, you probably still want the destination
  user to # be looked up from passdb that it really exists. pass=yes
  does that. #pass = yes
}
passdb {
  driver = passwd-file
  master = yes
  args = /etc/dovecot/master-users2

  # Unless you're using PAM, you probably still want the destination
  user to # be looked up from passdb that it really exists. pass=yes
  does that. #pass = yes
}


###
% cat /etc/dovecot/master-users1

master1:{SHA}xxx=::userdb_acl_groups=umareadmaster
allow_nets=127.0.0.1




master2 is the same. 


Greetz

Re: question on autch cache parameters

[matthias lay]

hi teemu,

thx for your reply.

the user is a masteruser that hast a static passwd file. this is where
the ACL_GROUPS is applied


cat /etc/dovecot/passwd.masteruser 
master@uma:{SHA}ojN+jsbELZbRJeRb0qj9+MMjPUs=::userdb_acl_groups=umareadmaster
allow_nets=127.0.0.1
##

* the standard lookup method for users is ldap. only masterusers are in
static user/passdbs
* auth cache is enabled

I cant post my whole conf but will paste the parts you requested.

if its not enough for you to reproduce, I will setup a clean instance
and reproduce it there.


##
# 2.2.16: /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.7
# OS: Linux 3.15.10-dist i686  
auth_cache_negative_ttl = 30 mins
auth_cache_size = 10 k
auth_master_user_separator = *
 snip

default namespace: (maildir gets overwritten by ldap lookup on most
users)

namespace {
  hidden = no
  inbox = no
  list = children
  location =
maildir:/var/data/vmail/public/%%Lu/Maildir:LAYOUT=fs:INBOX=/var/data/vmail/public/%%Lu/Maildir/INBOX
prefix = public/%%u/ separator = /
  subscriptions = no
  type = shared
}

userdb {
  args = uid=vmail gid=vmail home=/var/data/vmail/public/%Ln
  driver = static
}



protocol imap {
  imap_client_workarounds = tb-extra-mailbox-sep
  mail_plugins = acl notify mailbox_alias imap_acl 
  ssl = yes 
  ssl_cert = /etc/ssl/certs/imap.cert
  ssl_key = /etc/ssl/private/imap.key
  userdb {
args = /etc/dovecot/dovecot-imap-ldap.conf.ext
driver = ldap
name = 
  }
}

##
content of dovecot-imap-ldap.conf.ext:
base = dc=spdev, dc=local

# Filter for user lookups.
user_filter =
((sAMAccountName=%Ln)(!(userAccountControl:1.2.840.113556.1.4.803:=2))((sAMAccountType=805306368)))

# User attributes are given in LDAP-name=dovecot-internal-name list.
user_attrs = name=home=/var/data/vmail/private/%Ln,
=spuser=private/%L{ldap:sAMAccountName}


several masteruser passdbs

passdb {
  args = /etc/dovecot/passwd.masteruser
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/passwd.system
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/passwd.email-shredder
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/passwd.imap-set-del-flag
  driver = passwd-file
  master = yes
}

passdb {
  args = /etc/dovecot/dovecot-ldap-passdb.conf.ext
  driver = ldap
}
# contents of ldap-passdb.conf
# LDAP base.
base = dc=spdev, dc=local

# Filter for user lookups.
user_filter =
((sAMAccountName=%Ln)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))




protocols = imap
service auth {
  unix_listener auth-client {
group = 
mode = 0600
user = $default_internal_user
  }
  unix_listener auth-login {
group = 
mode = 0600
user = $default_internal_user
  }
  unix_listener auth-master {
group = 
mode = 0600
user = $default_internal_user
  }
  unix_listener auth-userdb {
group = vmail
mode = 0660
user = $default_internal_user
  }
  unix_listener login/login {
group = 
mode = 0666
user = $default_internal_user
  }
  user = $default_internal_user
}

##


We use a global ACL file. where masterusers have different rights and
the one mentioned is the only one that gets a ACL_GROUP in passdb.


my guess: it might be related to the several masteruser passdbs point.
but thats just a guess from the outside ;)



hope this helps, if not let me know



greetz Matze






On Thu, 27 Aug 2015 14:37:59 +0300
Teemu Huovila teemu.huov...@dovecot.fi wrote:

 Hello
 
 Thank you for your report. We really appreciate it, especially when
 you can pinpoint a commit.
 
 However, I am unable to reproduce this. Could you post your doveconf
 -n please? Im especially interested in your passdb and userdb
 configurations and auth-cache settings.
 
 br,
 Teemu Huovila
 
 
 On 08/06/2015 01:07 PM, matthias lay wrote:
  hi timo,
  
  I checked out the commit causing this.
  
  its this one:
  
  http://hg.dovecot.org/dovecot-2.2/diff/5e445c659f89/src/auth/auth-request.c#l1.32
  
  
  if I move this block back as it was. everything is fine
  
  
  diff -r a46620d6e0ff -r 5e445c659f89 src/auth/auth-request.c
  --- a/src/auth/auth-request.c   Tue May 05 13:35:52 2015 +0300
  +++ b/src/auth/auth-request.c   Tue May 05 14:16:31 2015 +0300
  @@ -618,30 +627,28 @@
 auth_request_want_skip_passdb(request, next_passdb))
  next_passdb = next_passdb-next;
  
  +   if (*result == PASSDB_RESULT_OK) {
  +   /* this passdb lookup succeeded, preserve its
  extra fields */
  +   auth_fields_snapshot(request-extra_fields);
  +   request-snapshot_have_userdb_prefetch_set =
  +   request-userdb_prefetch_set

Re: PATCH for Stephan: Re: pigeonhole/lda accessing -m folder

[matthias lay]

hi stephan,

On 08/10/2015 05:05 PM, Stephan Bosch wrote:
 
 I'll give this a look. Also, are there any other environment items that
 may be useful?


when I checked the environment plugin I was wondering cuz it was not
what I was thinking it would be, .before reading the RFC ;)

would it be a bad idea to have a generic environment extension working
with getenv() to be able to get all kind of self defined, prefixed
variables in program environment?


Greetz Matze

Re: pigeonhole/lda accessing -m folder

[matthias lay]

complete patch. some parts were missing before


diff --git a/src/lib-sieve/plugins/environment/ext-environment-common.c
b/src/lib-sieve/plugins/environment/ext-environment-common.c
--- a/src/lib-sieve/plugins/environment/ext-environment-common.c
+++ b/src/lib-sieve/plugins/environment/ext-environment-common.c
@@ -24,7 +24,8 @@ static const struct sieve_environment_item
*core_env_items[] = {
location_env_item,
phase_env_item,
name_env_item,
-   version_env_item
+   version_env_item,
+   defaultfolder_env_item
 };

 static unsigned int core_env_items_count = N_ELEMENTS(core_env_items);
@@ -233,7 +234,19 @@ const struct sieve_environment_item
version_env_item = {
.value = PIGEONHOLE_VERSION,
 };

+/* defaultfolder:
+ * the default folder where mail is stored when no rule matches
+ */
+static const char *envit_defaultfolder_get_value
+(struct sieve_instance *svinst,
+ const struct sieve_script_env *senv)
+{
+return senv-default_mailbox;
+}

-
+const struct sieve_environment_item defaultfolder_env_item = {
+.name = defaultfolder,
+.get_value = envit_defaultfolder_get_value,
+};


diff --git a/src/lib-sieve/plugins/environment/ext-environment-common.h
b/src/lib-sieve/plugins/environment/ext-environment-common.h
--- a/src/lib-sieve/plugins/environment/ext-environment-common.h
+++ b/src/lib-sieve/plugins/environment/ext-environment-common.h
@@ -38,6 +38,7 @@ extern const struct sieve_environment_item
location_env_item;
 extern const struct sieve_environment_item phase_env_item;
 extern const struct sieve_environment_item name_env_item;
 extern const struct sieve_environment_item version_env_item;
+extern const struct sieve_environment_item defaultfolder_env_item;

 /*
  * Initialization

















On 08/07/2015 04:03 PM, matthias lay wrote:
 fixed my problem.
 
 if somebody is interested.
 I added an variable to the environment extension, which is quite easy.
 And the value of the folder is already there. so this is all:
 
 
 src/lib-sieve/plugins/environment/ext-environment-common.c
 
 static const char *envit_spfolder_get_value
 (struct sieve_instance *svinst,
 const struct sieve_script_env *senv)
 {
 return senv-default_mailbox;
 }
 
 const struct sieve_environment_item spfolder_env_item = {
 .name = spfolder,
 .get_value = envit_spfolder_get_value,
 };
 
 
 after that the variable can easily be used in a script like
 
 if anyof ( environment :matches spfolder *) {
   set myfolder ${1};
 }
 
 
 
 
 
 
 
 
 
 On 08/07/2015 12:40 PM, matthias lay wrote:
 hi jost thx for the reply,

 by access I mean to read the variable


 require [fileinto, variables, ?destfolder? ];

 if anyof ( destfolder :matches *) {

 fileinto ${1}/subfolder;

 }else{

   fileinto INBOX/subfolder;

 }



 On 08/07/2015 12:26 PM, Jost Krieger wrote:
 On Fri Aug  7 12:19:22 2015, matthias lay wrote:
  
 anybody knows, if there is a way to acces the folder from a call like this

 deovecot-lda -m destfolder

 from within a sieve script?

 Depending on what you mean by access, this could be as easy as

  require fileinto;

  fileinto destfolder;
  
 Yours
 Jost Krieger

Re: pigeonhole/lda accessing -m folder

[matthias lay]

fixed my problem.

if somebody is interested.
I added an variable to the environment extension, which is quite easy.
And the value of the folder is already there. so this is all:


src/lib-sieve/plugins/environment/ext-environment-common.c

static const char *envit_spfolder_get_value
(struct sieve_instance *svinst,
const struct sieve_script_env *senv)
{
return senv-default_mailbox;
}

const struct sieve_environment_item spfolder_env_item = {
.name = spfolder,
.get_value = envit_spfolder_get_value,
};


after that the variable can easily be used in a script like

if anyof ( environment :matches spfolder *) {
set myfolder ${1};
}









On 08/07/2015 12:40 PM, matthias lay wrote:
 hi jost thx for the reply,
 
 by access I mean to read the variable
 
 
 require [fileinto, variables, ?destfolder? ];
 
 if anyof ( destfolder :matches *) {
 
 fileinto ${1}/subfolder;
 
 }else{
 
   fileinto INBOX/subfolder;
 
 }
 
 
 
 On 08/07/2015 12:26 PM, Jost Krieger wrote:
 On Fri Aug  7 12:19:22 2015, matthias lay wrote:
  
 anybody knows, if there is a way to acces the folder from a call like this

 deovecot-lda -m destfolder

 from within a sieve script?

 Depending on what you mean by access, this could be as easy as

  require fileinto;

  fileinto destfolder;
  
 Yours
 Jost Krieger

Re: pigeonhole/lda accessing -m folder

[matthias lay]

hi jost thx for the reply,

by access I mean to read the variable


require [fileinto, variables, ?destfolder? ];

if anyof ( destfolder :matches *) {

fileinto ${1}/subfolder;

}else{

  fileinto INBOX/subfolder;

}



On 08/07/2015 12:26 PM, Jost Krieger wrote:
 On Fri Aug  7 12:19:22 2015, matthias lay wrote:
  
 anybody knows, if there is a way to acces the folder from a call like this

 deovecot-lda -m destfolder

 from within a sieve script?
 
 Depending on what you mean by access, this could be as easy as
 
  require fileinto;
 
  fileinto destfolder;
  
 Yours
 Jost Krieger
 

Re: question on autch cache parameters

[matthias lay]

hi timo,

I checked out the commit causing this.

its this one:

http://hg.dovecot.org/dovecot-2.2/diff/5e445c659f89/src/auth/auth-request.c#l1.32


if I move this block back as it was. everything is fine


diff -r a46620d6e0ff -r 5e445c659f89 src/auth/auth-request.c
--- a/src/auth/auth-request.c   Tue May 05 13:35:52 2015 +0300
+++ b/src/auth/auth-request.c   Tue May 05 14:16:31 2015 +0300
@@ -618,30 +627,28 @@
   auth_request_want_skip_passdb(request, next_passdb))
next_passdb = next_passdb-next;

+   if (*result == PASSDB_RESULT_OK) {
+   /* this passdb lookup succeeded, preserve its extra fields */
+   auth_fields_snapshot(request-extra_fields);
+   request-snapshot_have_userdb_prefetch_set =
+   request-userdb_prefetch_set;
+   if (request-userdb_reply != NULL)
+   auth_fields_snapshot(request-userdb_reply);
+   } else {
+   /* this passdb lookup failed, remove any extra fields it set */
+   auth_fields_rollback(request-extra_fields);
+   if (request-userdb_reply != NULL) {
+   auth_fields_rollback(request-userdb_reply);
+   request-userdb_prefetch_set =
+   request-snapshot_have_userdb_prefetch_set;
+   }
+   }
+
if (passdb_continue  next_passdb != NULL) {
/* try next passdb. */
 request-passdb = next_passdb;
request-passdb_password = NULL;

-   if (*result == PASSDB_RESULT_OK) {
-   /* this passdb lookup succeeded, preserve its extra
-  fields */
-   auth_fields_snapshot(request-extra_fields);
-   request-snapshot_have_userdb_prefetch_set =
-   request-userdb_prefetch_set;
-   if (request-userdb_reply != NULL)
-   auth_fields_snapshot(request-userdb_reply);
-   } else {
-   /* this passdb lookup failed, remove any extra fields
-  it set */
-   auth_fields_rollback(request-extra_fields);
-   if (request-userdb_reply != NULL) {
-   auth_fields_rollback(request-userdb_reply);
-   request-userdb_prefetch_set =
-   
request-snapshot_have_userdb_prefetch_set;
-   }
-   }
-
if (*result == PASSDB_RESULT_USER_UNKNOWN) {
/* remember that we did at least one successful
   passdb lookup */














On 08/05/2015 05:33 PM, matthias lay wrote:
 just tested against dovecot 2.2.15
 
 everythings works fine. so might be a bug introduced between 2.2.16 and
 2.2.18
 
 
 
 
 
 On 08/05/2015 04:30 PM, matthias lay wrote:
 Hi list,

 I have a question on auth caching in 2.2.18.

 I am using acl_groups for a master user, appended in a static userdb file

 # snip ###
 master@uma:{SHA}=::userdb_acl_groups=umareadmaster
 allow_nets=127.0.0.1
 # snap ###

 and use this group in a global ACL file.
 I discovered this only works on first NOT-cached login



 environment in imap-postlogin script on first login:


 AUTH_TOKEN=e96b5a32ceb2cafc4460c210ad2e92e3d7ab388c
 MASTER_USER=master@uma
 SPUSER=private/pdf
 LOCAL_IP=127.0.0.1
 USER=pdf
 AUTH_USER=master@uma
 PWD=/var/run/dovecot
 USERDB_KEYS=ACL_GROUPS HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER
 SHLVL=1
 HOME=/var/data/vmail/private/pdf
 ACL_GROUPS=umareadmaster
 IP=127.0.0.1
 _=/usr/bin/env


 on the second cached login it looks like this


 AUTH_TOKEN=12703b11932f233520f6d4b33559c33aeb1cfc7f
 MASTER_USER=master@uma
 SPUSER=private/pdf
 LOCAL_IP=127.0.0.1
 USER=pdf
 AUTH_USER=master@uma
 PWD=/var/run/dovecot
 USERDB_KEYS=HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER
 SHLVL=1
 HOME=/var/data/vmail/private/pdf
 IP=127.0.0.1
 _=/usr/bin/env

 so the ACL_GROUPS is gone.

 is this intended to be like that.
 so groups not included in cache and I have to find another approach?

 anybody else encountered similar problems with some auth Variables and
 caching?


 Greetz Matze

 

question on autch cache parameters

[matthias lay]

Hi list,

I have a question on auth caching in 2.2.18.

I am using acl_groups for a master user, appended in a static userdb file

# snip ###
master@uma:{SHA}=::userdb_acl_groups=umareadmaster
allow_nets=127.0.0.1
# snap ###

and use this group in a global ACL file.
I discovered this only works on first NOT-cached login



environment in imap-postlogin script on first login:


AUTH_TOKEN=e96b5a32ceb2cafc4460c210ad2e92e3d7ab388c
MASTER_USER=master@uma
SPUSER=private/pdf
LOCAL_IP=127.0.0.1
USER=pdf
AUTH_USER=master@uma
PWD=/var/run/dovecot
USERDB_KEYS=ACL_GROUPS HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER
SHLVL=1
HOME=/var/data/vmail/private/pdf
ACL_GROUPS=umareadmaster
IP=127.0.0.1
_=/usr/bin/env


on the second cached login it looks like this


AUTH_TOKEN=12703b11932f233520f6d4b33559c33aeb1cfc7f
MASTER_USER=master@uma
SPUSER=private/pdf
LOCAL_IP=127.0.0.1
USER=pdf
AUTH_USER=master@uma
PWD=/var/run/dovecot
USERDB_KEYS=HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER
SHLVL=1
HOME=/var/data/vmail/private/pdf
IP=127.0.0.1
_=/usr/bin/env

so the ACL_GROUPS is gone.

is this intended to be like that.
so groups not included in cache and I have to find another approach?

anybody else encountered similar problems with some auth Variables and
caching?


Greetz Matze

Re: question on autch cache parameters

[matthias lay]

just tested against dovecot 2.2.15

everythings works fine. so might be a bug introduced between 2.2.16 and
2.2.18





On 08/05/2015 04:30 PM, matthias lay wrote:
 Hi list,
 
 I have a question on auth caching in 2.2.18.
 
 I am using acl_groups for a master user, appended in a static userdb file
 
 # snip ###
 master@uma:{SHA}=::userdb_acl_groups=umareadmaster
 allow_nets=127.0.0.1
 # snap ###
 
 and use this group in a global ACL file.
 I discovered this only works on first NOT-cached login
 
 
 
 environment in imap-postlogin script on first login:
 
 
 AUTH_TOKEN=e96b5a32ceb2cafc4460c210ad2e92e3d7ab388c
 MASTER_USER=master@uma
 SPUSER=private/pdf
 LOCAL_IP=127.0.0.1
 USER=pdf
 AUTH_USER=master@uma
 PWD=/var/run/dovecot
 USERDB_KEYS=ACL_GROUPS HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER
 SHLVL=1
 HOME=/var/data/vmail/private/pdf
 ACL_GROUPS=umareadmaster
 IP=127.0.0.1
 _=/usr/bin/env
 
 
 on the second cached login it looks like this
 
 
 AUTH_TOKEN=12703b11932f233520f6d4b33559c33aeb1cfc7f
 MASTER_USER=master@uma
 SPUSER=private/pdf
 LOCAL_IP=127.0.0.1
 USER=pdf
 AUTH_USER=master@uma
 PWD=/var/run/dovecot
 USERDB_KEYS=HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER
 SHLVL=1
 HOME=/var/data/vmail/private/pdf
 IP=127.0.0.1
 _=/usr/bin/env
 
 so the ACL_GROUPS is gone.
 
 is this intended to be like that.
 so groups not included in cache and I have to find another approach?
 
 anybody else encountered similar problems with some auth Variables and
 caching?
 
 
 Greetz Matze
 



0x7BCC653A.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: ldap attribute modifiers. how to lowercase non ASCII

[matthias lay]

update of investigation...

I encountered a different behavior in %Lu and %L{ldap:uid} when doing
the search with lower cased user.

# doveadm user blätula

field   value
uid vmail
gid vmail
home/var/data/vmail/private/blätula
mailmaildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX
spuser  private/blÄtula


so in this case. the name gets lowered in the %Lu but not the
%L{ldap:uid} case.

- The user in ldap is all upper case.
- system locale is en_US.UTF-8


log of the above doveadm query:


mail.debug: Jul 29 12:45:28 dovecot: auth: Debug: master in: USER   1
blätula service=doveadm
mail.debug: Jul 29 12:45:28 dovecot: auth: Debug: ldap(blätula): userdb
cache miss
mail.debug: Jul 29 12:45:28 dovecot: auth: Debug: ldap(blätula): user
search: base=ou=users, dc=uma,dc=local scope=subtree
filter=((objectClass=inetOrgPerson)(uid=blätula)) fields=uid,uid
mail.debug: Jul 29 12:45:28 dovecot: auth: Debug: ldap(blätula): result:
uid=BLÄTULA; uid unused
mail.debug: Jul 29 12:45:28 dovecot: auth: Debug: ldap(blätula): result:
uid=BLÄTULA
mail.debug: Jul 29 12:45:28 dovecot: auth: Debug: userdb out: USER  1
blätula home=/var/data/vmail/private/blätulaspuser=private/blÄtula






On 07/28/2015 05:15 PM, matthias lay wrote:
 hi all,
 
 I encountered some problem with lowercasing chars like an Ä
 
 with user_attrs like this:
 
 user_attrs = uid=home=/mail/%Lu, =myvar=%L{ldap:uid}
 
 the Ä doesnt get lower cased in both cases
 
 
 # doveadm user BLÄTULA
 field value
 user  blÄtula
 uid   vmail
 gid   vmail
 home  /mail/blÄtula
 mail  maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX
 myvar blÄtula
 
 
 somebody knows if there a fix for that?
 
 
 Greetz Matze
 

ldap attribute modifiers. how to lowercase non ASCII

[matthias lay]

hi all,

I encountered some problem with lowercasing chars like an Ä

with user_attrs like this:

user_attrs = uid=home=/mail/%Lu, =myvar=%L{ldap:uid}

the Ä doesnt get lower cased in both cases


# doveadm user BLÄTULA
field   value
userblÄtula
uid vmail
gid vmail
home/mail/blÄtula
mailmaildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX
myvar   blÄtula


somebody knows if there a fix for that?


Greetz Matze

Re: [Dovecot] lda+ldap multiple users

[matthias lay]

Hi Bob,

On 02/14/2014 07:36 PM, Bob Miller wrote:

I documented my sandbox qmail-ldap/dovecot system here, maybe it is
useful to you:

http://cocnm.computerisms.ca/index.php/Install_Qmail-ldap,_Dovecot,_and_Related_Email_Services




thx for your help and support, but changing stuff in qmail is not what I 
really want.
in my case the system is not the real Mailserver its just an archieve 
sitting in between an smtp proxy and the real server, capturing just the 
mails I want, but deliver *everything* to the real server.


so not much to do on the qmail side. and thats the reason everything 
user related is done in dovecot itself.

we do passwd lookups and user lookups only within dovecot. Filter looks like


user_filter = 
((|(objectClass=msExchExchangeServerRecipient)(|(objectClass=group)))(|(proxyAddresses=smtp:%Ln@%Ld)(|(mail=%Ln@%Ld



...but if nobody knows if its possible inside the dovecot lookup 
process, a new/seperate lookup process that calls dovecot-lda with the 
username instead of the mailaddress might be the only opportunity.


Greetz
Matze

Re: [Dovecot] lda+ldap multiple users

[matthias lay]

On 02/14/2014 08:27 AM, Steffen Kaiser wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 7 Feb 2014, matthias lay wrote:

I experienced that if a Mailaddress matches several users the 
delivery is aborted.



dovecot: auth: Error: ldap(christian.t...@securepoint.de): LDAP 
search returned multiple entries

dovecot: auth: ldap(christian.t...@securepoint.de): unknown user
dovecot: lda: Error: user christian.t...@securepoint.de: Auth USER 
lookup failed

-

now my question, is there a way to have a mail like that delivered to 
all users that matches the lookup?

havent found anything in the docs.


This is a job of your MTA.



Hi Steffen,

MTA is qmail and doesnt know anything about users. dovecot uses 
usernames for mailboxes. lets say


/var/mail/userA/Maildir
/var/mail/userB/Maildir

both users have mailaddr x...@example.com mapped in Active Directory. Now 
when mails arrive lda is called like


dovecot-lda -d x...@example.com mailto:dafan.z...@securepoint.de -m INBOX

and theres my problem.

[Dovecot] lda+ldap multiple users

[matthias lay]

Hi list and timo,


I use dovecot lda with ldap to do a email = user lookup.

I experienced that if a Mailaddress matches several users the delivery 
is aborted.



dovecot: auth: Error: ldap(christian.t...@securepoint.de): LDAP search 
returned multiple entries

dovecot: auth: ldap(christian.t...@securepoint.de): unknown user
dovecot: lda: Error: user christian.t...@securepoint.de: Auth USER 
lookup failed

-

now my question, is there a way to have a mail like that delivered to 
all users that matches the lookup?

havent found anything in the docs.


thx in advance

Matze

Re: [Dovecot] ntlm auth / usernames with white spaces

[Matthias Lay]

works the same way! at least for ntlm but should do the same for spnego.

thx. didnt see your strarray join function.


On Tue, 2012-07-31 at 22:10 +0300, Timo Sirainen wrote:
 On 27.7.2012, at 17.32, Matthias Lay wrote:
 
  we experienced some problems with white spaces in usernames (this was
  NOT our idea) authenticated bei dovecots ntlmhelper
  
  I wrote a little patch fixing this, if you would like.
  
  patch applies on 2.1.7 and 2.1.8 (tested those)
 
 Does this simpler patch also work?

[Dovecot] authenticate plain and utf-8 with special chars

[Matthias Lay]

Hi,

I experienced some problems with authenticate. 

seems like the mail clients like thunderbird send their base64 string in
ISO- encoding, which doesnt work on my dovecot setup if there are some
Umlauts

I am not able to login as 
 ömer for example


If I authenticate from telnet with an base64 encoded utf-8 string like
 'echo -en \0ömer\0Start12 |base64'
everything works fine


-dovecot 2.1.7
-userdb is an openldap which queries an windows AD so results are in
utf8, I think.

any clues at which point to fix this are welcome ;)

[Dovecot] ntlm auth / usernames with white spaces

[Matthias Lay]

Hi,

we experienced some problems with white spaces in usernames (this was
NOT our idea) authenticated bei dovecots ntlmhelper

I wrote a little patch fixing this, if you would like.

patch applies on 2.1.7 and 2.1.8 (tested those)


diff -up ./src/auth/mech-winbind.c.username ./src/auth/mech-winbind.c
--- ./src/auth/mech-winbind.c.username  2012-07-27 09:52:33.422553807
+0200
+++ ./src/auth/mech-winbind.c   2012-07-27 14:23:03.718619388 +0200
@@ -239,8 +239,16 @@ do_auth_continue(struct auth_request *au
return HR_FAIL;
} else if (strcmp(token[0], AF) == 0) {
const char *user, *p, *error;
+   int ti=1;
+   
+   if(gss_spnego)
+ti=2;
+   /* some braindead administrators like spaces in usernames */
+   user = token[ti];
+   while(token[++ti]){
+user = t_strconcat(user,  , token[ti], NULL);
+   }
 
-   user = gss_spnego ? token[2] : token[1];
i_assert(user != NULL);
 
p = strchr(user, '\\');