Re: Sieve body test

2020-10-22 Thread Stephan Bosch 




On 20/10/2020 23:37, @lbutlr wrote:

On 20 Oct 2020, at 13:46, @lbutlr  wrote:

It looks like what I need to do is enable and use vnd.dovecot.filter

error: require command: unknown Sieve capability `vnd.dovecot.filter'.

Is this not available to a user? I guess I can put the in global, but ick.


You need to include the extprograms plugin:

https://doc.dovecot.org/configuration_manual/sieve/plugins/extprograms/

The language extension is described here:

https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-extprograms.txt

Regards,

Stephan.

Re: Sieve_before

2020-10-22 Thread Sean Kamath 



> On Oct 22, 2020, at 15:58, @lbutlr  wrote:
> 
> On 22 Oct 2020, at 15:46, @lbutlr  wrote:
>> And it doesn't explain why "if address :matches ["To", "Cc"] ["*@*."] {" 
>> also failed to match despite also showing the email address.
>> 
>>> “krem...@kreme.com” does not have a :detail part 
>>> (https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples talks about this).
>> 
>> And I am not asking for :detail
> 
> OK, fair enough. I looked at that line many times for some reason.
> But the first point still stand.

Does you To or CC address end with a .?

One problem I had with :matches (if I recall correctly, I can’t find where I 
ran into this) is that :matches has an implicit “^” and “$” wrapped around what 
you’re looking for.

“some random string”

then trying to :match “random” won’t match that string.  “*random*” *will* 
match (as will “some*” and “*string”).

Like I said, I’m not 100% sure, but I suspect if you add * to the end:

if address :matches ["To", "Cc"] ["*@*.*"] {

It might work.

Sean

Re: Sieve_before

2020-10-22 Thread @lbutlr 
On 22 Oct 2020, at 15:46, @lbutlr  wrote:
> And it doesn't explain why "if address :matches ["To", "Cc"] ["*@*."] {" also 
> failed to match despite also showing the email address.
> 
>> “krem...@kreme.com” does not have a :detail part 
>> (https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples talks about this).
> 
> And I am not asking for :detail

OK, fair enough. I looked at that line many times for some reason.
But the first point still stand.


-- 
For my birthday I got a humidifier and a de-humidifier... I put them
in the same room and let them fight it out.

Directors and LMTP: mailAlternateAddress of same user

2020-10-22 Thread Paterakis E. Ioannis 



Good evening everyone,

I'd like to paste a problem i experience the last days with a mail 
cluster system based on three dovecot servers and two directors in front 
of them. The Directors are proxying the auth requests to the dovecot 
servers below (LDAP). The protocols i use are imap/managesieve/lmtp.  
The setup works as intented, exept one detail. There are a lot of users 
that happen to have mailAlternateAddress attriutes with different e-mail 
addresses. The problem is that when a mail arrives to the lmtp of the 
directors, and the recipient is one of the mailalternateAddresses the 
same user has, director/lmtp doesn't recognise the real user of that 
e-mail address, and routes the e-mail to a different dovecot server than 
the real user is logged in, resulting in a corrupted .index.cache file.


I've searched around for a solution to this, and figured out that a 
special userdb {} section in the protocol lmtp {} section, with driver = 
ldap etc etc so that i 'd be able to "show" the correct user to the lmtp 
might work, but strangely that userdb {} setting looks like being 
ignored by lmtp.


Has any of you guys have an idea how will i be able to solve that issue 
because i get thousands of corrupted .index.cache files every day



Thank you very much in advance,

Ioannis

Re: Sieve_before

2020-10-22 Thread @lbutlr 
On 22 Oct 2020, at 15:10, Sean Kamath  wrote:
> On Oct 22, 2020, at 12:19, @lbutlr  wrote:
>> On 22 Oct 2020, at 10:05, @lbutlr  wrote:
>>> if address :matches ["To", "Cc"] ["*@*."] {

>>  if envelope :matches :detail "to" "*" {
>> set :lower "name" "${1}";
>> redirect :copy "backup+295.${name}@@*example.com*";
>>   }
>> 
>>  ## Started executing script 'bcc'
>>   5: envelope test
>>   5:   starting `:matches' match with `i;ascii-casemap' comparator:
>>   5:   getting `to' part from message envelope
>>   5:   extracting `detail' part from address 
>>   5:   finishing match with result: not matched
>>   5: jump if result is false
>>   5:   jumping to line 7
>>  ## Finished executing script 'bcc'

> If I read that bcc trace, it says “extracting ‘detail’ part from address 
> ”.

OK, but that exact line is used in several examples online to get the email 
address.

And it doesn't explain why "if address :matches ["To", "Cc"] ["*@*."] {" also 
failed to match despite also showing the email address.

> “krem...@kreme.com” does not have a :detail part 
> (https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples talks about this).

And I am not asking for :detail, nor am I trying to the the address 
extension/plus address.

> If it were “kremels+th...@kreme.com” then the net line out to say “finished 
> match with result: ‘thing’” or some such.

It should not because I am not asking for the detail, I am only asking for 
:matches "*"

-- 
"Are you pondering what I'm pondering?"
"Well, I think so, Brain, but I can't memorize a whole opera in
Yiddish."

Re: Sieve_before

2020-10-22 Thread Sean Kamath 


> On Oct 22, 2020, at 12:19, @lbutlr  wrote:
> 
> On 22 Oct 2020, at 10:05, @lbutlr  wrote:
>> require ["variables", "copy"];
>> 
>> if address :matches ["To", "Cc"] ["*@*."] {
>>  redirect :copy "backup+295.${1}.${2}@*example.com*";
>> }
> 
> I have tried this with similar results:
> 
> require ["copy", "variables", "envelope", "fileinto", "subaddress"];
> 
>   if envelope :matches :detail "to" "*" {
> set :lower "name" "${1}";
> redirect :copy "backup+295.${name}@@*example.com*";
>   }
> 
>  ## Started executing script 'bcc'
>   5: envelope test
>   5:   starting `:matches' match with `i;ascii-casemap' comparator:
>   5:   getting `to' part from message envelope
>   5:   extracting `detail' part from address 
>   5:   finishing match with result: not matched
>   5: jump if result is false
>   5:   jumping to line 7
>  ## Finished executing script 'bcc'
> 
> So why are these not matching? I can see the 'detail' right there, and this 
> matches examples that I see online.
> 
> (I've read, I think, every page google or DDG can find that contains 
> "redirect :copy" to no avail).
> 

If I read that bcc trace, it says “extracting ‘detail’ part from address 
mailto:krem...@kreme.com>>”.

“krem...@kreme.com ” does not have a :detail part 
(https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples 
 talks about this).

If it were “kremels+th...@kreme.com ” then the 
net line out to say “finished match with result: ‘thing’” or some such.

My notes from my sieve file:

# :localpart = :user+:detail
# address are :localpart @ :domain, or :user + :detail @ :domain
# See RFC5233

Sean

Re: Sieve_before

2020-10-22 Thread @lbutlr 
On 22 Oct 2020, at 10:05, @lbutlr  wrote:
> require ["variables", "copy"];
> 
> if address :matches ["To", "Cc"] ["*@*."] {
>   redirect :copy "backup+295.${1}.${2}@*example.com*";
> }

I have tried this with similar results:

require ["copy", "variables", "envelope", "fileinto", "subaddress"];

   if envelope :matches :detail "to" "*" {
 set :lower "name" "${1}";
 redirect :copy "backup+295.${name}@@*example.com*";
   }

  ## Started executing script 'bcc'
   5: envelope test
   5:   starting `:matches' match with `i;ascii-casemap' comparator:
   5:   getting `to' part from message envelope
   5:   extracting `detail' part from address 
   5:   finishing match with result: not matched
   5: jump if result is false
   5:   jumping to line 7
  ## Finished executing script 'bcc'

So why are these not matching? I can see the 'detail' right there, and this 
matches examples that I see online.

(I've read, I think, every page google or DDG can find that contains "redirect 
:copy" to no avail).


-- 
I'm no psychologist (although I play one when I'm picking up chicks
over by the asylum)

Re: Sieve_before

2020-10-22 Thread @lbutlr 
On 22 Oct 2020, at 09:35, @lbutlr  wrote:
> Is there a way to force the default and sieve_before scripts to log to syslog?

OK, the "top level" sieves are also logged to the users trace files along with 
the user's sieve files, if any. So, I am getting this:

   3: address test
   3:   starting `:matches' match with `i;ascii-casemap' comparator:
   3:   extracting `To' headers from message
   3:   parsing address header value `@lbutlr '
   3:   address value `krem...@kreme.com'
   3:   extracting `all' part from address 
   3:   matching value `krem...@kreme.com'
   3: with key `*@*.' => 0
   3:   extracting `Cc' headers from message
   3:   finishing match with result: not matched
   3: jump if result is false
   3:   jumping to line 4
  ## Finished executing script 'bcc'

bcc.sieve:
require ["variables", "copy"];

if address :matches ["To", "Cc"] ["*@*."] {
   redirect :copy "backup+295.${1}.${2}@*example.com*";
}

I've obviously done something wrong there, but what? I don't think 'redirect' 
has any required plugins (or at least I could find no mention of one, maybe 
it's implied to be part of fileinto?

-- 
Cause love's such an old fashioned word
And love dares you to care for the people on the
Edge of the night, and love dares you to change
Our way of caring about ourselves,
This is our last dance This is ourselves.
Under Pressure.

Re: Sieve_before

2020-10-22 Thread @lbutlr 
On 21 Oct 2020, at 14:10, @lbutlr  wrote:
> I have trace logs enabled for user scripts, but I think only error get logged 
> for upper level, and only along the lines of "could not compile  sieve>".

Any ideas?

Is there a way to force the default and sieve_before scripts to log to syslog?

-- 
The real world was far too real to leave neat little hints. It was
full of too many things. It wasn't by eliminating the impossible
that you got at the truth, however improbable; it was by the much
harder process of eliminating the possibilities. --Feet of Clay

net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied

2020-10-22 Thread mj 

Hi,

We are getting very occasional messags from dovecot:


net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied


Over the last week, the message appeared five times. (on a mail server 
with over 100 users, to that's basically almost never)


doveconf -n below


# 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.4 ()
# OS: Linux 4.19.0-10-amd64 x86_64 Debian 10.6 xfs


snip...


service stats {
  unix_listener stats-reader {
group = vmail
mode = 0660
user = vmail
  }
  unix_listener stats-writer {
group = vmail
mode = 0660
user = vmail
  }
}


and the on-disk permissions are:


root@dovecot:~# ls -l /var/run/dovecot/*stat*
srw--- 1 root  root  0 Oct  6 00:25 /var/run/dovecot/old-stats
prw--- 1 root  root  0 Oct  6 00:25 /var/run/dovecot/old-stats-mail
prw--- 1 root  root  0 Oct  6 00:25 /var/run/dovecot/old-stats-user
srw-rw 1 vmail vmail 0 Oct  6 00:25 /var/run/dovecot/stats-reader
srw-rw 1 vmail vmail 0 Oct  6 00:25 /var/run/dovecot/stats-writer


We're not sure what makes the Permission denied error happen...

Anyone with an idea?

MJ

Error: Mailbox INBOX: Transaction commit failed: FTS transaction commit failed: transaction context

2020-10-22 Thread Graham Leggett 
Hi all,

While trying to reindex a large mailbox to take advantage of Apache Tika, 
dovecot logs the following message:

Error: Mailbox INBOX: Transaction commit failed: FTS transaction commit failed: 
transaction context

The message is vague in that a transaction has failed, but nowhere does it say 
what the transaction boundaries are.

Did the transaction affect the indexing of one message, or many messages?

Did the indexing of the whole mailbox get aborted and rolled back, or just a 
single message?

A wide question. How do I ensure that my shiny new dovecot index gets 
completely reindexed? How do I know if dovecot is indexing just the new 
messages, and ignoring the existing messages?

Regards,
Graham
—

Re: Best prometheus exporter for Dovecot 2.3 with the new metrics/stats module

2020-10-22 Thread Aki Tuomi 


> On 22/10/2020 12:18 Sidsel Jensen  wrote:
> 
> 
> Hi ppl.
> 
> I am trying to locate the best prometheus exporter for use with Dovecot 2.3 
> in connection with the new metrics/stats module.
> Any pointers in the right direction will be appreciated - seems like there is 
> a bunch out there which all works with the old stats module.
> 
> Anybody have something “in the works” for 2.3 ?
> 
> Kind Regards,
> Sidsel Jensen
> 
>

I'd say the best is the one in-built to dovecot. 
https://doc.dovecot.org/configuration_manual/stats/openmetrics/

Aki

Best prometheus exporter for Dovecot 2.3 with the new metrics/stats module

2020-10-22 Thread Sidsel Jensen 
Hi ppl.

I am trying to locate the best prometheus exporter for use with Dovecot 2.3 in 
connection with the new metrics/stats module.
Any pointers in the right direction will be appreciated - seems like there is a 
bunch out there which all works with the old stats module.

Anybody have something “in the works” for 2.3 ?

Kind Regards,
  Sidsel Jensen









signature.asc
Description: Message signed with OpenPGP

Re: Indexer error after upgrade to 2.3.11.3

2020-10-22 Thread John Fawcett 
On 21/10/2020 19:00, John Fawcett wrote:
> On 21/10/2020 16:44, Patrik Peng wrote:
>> On 16.10.20 18:34, Patrik Peng wrote:
>>> On 16.10.20 18:00, Scott Q. wrote:
 This reminds me, the way I was able to reproduce this consistently
 was by having large headers ( 100+ lines ).


 On Friday, 16/10/2020 at 11:49 Patrik Peng wrote:

 On 19.08.20 17:37, Josef 'Jeff' Sipek wrote:

> On Wed, Aug 19, 2020 at 17:03:57 +0200, Alessio Cecchi wrote:
>> Hi,
>> after the upgrade to Dovecot 2.3.11.3, from 2.3.10.1, I see 
>> frequently 
>> these errors from different users:
> It looks like this has been around for a while and you just got 
> unlucky and
> started seeing this now.  Here's a quick & dirty patch that should 
> fix this.
> If you can try it, let us know how it went.
> Jeff.
>
> diff --git a/src/plugins/fts-solr/solr-connection.c 
> b/src/plugins/fts-solr/solr-connection.c
> index 
> ae720b5e2870a852c1b6c440939e3c7c0fa72b5c..9d364f93e2cd1b716b9ab61bd39656a6c5b1ea04
>  100644
> --- a/src/plugins/fts-solr/solr-connection.c
> +++ b/src/plugins/fts-solr/solr-connection.c
> @@ -103,7 +103,7 @@ int solr_connection_init(const struct 
> fts_solr_settings *solr_set,
>   http_set.ssl = ssl_client_set;
>   http_set.debug = solr_set->debug;
>   http_set.rawlog_dir = solr_set->rawlog_dir;
> - solr_http_client = http_client_init(_set);
> + solr_http_client = http_client_init_private(_set);
>   }
>   *conn_r = conn;
> diff --git a/src/plugins/fts/fts-parser-tika.c 
> b/src/plugins/fts/fts-parser-tika.c
> index 
> a4b8b5c3034f57e22e77caa759c090da6b62f8ba..b8b57a350b9a710d101ac7ccbcc14560d415d905
>  100644
> --- a/src/plugins/fts/fts-parser-tika.c
> +++ b/src/plugins/fts/fts-parser-tika.c
> @@ -77,7 +77,7 @@ tika_get_http_client_url(struct mail_user *user, 
> struct http_url **http_url_r)
>   http_set.request_timeout_msecs = 60*1000;
>   http_set.ssl = _set;
>   http_set.debug = user->mail_debug;
> - tika_http_client = http_client_init(_set);
> + tika_http_client = http_client_init_private(_set);
>   }
>   *http_url_r = tuser->http_url;
>   return 0;

 Greetings

 I'm also experiencing these issues while running Dovecot
 2.3.11.3 with Solr 8.6.3 on FreeBSD 11.4. As mentioned in a
 previous mail, the above patch is already applied to Dovecot's
 FreeBSD Port, confirmed by the patches being present in the
 portstree
 (https://svnweb.freebsd.org/ports/branches/2020Q3/mail/dovecot/files/).

 In a FreeBSD VM with the official image
 
 (https://download.freebsd.org/ftp/releases/VM-IMAGES/12.1-RELEASE/amd64/Latest/)
 I compiled dovecot from git and was able to reproduce the error
 with the patch mentioned above applied and also without any
 patches at all. From these results i conclude, that neither the
 patches applied in FreeBSDs portstree or the patch above have
 any influence.

 I also managed to reproduce the same results on a Debian 10
 machine (also with and without the patch):

 doveadm(some.u...@example.com): Panic: file http-client-request.c: 
 line 1232 (http_client_request_send_more): assertion failed: 
 (req->payload_input != NULL)
 doveadm(some.u...@example.com): Error: Raw backtrace: 
 /usr/local/lib/dovecot/libdovecot.so 
 .0(backtrace_append+0x42) [0x7f093f7fc3c2]
 -> /usr/local/lib/dovecot/libdovecot.so 
 .0(backtrace_get+0x1e) [0x7f093f7fc4ce] -> 
 /usr/local/lib/dovecot/libdovecot.so .0(+0xea341) 
 [0x7f093f807341]
 -> /usr/local/lib/dovecot/libdovecot.so 
 .0(+0xea381) [0x7f093f807381] -> 
 /usr/local/lib/dovecot/libdovecot.so .0(i_fatal+0) 
 [0x7f093f75c074]
 -> /usr/local/lib/dovecot/libdovecot.so 
 .0(http_client_request_send_more+0x378) 
 [0x7f093f7a47a8]
 -> /usr/local/lib/dovecot/libdovecot.so 
 .0(http_client_connection_output+0xe4) 
 [0x7f093f7a90f4]
 -> /usr/local/lib/dovecot/libssl_iostream_openssl.so 
 (+0x8bff) [0x7f093ec71bff]
 -> /usr/local/lib/dovecot/libdovecot.so 
 .0(+0x1148b0) [0x7f093f8318b0]
 -> /usr/local/lib/dovecot/libdovecot.so 
 .0(io_loop_call_io+0x69) [0x7f093f820259]
 ->

Re: Testing with imaptest to non existing folder makes imap service crash

2020-10-22 Thread Aki Tuomi 


> On 21/10/2020 22:09 Marc Roos  wrote:
> 
>  
> FWIIW I had this message still in drafts
> 
> /usr/bin/imaptest - append=100,0 logout=0 host=xxx port=143 user=xxx 
> pass=xxx  seed=100 secs=1 clients=1 mbox=64kb.mbox box=inbox/test 
> msgs=10
> 
> And inbox/test does not exist, results in:
> 
>  Fatal: master: service(imap): child 20282 killed with signal 11 (core 
> dumped)

Can you provide the core file using dovecot-sysreport tool?

https://raw.githubusercontent.com/dovecot/core/master/src/util/dovecot-sysreport

dovecot-sysreport --core /path/to/imap /path/to/core 

Make sure you have gdb and dovecot-dbg installed for best results.

Aki