Re: Indexer worker small bug
On 10 Dec 2018, at 7.50, André Rodier wrote: > > On 2018-12-09 23:13, Timo Sirainen wrote: >> On 9 Dec 2018, at 16.44, André Rodier via dovecot >> wrote: >>> Hello, >>> I think I submitted this before, but I am not sure this has been addressed >>> I am using AppArmor with Dovecot, without any issue. >>> However, I think there is a bug in the indexer working, from what I can >>> see, a missing trailing slash. See: >>> >>> Dec 09 14:35:53 portal2 kernel: audit: type=1400 >>> audit(1544366153.379:3035): apparmor="DENIED" operation="file_mmap" >>> info="Failed name lookup - disconnected path" error=-13 >>> profile="/usr/lib/dovecot/indexer-worker" name="var/cache/nscd/hosts" >>> pid=10540 comm="indexer-worker" requested_mask="r" denied_mask="r" >>> fsuid=1001 ouid=0 >>> >>> The indexer worker is trying to open the file "var/cache/nscd/hosts" >>> instead of "/var/cache/nscd/hosts", which of course fails. >>> Can someone double check the code of the indexer worker, or this has been >>> fixed? >> Dovecot is definitely not trying to open that file itself. It has to >> be libc or some other library. I also can't think of anything special >> in indexer-worker compared to other Dovecot binaries that could cause >> this. What's your doveconf -n? > > You are probably right, I will continue to investigate on my side. > My configuration is attached. Does it make a difference if you use "127.0.0.1" instead of "localhost" in fts_solr setting?
Re: Solr
Hi!
Thank you for your report, we'll look thru it.
Aki
On 9.12.2018 11.24, Joan Moreau via dovecot wrote:
>
> However, Solr crashes and Dovecot plugin ftS_solr returns multitude of
> errors
>
> Dec 09 09:21:09 imap(j...@grosjo.net)<3349>: Error:
> fts_solr: received invalid uid '0'
> Dec 09 09:21:10 imap(j...@grosjo.net)<3349>: Error:
> fts_solr: received invalid uid '0'
>
> And returns are totaly funny (keywords not presentin teh results)
>
>
> I am back to fs_squat
>
>
>
>
> On 2018-12-08 18:28, Joan Moreau via dovecot wrote:
>
>> After some testsing, I managed to get proper functionning
>>
>> - The schema.xml is attached below (quite different from the one
>> provided on teh wiki) (in bold the core differences)
>> (NGramFilterFactory is the class that replace the fts_squat
>> "partial=3 full=15", everything else is just a big hammer to smash a
>> tiny fly)
>>
>> - One need to remove the "managed-schema" file in the
>> {prefix}/server/solr/dovecot/conf.
>>
>> - One need to remove everything under {prefix}/server/solr/dovecot/data/
>>
>> - The {prefix}/server/solr/dovecot/conf/solrconfig.xml is as below
>> (see diff)
>>
>> - Restart Solr
>>
>> - my dovecot.conf is :
>>
>> fts = solr
>> fts_autoindex = yes
>> fts_enforced = yes
>> fts_decoder = decode2text
>>
>> fts_solr = url=http://(SOLR SERVER):8983/solr/dovecot/
>>
>>
>> --- schema.xml
>>
>>
>>
>> id
>>
>> **
>> **
>> **
>> > autoGeneratePhraseQueries="true" positionIncrementGap="100">
>>
>>
>> > ignoreCase="true"/>
>> > generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1"
>> splitOnNumerics="1" catenateWords="1" catenateNumbers="1"
>> catenateAll="1"/>
>>
>>
>> *> maxGramSize="15" />*
>> > protected="protwords.txt"/>
>>
>>
>>
>>
>> > ignoreCase="true" synonyms="synonyms.txt"/>
>>
>> > ignoreCase="true"/>
>> > generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1"
>> splitOnNumerics="1" catenateWords="1" catenateNumbers="1"
>> catenateAll="1"/>
>>
>> *> maxGramSize="15" />*
>> > protected="protwords.txt"/>
>>
>>
>>
>>
>>
>>
>>
>> > stored="true"/>
>>
>>
>>
>> > stored="true"/>
>>
>>
>> > stored="true"/>
>> > stored="true"/>
>>
>>
>>
>> -- diff solrconfig
>>
>> --- /data/backup/solr/solrconfig.xml.joan 2018-12-08
>> 14:31:47.716344505 +
>> +++ solrconfig.xml 2018-12-08 15:36:28.948267225 +
>> @@ -1128,6 +1128,7 @@
>>
>> See http://wiki.apache.org/solr/GuessingFieldTypes
>> -->
>> +
>>
>> > name="remove-blank"/>
>> > name="field-name-mutating">
>> @@ -1158,43 +1159,8 @@
>> -MM-dd
>>
>>
>> - > name="add-schema-fields">
>> -
>> - java.lang.String
>> - text_general
>> -
>> - *_str
>> - 256
>> -
>> -
>> - true
>> -
>> -
>> - java.lang.Boolean
>> - booleans
>> -
>> -
>> - java.util.Date
>> - pdates
>> -
>> -
>> - java.lang.Long
>> - java.lang.Integer
>> - plongs
>> -
>> -
>> - java.lang.Number
>> - pdoubles
>> -
>> -
>>
>>
>> - > name="add-unknown-fields-to-the-schema"
>> default="${update.autoCreateFields:true}"
>> -
>> processor="uuid,remove-blank,field-name-mutating,parse-boolean,parse-long,parse-double,parse-date,add-schema-fields">
>> -
>> -
>> -
>> -
>>
>>
>> -
>>
>>
>>
>> text/plain; charset=UTF-8
>>
>> -
>> +
>>
>>
>>
>>
>>
Re: Indexer worker small bug
On 2018-12-09 23:13, Timo Sirainen wrote:
On 9 Dec 2018, at 16.44, André Rodier via dovecot
wrote:
Hello,
I think I submitted this before, but I am not sure this has been
addressed
I am using AppArmor with Dovecot, without any issue.
However, I think there is a bug in the indexer working, from what I
can see, a missing trailing slash. See:
Dec 09 14:35:53 portal2 kernel: audit: type=1400
audit(1544366153.379:3035): apparmor="DENIED" operation="file_mmap"
info="Failed name lookup - disconnected path" error=-13
profile="/usr/lib/dovecot/indexer-worker" name="var/cache/nscd/hosts"
pid=10540 comm="indexer-worker" requested_mask="r" denied_mask="r"
fsuid=1001 ouid=0
The indexer worker is trying to open the file "var/cache/nscd/hosts"
instead of "/var/cache/nscd/hosts", which of course fails.
Can someone double check the code of the indexer worker, or this has
been fixed?
Dovecot is definitely not trying to open that file itself. It has to
be libc or some other library. I also can't think of anything special
in indexer-worker compared to other Dovecot binaries that could cause
this. What's your doveconf -n?
You are probably right, I will continue to investigate on my side.
My configuration is attached.
--
https://github.com/progmaticltd/homebox# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6
auth_debug = yes
auth_verbose = yes
auth_verbose_passwords = yes
lmtp_save_to_detail_mailbox = yes
mail_debug = yes
mail_plugins = " quota fts fts_solr"
mailbox_list_index = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext imapsieve
namespace inbox {
inbox = yes
location = maildir:~/mails/maildir:INDEX=~/mails/indexes/
mailbox Archives {
auto = subscribe
special_use = \Archive
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
fts = solr
fts_decoder = decode2text
fts_index_timeout = 60
fts_solr = url=http://localhost:8080/solr/
imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_from = *
imapsieve_mailbox1_name = Junk
imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_name = *
imapsieve_mailbox3_after = file:/etc/dovecot/sieve/duplicate-checks.sieve
imapsieve_mailbox3_causes = APPEND
imapsieve_mailbox3_from = *
imapsieve_mailbox3_name = INBOX
imapsieve_mailbox4_after = file:/etc/dovecot/sieve/sent-checks.sieve
imapsieve_mailbox4_causes = APPEND
imapsieve_mailbox4_from = Sent
imapsieve_mailbox4_name = *
imapsieve_mailbox5_after = file:/etc/dovecot/sieve/tags-important.sieve
imapsieve_mailbox5_causes = APPEND, COPY
imapsieve_mailbox5_from = *
imapsieve_mailbox5_name = Important
quota = maildir:user-quota
quota_rule = *:storage=500M
recipient_delimiter = ~
sieve = file:~/mails/sieve;active=~/mails/sieve/active.sieve
sieve_before = file:/etc/dovecot/sieve/before-global.sieve
sieve_default_name = default
sieve_global = /etc/dovecot/sieve
sieve_global_extensions = +vnd.dovecot.pipe
sieve_pipe_bin_dir = /etc/dovecot/sieve
sieve_plugins = sieve_imapsieve sieve_extprograms
sieve_trace_addresses = yes
sieve_trace_debug = yes
sieve_trace_dir = ~/mails/sieve/logs/
sieve_trace_level = commands
sieve_user_log = ~/mails/sieve/logs/main.log
}
protocols = " imap lmtp sieve pop3"
recipient_delimiter = ~
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0666
}
}
service decode2text {
executable = script /usr/local/bin/decode2text
unix_listener decode2text {
mode = 0666
}
user = $default_internal_user
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service imap {
executable = imap
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
service managesieve {
process_limit = 1024
}
service pop3-login {
inet_listener pop3 {
port = 110
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service quota-warning {
executable = quota-status -p postfix
inet_listener {
port
Re: BUG: sieve does not set seen-Flag [SOLVED]
Hello! I think i got it. This weekend i setup a totally new Mailserver for testing purpose. First to the default-setup and used a file for userdb. Then I activated ldap-authentication. Moving emails around does not changed the \Seen-state (how it should be) Then i copied my 10-mail.conf to the new server. And moving a message to an other folder sets it as \Recent and unseen... So the bug must be in this file... My setup uses separat directories for INDEX anc CONTROL in /var/lib/dovecot... and INDEXPVT for shared boxes and even the INBOX... Then I removed all the separate INDEX, CONTROL an INDEXPVT-options for mail_location so thant only mail_location = maildir:/var/mail/%u/Maildir:LAYOUT=fs was left. And moving messages arround let them stay unseen. Then i added INDEX, CONTROL and INDEXPVT step by step and tested the setup... At least i had again mail_location = maildir:/var/mail/%u/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:INDEXPVT=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u Moving messages with INDEXPVT in the config sets messages again to \Recent and unseen in the new Folder. Without INDEXPVT an unseen message stays unseen... So i think, the problem is solved. Bye jakob
Re: Indexer worker small bug
On 9 Dec 2018, at 16.44, André Rodier via dovecot wrote: > > Hello, > > I think I submitted this before, but I am not sure this has been addressed > > I am using AppArmor with Dovecot, without any issue. > > However, I think there is a bug in the indexer working, from what I can see, > a missing trailing slash. See: > > > Dec 09 14:35:53 portal2 kernel: audit: type=1400 audit(1544366153.379:3035): > apparmor="DENIED" operation="file_mmap" info="Failed name lookup - > disconnected path" error=-13 profile="/usr/lib/dovecot/indexer-worker" > name="var/cache/nscd/hosts" pid=10540 comm="indexer-worker" > requested_mask="r" denied_mask="r" fsuid=1001 ouid=0 > > > The indexer worker is trying to open the file "var/cache/nscd/hosts" instead > of "/var/cache/nscd/hosts", which of course fails. > > Can someone double check the code of the indexer worker, or this has been > fixed? Dovecot is definitely not trying to open that file itself. It has to be libc or some other library. I also can't think of anything special in indexer-worker compared to other Dovecot binaries that could cause this. What's your doveconf -n?
Re: Sieve notify variables not working with body?
Op 16/11/2018 om 20:36 schreef Bambero:
Thanks, now it works.
But it seems that :message string (of notify command) is limited to
255 chars.
Is there any chance to change limit to 512 ?
Hmm, you are right:
https://github.com/dovecot/pigeonhole/blob/master/src/lib-sieve/plugins/enotify/mailto/ntfy-mailto.c#L49
That is quite a low limit, especially when the message is in e.g.
Japanese symbols. We got complaints about a similar problem a while back
for the vacation extension. I can fix notify as well when I get to work
on that ticket.
Still, I would not recommend putting the body in the subject of the
notification message (which is what ":message" does). Instead, you could
use the "body" header field of the mailto URL to put your message in the
body instead.
Regards,
Stephan.
require
["body","enotify","variables","foreverypart","mime","extracttext"];
foreverypart
{
if header :mime :type :is "Content-Type" "text"
{
extracttext :first 512 "body";
break;
}
}
if not header :contains "subject" "*"
{
notify :importance "3" :message "${body}" "mailto:t...@example.com
";
}
On Fri, Nov 16, 2018 at 2:53 PM Stephan Bosch > wrote:
Op 15-11-2018 om 23:54 schreef Bambero:
> Hi,
>
> I want to send notification like this:
>
> if body :text :matches "*" {
> notify :importance "3"
> :message "[NOTIFY] ${1}"
> "mailto:test@example
>,com";
> }
>
> The notification works but the variable ${1} seems to be empty.
>
> But in this case:
>
> if header :matches "Subject" "*" {
> notify :importance "3"
> :message "[NOTIFY] ${1}"
> "mailto:test@example
>,com";
> }
>
> Everything works correct.
https://tools.ietf.org/html/rfc5173#section-6
Use the extracttext extension instead:
https://tools.ietf.org/html/rfc5703#section-7
Regards,
Stephan.
Indexer worker small bug
Hello, I think I submitted this before, but I am not sure this has been addressed I am using AppArmor with Dovecot, without any issue. However, I think there is a bug in the indexer working, from what I can see, a missing trailing slash. See: Dec 09 14:35:53 portal2 kernel: audit: type=1400 audit(1544366153.379:3035): apparmor="DENIED" operation="file_mmap" info="Failed name lookup - disconnected path" error=-13 profile="/usr/lib/dovecot/indexer-worker" name="var/cache/nscd/hosts" pid=10540 comm="indexer-worker" requested_mask="r" denied_mask="r" fsuid=1001 ouid=0 The indexer worker is trying to open the file "var/cache/nscd/hosts" instead of "/var/cache/nscd/hosts", which of course fails. Can someone double check the code of the indexer worker, or this has been fixed? Thanks, André -- https://github.com/progmaticltd/homebox
QUERY string clarification
The manpage https://wiki.dovecot.org/Tools/Doveadm/SearchQuery specifies: BODY string Matches messages, which contain string in the body part. and a bit further down: TEXT string Matches messages, which contain string in the message body. Where's the difference? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | https://www.charite.de
Re: Solr
However, Solr crashes and Dovecot plugin ftS_solr returns multitude of
errors
Dec 09 09:21:09 imap(j...@grosjo.net)<3349>: Error:
fts_solr: received invalid uid '0'
Dec 09 09:21:10 imap(j...@grosjo.net)<3349>: Error:
fts_solr: received invalid uid '0'
And returns are totaly funny (keywords not presentin teh results)
I am back to fs_squat
On 2018-12-08 18:28, Joan Moreau via dovecot wrote:
After some testsing, I managed to get proper functionning
- The schema.xml is attached below (quite different from the one provided on teh wiki) (in bold the core differences) (NGramFilterFactory is the class that replace the fts_squat "partial=3 full=15", everything else is just a big hammer to smash a tiny fly)
- One need to remove the "managed-schema" file in the {prefix}/server/solr/dovecot/conf.
- One need to remove everything under {prefix}/server/solr/dovecot/data/
- The {prefix}/server/solr/dovecot/conf/solrconfig.xml is as below (see diff)
- Restart Solr
- my dovecot.conf is :
fts = solr
fts_autoindex = yes
fts_enforced = yes
fts_decoder = decode2text
fts_solr = url=http://(SOLR SERVER):8983/solr/dovecot/
--- schema.xml
id
-- diff solrconfig
--- /data/backup/solr/solrconfig.xml.joan 2018-12-08 14:31:47.716344505 +
+++ solrconfig.xml 2018-12-08 15:36:28.948267225 +
@@ -1128,6 +1128,7 @@
See http://wiki.apache.org/solr/GuessingFieldTypes
-->
+
@@ -1158,43 +1159,8 @@
-MM-dd
-
-
- java.lang.String
- text_general
-
- *_str
- 256
-
-
- true
-
-
- java.lang.Boolean
- booleans
-
-
- java.util.Date
- pdates
-
-
- java.lang.Long
- java.lang.Integer
- plongs
-
-
- java.lang.Number
- pdoubles
-
-
-
-
-
-
-
-
text/plain; charset=UTF-8
-
+
