Re: Message searching in Dovecot

[John Gateley via dovecot]

On 4/19/23 4:34 PM, John Gateley via dovecot wrote:


Hello,

For mobile clients (gmail on Android), I need a server side searching 
solution.


...



Thank you everyone for the suggestions. I will try them and report back.


John

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Mailing list is being Spam Filtered by O-365

[Benny Pedersen]

Kees van Vloten skrev den 2023-04-20 17:52:


I would guess the same applies for this list.


ARC-Authentication-Results	i=1; talvi.dovecot.org; dkim=pass 
header.d=gmail.com header.s=20221208 header.b=lWjGikqM; spf=pass 
(talvi.dovecot.org: domain of keesvanvlo...@gmail.com designates 
2a00:1450:4864:20::62f as permitted sender) 
smtp.mailfrom=keesvanvlo...@gmail.com; dmarc=pass (policy=none) 
header.from=gmail.com


why not see this header ?

in my spamassassin authres i have added talvi.dovecot.org to trusted 
signer


the original dkim is just still breaked :/

so is email in 2023 being on maillists that running mailman



___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Mailing list is being Spam Filtered by O-365

[Kees van Vloten]

On 20-04-2023 14:05, White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot wrote:


From headers:

Received: from BL0GCC02FT014.eop-gcc02.prod.protection.outlook.com

(2a01:111:f400:7d05::209) by CYXPR09CA0010.outlook.office365.com

(2603:10b6:930:d4::15) with Microsoft SMTP Server (version=TLS1_2,

cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via 
Frontend


Transport; Thu, 20 Apr 2023 11:49:06 +

Authentication-Results: spf=softfail (sender IP is 63.88.93.251)

smtp.mailfrom=open-xchange.com; dkim=fail (signature did not verify)

header.d=open-xchange.com;dmarc=fail action=oreject

header.from=open-xchange.com;compauth=none reason=452

Received-SPF: SoftFail (protection.outlook.com: domain of transitioning

open-xchange.com discourages use of 63.88.93.251 as permitted sender)

*From: *Aki Tuomi 
*Date: *Thursday, April 20, 2023 at 07:49
*Subject: *[EXTERNAL] Re: Mailing list is being Spam Filtered by O-365

On 20/04/2023 14:18 EEST White, Daniel E. (GSFC-770.0)[AEGIS] via
dovecot  wrote:

Is there any chance that SPF and DKIM records could be added to
appear in the headers ?

The gubba-mint folks are getting extremely medieval about email
security.

Um? Those already are added? Or what do you mean?

Aki

Wietse Venema gave this answer on the same question on the postfix 
mailinglist:


"
The list server adds its own DKIM-Signature: on behalf of the domain
postfix.org, AND it adds ARC headers and Authentication-Results:
for the message as received before modiification and redistribution.

However it does NOT remove existing DKIM-Signature: headers that
are no longer matching header and body content.
"

I would guess the same applies for this list.

- Kees.


___
dovecot mailing list --dovecot@dovecot.org
To unsubscribe send an email todovecot-le...@dovecot.org___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Message searching in Dovecot

[Thomas Zajic]

* Aki Tuomi via dovecot, 20.04.23 11:46


[...]
Biggest issue in my mind is that you will need to tell Solr to update
it's indexes (somehow) when using version 8 before upgrading to 9.
Because the older indexes are no longer compatible with 9.

If by that you mean migrating from [Fast]LRUCache to CaffeineCache, I found
PGNet Dev's post here [1] and Shawn Heisey's followup here [2] (including a
very handy script) extremely helpful when I did that last year:

[1] https://dovecot.org/pipermail/dovecot/2022-May/124701.html
[2] https://dovecot.org/pipermail/dovecot/2022-May/124711.html


[...]
Other than that, it was pretty simple to get it working in the end.


ACK, same here. :-) Have not yet upgraded to Solr 9.x, though, I'm currently
still at 8.11.2.

Bye,
Thomas
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Mailing list is being Spam Filtered by O-365

[White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot]

From headers:

Received: from BL0GCC02FT014.eop-gcc02.prod.protection.outlook.com
(2a01:111:f400:7d05::209) by CYXPR09CA0010.outlook.office365.com
(2603:10b6:930:d4::15) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via Frontend
Transport; Thu, 20 Apr 2023 11:49:06 +
Authentication-Results: spf=softfail (sender IP is 63.88.93.251)
smtp.mailfrom=open-xchange.com; dkim=fail (signature did not verify)
header.d=open-xchange.com;dmarc=fail action=oreject
header.from=open-xchange.com;compauth=none reason=452
Received-SPF: SoftFail (protection.outlook.com: domain of transitioning
open-xchange.com discourages use of 63.88.93.251 as permitted sender)

From: Aki Tuomi 
Date: Thursday, April 20, 2023 at 07:49
Subject: [EXTERNAL] Re: Mailing list is being Spam Filtered by O-365

On 20/04/2023 14:18 EEST White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot 
mailto:dovecot@dovecot.org>> wrote:


Is there any chance that SPF and DKIM records could be added to appear in the 
headers ?

The gubba-mint folks are getting extremely medieval about email security.


Um? Those already are added? Or what do you mean?

Aki

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Mailing list is being Spam Filtered by O-365

[Aki Tuomi via dovecot]

> On 20/04/2023 14:18 EEST White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot 
>  wrote:
> 
>  
> Is there any chance that SPF and DKIM records could be added to appear in the 
> headers ?
> 
> The gubba-mint folks are getting extremely medieval about email security.
> 

Um? Those already are added? Or what do you mean?

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Mailing list is being Spam Filtered by O-365

[White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot]

Is there any chance that SPF and DKIM records could be added to appear in the 
headers ?

The gubba-mint folks are getting extremely medieval about email security.

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Request - extra variable in passdb for logging

[Aki Tuomi via dovecot]

> On 20/04/2023 13:18 EEST Bogusław Juza  wrote:
> 
>  
> Hi Staff,
> 
> I'm creating the dovecot configuration for multiple user passwords
> e-mail site. It's working well, but I've one problem - I can't
> log, which password was used.
> 
> I'm using auth-sql, the query looks like:
> 
> password_query = \
>SELECT `users`.`email` AS `user`, \
>   `shadow`.`passwd` AS `password`, \
>FROM `shadow`
>WHERE `shadow`.`email` = '%u' AND \
>  ( ('%r'='127.0.0.1' AND `shadow`.`webmail`<>0) OR \
>('%r'<>'127.0.0.1' AND `shadow`.`imap`<>0 AND \
> `shadow`.`hash`='%{sha512;rounds=5000:password}') \
>  ) LIMIT 1
> 
> The hash from random generated application password works as
> the selector, which password should be checked.
> 
> It works fine, but in the log I have got only the e-mail and both
> IP addresses. I need to log one more information - which password
> was used (`shadow`.`id` column). It would be a great feature
> to have one more extra variable, which I could set in this
> query and which goes directly to log and nowhere else.
> 
> I have tried something like:
> 
> password_query = \
>SELECT CONCAT(`users`.`email`,'#',`shadow`.`id`) AS `user`, ...
> 
> and then "repair it" in user_query:
>   SELECT `email` AS `user`
>   WHERE `email` = REGEXP_SUBSTR('%u','[^#]+')
> 
> and it even works well, except postfix-auth, which received email
> with #number as the username and it was problematic.
> 
> So I'm kindly asking for this extra variable in next versions
> of Dovecot ;)
> 
> Bogusław Juza
> ___
> dovecot mailing list -- dovecot@dovecot.org
> To unsubscribe send an email to dovecot-le...@dovecot.org

Try setting

login_log_format_elements = $login_log_format_elements 
%{passdb:some_variable_name}

and try set it with 

password_query = SELECT ... ,'something' AS some_variable_name, 

Hopefully it works.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Blacklistd

[Benny Pedersen]

Odhiambo Washington skrev den 2023-04-20 11:04:


Since blacklistd uses PF, you can already use fail2ban or sshguard [1]
to achieve the same thing you are after.
Given that blacklistd is just an intermediary like fail2ban, is there
a real need for dovecot interfacing with it?


fail2ban cant see dovecot internal fails of auth, this is why weakforced 
is buildt, in gentoo i use it now, i have not regreted doing this ebuild 
request to keep it going


https://github.com/PowerDNS/weakforced
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Blacklistd

[Benny Pedersen]

Doug Hardie skrev den 2023-04-20 08:07:

Are there any plans to interface to blacklistd?


link ?

imho only weakforced is currently supported, more info for your needs 
would help find more info



https://github.com/PowerDNS/weakforced

https://github.com/paul-chambers/blacklistd

or other ?
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Investigating inconsistencies between converted and original mailboxes

[Andrey Repin]

Greetings, kjohn...@eclypse.org!

> Summary:  Looking for a tool to report vsize of messages by UIDL, or other
> ways to investigate vsize inconsistency between converted (maildir) and 
> original (mbox) mailboxes.

The best way to investigate the differences is, quite unsurprisingly, to use
diff tool directly on the messages in question.

Based on my observation from my own conversion endeavor, the messages imported
are altered in a non-intrusive, standards-compliant, but sometimes quite
significant way. Which may lead to significant differences in message size.


-- 
With best regards,
Andrey Repin
Thursday, April 20, 2023 13:01:22

Sorry for my terrible english...

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Request - extra variable in passdb for logging

[Bogusław Juza]

  Hi Staff,

I'm creating the dovecot configuration for multiple user passwords
e-mail site. It's working well, but I've one problem - I can't
log, which password was used.

I'm using auth-sql, the query looks like:

password_query = \
  SELECT `users`.`email` AS `user`, \
 `shadow`.`passwd` AS `password`, \
  FROM `shadow`
  WHERE `shadow`.`email` = '%u' AND \
( ('%r'='127.0.0.1' AND `shadow`.`webmail`<>0) OR \
  ('%r'<>'127.0.0.1' AND `shadow`.`imap`<>0 AND \
   `shadow`.`hash`='%{sha512;rounds=5000:password}') \
) LIMIT 1

The hash from random generated application password works as
the selector, which password should be checked.

It works fine, but in the log I have got only the e-mail and both
IP addresses. I need to log one more information - which password
was used (`shadow`.`id` column). It would be a great feature
to have one more extra variable, which I could set in this
query and which goes directly to log and nowhere else.

I have tried something like:

password_query = \
  SELECT CONCAT(`users`.`email`,'#',`shadow`.`id`) AS `user`, ...

and then "repair it" in user_query:
 SELECT `email` AS `user`
 WHERE `email` = REGEXP_SUBSTR('%u','[^#]+')

and it even works well, except postfix-auth, which received email
with #number as the username and it was problematic.

So I'm kindly asking for this extra variable in next versions
of Dovecot ;)

   Bogusław Juza
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Message searching in Dovecot

[Aki Tuomi via dovecot]

> On 20/04/2023 07:35 EEST Aki Tuomi via dovecot  wrote:
> 
> 
> 
> > On 20/04/2023 00:34 EEST John Gateley via dovecot  
> > wrote:
> > 
> > 
> > Hello,
> > For mobile clients (gmail on Android), I need a server side searching 
> > solution.
> > A few months ago, I tried getting Solr to work with Dovecot, but failed.
> > The instructions here 
> > https://doc.dovecot.org/configuration_manual/fts/solr/ are quite out of 
> > date, they reference Debian 8 and 9 (current version 11), and Solr 7.7 
> > (current version 9.2)
> > My Solr experience is pretty much zero.
> > Questions/Requests:
> >   * Is there a different tool than Solr I should be using for this?
> >   * Does anyone have a recent version of Solr and Debian working?
> >   * If you do, would you be willing to help me?
> >   * I cannot afford much, this is a home mailserver for my wife and I, but 
> > I could pay a small consulting fee.
> > I use ansible to deploy the server, I'd be willing to contribute the 
> > playbooks somewhere for documentation, or help with the docs.
> > Thanks!
> > John
> > ___
> > dovecot mailing list -- dovecot@dovecot.org
> > To unsubscribe send an email to dovecot-le...@dovecot.org
> 
> You can try https://github.com/slusarz/dovecot-fts-flatcurve .
> 
> Can you give more details on how Solr didn't work for you? I am using solr 
> myself.
> 
> Aki

I now updated my Solr to 9.2. It took a little bit more effort, but I'll see if 
I can update our docs to indicate how you could use Solr 9.

Biggest issue in my mind is that you will need to tell Solr to update it's 
indexes (somehow) when using version 8 before upgrading to 9. Because the older 
indexes are no longer compatible with 9.

Other than that, it was pretty simple to get it working in the end.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

RE: Blacklistd

[Aki Tuomi via dovecot]

> On 20/04/2023 12:17 EEST Marc  wrote:
> 
>  
> > 
> > Are there any plans to interface to blacklistd?
> > 
> > -- Doug
> > 
> > 
> > Hi Doug,
> > 
> > Since blacklistd uses PF, you can already use fail2ban or sshguard
> >   to achieve the same thing you are after.
> > Given that blacklistd is just an intermediary like fail2ban, is there a
> > real need for dovecot interfacing with it?
> > 
> 
> Maybe because fail2ban and logs are on a remote server?

Hi!

My suggestions:

1: Write simple (e.g.) Flask/Twisted based adapter for Auth Policy (not really 
very difficult)
2: Use passdb lua to interface with blacklistd.
3: Convince blacklistd authors to support weakforced protocol.

And no, we do not have currently plans to add support for blacklistd. Sorry.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

RE: Blacklistd

[Marc]

> 
>   Are there any plans to interface to blacklistd?
> 
>   -- Doug
> 
> 
> Hi Doug,
> 
> Since blacklistd uses PF, you can already use fail2ban or sshguard
>   to achieve the same thing you are after.
> Given that blacklistd is just an intermediary like fail2ban, is there a
> real need for dovecot interfacing with it?
> 

Maybe because fail2ban and logs are on a remote server?
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

RE: Investigating inconsistencies between converted and original mailboxes

[Marc]

> 
> Before attempting the conversion, I used this shell script to get status
> information about each mailbox.  It uses 'doveadm mailbox list' to get a
> list of mailboxes for each user, and 'doveadm mailbox status' to provide
> summary information.  For example (username redacted)
> 
> zxxx: INBOX   messages=6 recent=0 unseen=4 vsize=7820215
> 
> 
> I used my mailbox status script again after the conversion to report on
> the maildir mailboxes.
> 
> Of the 143 mailboxes found by the script, there are three where the
> status does not exactly match between the old mailboxes and the new
> mailboxes.  The difference is only in the vsize.  Sometimes the new
> vsize is large; sometimes it is smaller.
> 

I think you did a proper check already. How big are these differences in vsize? 
Could this not be related to a different way the filesystem is being used?


___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Blacklistd

[Odhiambo Washington]

On Thu, Apr 20, 2023 at 9:08 AM Doug Hardie  wrote:

> Are there any plans to interface to blacklistd?
>
> -- Doug
>

Hi Doug,

Since blacklistd uses PF, you can already use fail2ban or sshguard
 to achieve the same thing you are after.
Given that blacklistd is just an intermediary like fail2ban, is there a
real need for dovecot interfacing with it?

-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org