Finding Users Over Quota
Using doveadm quota will let you see if a specific user is over quota. Is there a way to have dovecot tell you which users are currently over quota? Is the quota_clone plugin a way to save user quota to dict like how last_login saves to dict? The documentation for quota_clone is MIA. (As of 3/16/22) https://wiki2.dovecot.org/Plugins v https://wiki2.dovecot.org/Plugins/QuotaClone v https://doc.dovecot.org/plugin-settings/quota-clone-plugin/ v 404 Not Found
Re: SIS and tracing the origin of an attachment
On 3/16/2022 6:05 AM, Patrick Cernko wrote:
Hi all,
On 15.03.22 22:40, doug wrote:
On 3/15/2022 3:45 PM, Oscar del Rio wrote:
On 2022-03-15 9:02 a.m., doug wrote:
On 3/8/2022 5:51 PM, doug wrote:
I'm trying to trace an attachment within an SIS subdirectory to
the email message(s) that link to it. I say messages because I'm
also using dovecot dedup. My understanding is the linked file name
is the hash value of the attachments contents concatenated with
the GUID of the email message. I have had marginal success with a
message I created myself.
Example: I generated an email with two attachments. Here are the
links in my attachment directory.
./26/c5/26c5c540d41779d83d2f5388041d05c67d720d9a-73eca8051acd27627231f2bc99a3
./65/cd/65cd73112a489ef07f17ed5740aa60358e2dd3fb-74eca8051acd27627231f2bc99a3
I keep experimenting with this and I still haven't found a reliable
way to track an attachment back to it's original message so I can
either notify the user or delete the message with doveadm. Is this
not possible? I'm using mdbox if that matters. I see a similar
thread going right now about virus scanning and deleting messages
but that is maildir and I suspect not using SIS for attachments.
The very few times I've needed to trace a SIS attachment to a
mailbox, I just grep the "storage" folders for the file hash
find username/storage -type f -exec grep
9ffa4b246589f8039d123ea909f1520e791bd880 {} +
username/storage/m.46588:X908 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-c9ee303687e13062cf740012bfe47a40
username/storage/m.46589:X1918 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-080ce71390e1306299730012bfe47a40
username/storage/m.46588:
BSent
X908 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-c9ee303687e13062cf740012bfe47a40
username/storage/m.46589:
BINBOX
X1918 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-080ce71390e1306299730012bfe47a40
-> Attachment in username's INBOX and Sent folders.
Thank you for the suggestion Oscar. My mdbox files are encrypted and
compressed, so unfortunately directly grepping them will not work.
You can use "doveadm dump" to decompress the files for grepping them,
not sure about encryption:
find path/to/userhomes/mdbox/storage -name 'm.*' | \
while read f; do
doveadm dump $f | \
grep -E '^msg.(ext-ref|orig-mailbox|guid)' | \
grep -B2 xx/yy/hash-guid || continue
echo "Match in $f"
done
The dump also contains several other fields you might want to display.
Best,
I'll give that a try. With access to the encryption key doveadm dump
should handle it just fine. I was hopeful there was a method using
search and index files to minimize overhead.
To summarize what I think I have learned on this journey is the link to
the hash file only exists within the contents of the email body, but not
in a way that doveadm search will find it. Hence raw scanning the
contents of the emails is required.
Many thanks for everyone's help.
--
Doug
Re: [EXT] AW: AW: AW: invalid lz4 chunk size??
What's with the "AW: AW: AW:" business? It sounds for all the world like a guy outbid at the Sotheby's auction or something like that. There's got to be a lot of artwork online with that fancy lz4 compression algorithm. On March 16, 2022 2:32:32 AM AKDT, Joachim Lindenberg wrote: >What might go wrong? Or should I copy the directory myself from remote? What >can go wrong with that? >Thanks, Joachim > >-Ursprüngliche Nachricht- >Von: Aki Tuomi >Gesendet: Wednesday, 16 March 2022 11:31 >An: Joachim Lindenberg >Betreff: Re: [EXT] AW: AW: AW: invalid lz4 chunk size?? > >It might, I can't promise it =) > >Aki > >> On 16/03/2022 12:29 Joachim Lindenberg wrote: >> >> >> With remote I am referring to the second instance of mailcow including >> dovecot I am running, which has dovecot replication active. I am observing >> the issue on just one box. Thus I am hoping that when I remove the user >> mail directory on one host, replication will restore it from remote. >> Thanks, Joachim >> >> -Ursprüngliche Nachricht- >> Von: Aki Tuomi >> Gesendet: Wednesday, 16 March 2022 11:17 >> An: Joachim Lindenberg ; dovecot@dovecot.org >> Betreff: Re: AW: AW: invalid lz4 chunk size?? >> >> Not sure what "remote" you are talking about. You can remove the file, and >> dovecot will notice it's absence and update it's indexes if this is >> maildir++ format. >> >> If this is sdbox, you need to run `doveadm force-resync -u user FolderName`. >> >> Aki >> >> > On 16/03/2022 12:08 Joachim Lindenberg wrote: >> > >> > >> > Hi Aki, >> > sure I can delete (or rename) the file(s) in the mail directory. >> > Will sync then restore it from remote or should I do this manually? Should >> > I stop dovecot during the process?` I am not very familiar with dovecot >> > yet, as most of the complexity is hidden by mailcow. >> > Thanks, Joachim >> > >> > -Ursprüngliche Nachricht- >> > Von: Aki Tuomi >> > Gesendet: Wednesday, 16 March 2022 08:38 >> > An: Joachim Lindenberg ; dovecot@dovecot.org >> > Betreff: Re: AW: invalid lz4 chunk size?? >> > >> > Hi, >> > >> > looks a lot like your mail file is corrupted. Not much you can do about it >> > other than maybe delete the file? You can try recover it with `doveadm >> > fetch -u someone text uid 1553 mailbox Sent` and then using `doveadm save >> > -u someone -m Sent` to store it back. >> > >> > Aki >> > >> > > On 16/03/2022 09:35 Joachim Lindenberg wrote: >> > > >> > > >> > > Nobody that can help? >> > > Thanks, >> > > Joachim >> > > >> > > -- >> > > >> > > I am still experiencing the issue. Any suggestion? >> > > As I do have replication between two nodes and only one is showing the >> > > issue - can I rename the mailbox easily on one side and rely on >> > > replication to get the copy replaced? If that makes sense, which >> > > commands do you recommend? >> > > Thanks, >> > > Joachim >> > > >> > > -Ursprüngliche Nachricht- >> > > Von: Joachim Lindenberg >> > > Gesendet: Thursday, 3 March 2022 12:06 >> > > An: 'Aki Tuomi' ; dovecot@dovecot.org >> > > Betreff: AW: invalid lz4 chunk size?? >> > > >> > > dovecot --version reports 2.3.17.1 (476cd46418) Joachim >> > > >> > > -Ursprüngliche Nachricht- >> > > Von: Aki Tuomi >> > > Gesendet: Thursday, 3 March 2022 11:56 >> > > An: Joachim Lindenberg ; >> > > dovecot@dovecot.org >> > > Betreff: Re: invalid lz4 chunk size?? >> > > >> > > >> > > > On 03/03/2022 12:24 Joachim Lindenberg wrote: >> > > > >> > > > >> > > > Hello, >> > > > when accessing one mailbox via ActiveSync / SoGo / Dovecot I get the >> > > > following error repeatedly in dovecot log: >> > > > imap(somemail...@example.org)<1579><***>: Error: Mailbox Sent: >> > > > UID=1553: read(compress()) failed: read() failed: lz4.read(): invalid >> > > > lz4 chunk size: 1601505441 at 16842752 (read reason=) I can still >> > > > access the mailbox via IMAP though. >> > > > What can I do to resolve the issue? >> > > > Thanks, >> > > > Joachim >> > > >> > > Which version of Dovecot? >> > > >> > > Aki > -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
AW: [EXT] AW: AW: AW: invalid lz4 chunk size??
What might go wrong? Or should I copy the directory myself from remote? What can go wrong with that? Thanks, Joachim -Ursprüngliche Nachricht- Von: Aki Tuomi Gesendet: Wednesday, 16 March 2022 11:31 An: Joachim Lindenberg Betreff: Re: [EXT] AW: AW: AW: invalid lz4 chunk size?? It might, I can't promise it =) Aki > On 16/03/2022 12:29 Joachim Lindenberg wrote: > > > With remote I am referring to the second instance of mailcow including > dovecot I am running, which has dovecot replication active. I am observing > the issue on just one box. Thus I am hoping that when I remove the user mail > directory on one host, replication will restore it from remote. > Thanks, Joachim > > -Ursprüngliche Nachricht- > Von: Aki Tuomi > Gesendet: Wednesday, 16 March 2022 11:17 > An: Joachim Lindenberg ; dovecot@dovecot.org > Betreff: Re: AW: AW: invalid lz4 chunk size?? > > Not sure what "remote" you are talking about. You can remove the file, and > dovecot will notice it's absence and update it's indexes if this is maildir++ > format. > > If this is sdbox, you need to run `doveadm force-resync -u user FolderName`. > > Aki > > > On 16/03/2022 12:08 Joachim Lindenberg wrote: > > > > > > Hi Aki, > > sure I can delete (or rename) the file(s) in the mail directory. > > Will sync then restore it from remote or should I do this manually? Should > > I stop dovecot during the process?` I am not very familiar with dovecot > > yet, as most of the complexity is hidden by mailcow. > > Thanks, Joachim > > > > -Ursprüngliche Nachricht- > > Von: Aki Tuomi > > Gesendet: Wednesday, 16 March 2022 08:38 > > An: Joachim Lindenberg ; dovecot@dovecot.org > > Betreff: Re: AW: invalid lz4 chunk size?? > > > > Hi, > > > > looks a lot like your mail file is corrupted. Not much you can do about it > > other than maybe delete the file? You can try recover it with `doveadm > > fetch -u someone text uid 1553 mailbox Sent` and then using `doveadm save > > -u someone -m Sent` to store it back. > > > > Aki > > > > > On 16/03/2022 09:35 Joachim Lindenberg wrote: > > > > > > > > > Nobody that can help? > > > Thanks, > > > Joachim > > > > > > -- > > > > > > I am still experiencing the issue. Any suggestion? > > > As I do have replication between two nodes and only one is showing the > > > issue - can I rename the mailbox easily on one side and rely on > > > replication to get the copy replaced? If that makes sense, which commands > > > do you recommend? > > > Thanks, > > > Joachim > > > > > > -Ursprüngliche Nachricht- > > > Von: Joachim Lindenberg > > > Gesendet: Thursday, 3 March 2022 12:06 > > > An: 'Aki Tuomi' ; dovecot@dovecot.org > > > Betreff: AW: invalid lz4 chunk size?? > > > > > > dovecot --version reports 2.3.17.1 (476cd46418) Joachim > > > > > > -Ursprüngliche Nachricht- > > > Von: Aki Tuomi > > > Gesendet: Thursday, 3 March 2022 11:56 > > > An: Joachim Lindenberg ; > > > dovecot@dovecot.org > > > Betreff: Re: invalid lz4 chunk size?? > > > > > > > > > > On 03/03/2022 12:24 Joachim Lindenberg wrote: > > > > > > > > > > > > Hello, > > > > when accessing one mailbox via ActiveSync / SoGo / Dovecot I get the > > > > following error repeatedly in dovecot log: > > > > imap(somemail...@example.org)<1579><***>: Error: Mailbox Sent: > > > > UID=1553: read(compress()) failed: read() failed: lz4.read(): invalid > > > > lz4 chunk size: 1601505441 at 16842752 (read reason=) I can still > > > > access the mailbox via IMAP though. > > > > What can I do to resolve the issue? > > > > Thanks, > > > > Joachim > > > > > > Which version of Dovecot? > > > > > > Aki
Re: AW: AW: invalid lz4 chunk size??
Not sure what "remote" you are talking about. You can remove the file, and dovecot will notice it's absence and update it's indexes if this is maildir++ format. If this is sdbox, you need to run `doveadm force-resync -u user FolderName`. Aki > On 16/03/2022 12:08 Joachim Lindenberg wrote: > > > Hi Aki, > sure I can delete (or rename) the file(s) in the mail directory. Will sync > then restore it from remote or should I do this manually? Should I stop > dovecot during the process?` > I am not very familiar with dovecot yet, as most of the complexity is hidden > by mailcow. > Thanks, Joachim > > -Ursprüngliche Nachricht- > Von: Aki Tuomi > Gesendet: Wednesday, 16 March 2022 08:38 > An: Joachim Lindenberg ; dovecot@dovecot.org > Betreff: Re: AW: invalid lz4 chunk size?? > > Hi, > > looks a lot like your mail file is corrupted. Not much you can do about it > other than maybe delete the file? You can try recover it with `doveadm fetch > -u someone text uid 1553 mailbox Sent` and then using `doveadm save -u > someone -m Sent` to store it back. > > Aki > > > On 16/03/2022 09:35 Joachim Lindenberg wrote: > > > > > > Nobody that can help? > > Thanks, > > Joachim > > > > -- > > > > I am still experiencing the issue. Any suggestion? > > As I do have replication between two nodes and only one is showing the > > issue - can I rename the mailbox easily on one side and rely on replication > > to get the copy replaced? If that makes sense, which commands do you > > recommend? > > Thanks, > > Joachim > > > > -Ursprüngliche Nachricht- > > Von: Joachim Lindenberg > > Gesendet: Thursday, 3 March 2022 12:06 > > An: 'Aki Tuomi' ; dovecot@dovecot.org > > Betreff: AW: invalid lz4 chunk size?? > > > > dovecot --version reports 2.3.17.1 (476cd46418) Joachim > > > > -Ursprüngliche Nachricht- > > Von: Aki Tuomi > > Gesendet: Thursday, 3 March 2022 11:56 > > An: Joachim Lindenberg ; dovecot@dovecot.org > > Betreff: Re: invalid lz4 chunk size?? > > > > > > > On 03/03/2022 12:24 Joachim Lindenberg wrote: > > > > > > > > > Hello, > > > when accessing one mailbox via ActiveSync / SoGo / Dovecot I get the > > > following error repeatedly in dovecot log: > > > imap(somemail...@example.org)<1579><***>: Error: Mailbox Sent: > > > UID=1553: read(compress()) failed: read() failed: lz4.read(): invalid lz4 > > > chunk size: 1601505441 at 16842752 (read reason=) I can still access the > > > mailbox via IMAP though. > > > What can I do to resolve the issue? > > > Thanks, > > > Joachim > > > > Which version of Dovecot? > > > > Aki
AW: AW: invalid lz4 chunk size??
Hi Aki, sure I can delete (or rename) the file(s) in the mail directory. Will sync then restore it from remote or should I do this manually? Should I stop dovecot during the process?` I am not very familiar with dovecot yet, as most of the complexity is hidden by mailcow. Thanks, Joachim -Ursprüngliche Nachricht- Von: Aki Tuomi Gesendet: Wednesday, 16 March 2022 08:38 An: Joachim Lindenberg ; dovecot@dovecot.org Betreff: Re: AW: invalid lz4 chunk size?? Hi, looks a lot like your mail file is corrupted. Not much you can do about it other than maybe delete the file? You can try recover it with `doveadm fetch -u someone text uid 1553 mailbox Sent` and then using `doveadm save -u someone -m Sent` to store it back. Aki > On 16/03/2022 09:35 Joachim Lindenberg wrote: > > > Nobody that can help? > Thanks, > Joachim > > -- > > I am still experiencing the issue. Any suggestion? > As I do have replication between two nodes and only one is showing the issue > - can I rename the mailbox easily on one side and rely on replication to get > the copy replaced? If that makes sense, which commands do you recommend? > Thanks, > Joachim > > -Ursprüngliche Nachricht- > Von: Joachim Lindenberg > Gesendet: Thursday, 3 March 2022 12:06 > An: 'Aki Tuomi' ; dovecot@dovecot.org > Betreff: AW: invalid lz4 chunk size?? > > dovecot --version reports 2.3.17.1 (476cd46418) Joachim > > -Ursprüngliche Nachricht- > Von: Aki Tuomi > Gesendet: Thursday, 3 March 2022 11:56 > An: Joachim Lindenberg ; dovecot@dovecot.org > Betreff: Re: invalid lz4 chunk size?? > > > > On 03/03/2022 12:24 Joachim Lindenberg wrote: > > > > > > Hello, > > when accessing one mailbox via ActiveSync / SoGo / Dovecot I get the > > following error repeatedly in dovecot log: > > imap(somemail...@example.org)<1579><***>: Error: Mailbox Sent: > > UID=1553: read(compress()) failed: read() failed: lz4.read(): invalid lz4 > > chunk size: 1601505441 at 16842752 (read reason=) I can still access the > > mailbox via IMAP though. > > What can I do to resolve the issue? > > Thanks, > > Joachim > > Which version of Dovecot? > > Aki
Re: SIS and tracing the origin of an attachment
Hi all,
On 15.03.22 22:40, doug wrote:
On 3/15/2022 3:45 PM, Oscar del Rio wrote:
On 2022-03-15 9:02 a.m., doug wrote:
On 3/8/2022 5:51 PM, doug wrote:
I'm trying to trace an attachment within an SIS subdirectory to the
email message(s) that link to it. I say messages because I'm also
using dovecot dedup. My understanding is the linked file name is the
hash value of the attachments contents concatenated with the GUID of
the email message. I have had marginal success with a message I
created myself.
Example: I generated an email with two attachments. Here are the
links in my attachment directory.
./26/c5/26c5c540d41779d83d2f5388041d05c67d720d9a-73eca8051acd27627231f2bc99a3
./65/cd/65cd73112a489ef07f17ed5740aa60358e2dd3fb-74eca8051acd27627231f2bc99a3
I keep experimenting with this and I still haven't found a reliable
way to track an attachment back to it's original message so I can
either notify the user or delete the message with doveadm. Is this
not possible? I'm using mdbox if that matters. I see a similar thread
going right now about virus scanning and deleting messages but that
is maildir and I suspect not using SIS for attachments.
The very few times I've needed to trace a SIS attachment to a mailbox,
I just grep the "storage" folders for the file hash
find username/storage -type f -exec grep
9ffa4b246589f8039d123ea909f1520e791bd880 {} +
username/storage/m.46588:X908 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-c9ee303687e13062cf740012bfe47a40
username/storage/m.46589:X1918 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-080ce71390e1306299730012bfe47a40
username/storage/m.46588:
BSent
X908 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-c9ee303687e13062cf740012bfe47a40
username/storage/m.46589:
BINBOX
X1918 2409141 B72
9f/fa/9ffa4b246589f8039d123ea909f1520e791bd880-080ce71390e1306299730012bfe47a40
-> Attachment in username's INBOX and Sent folders.
Thank you for the suggestion Oscar. My mdbox files are encrypted and
compressed, so unfortunately directly grepping them will not work.
You can use "doveadm dump" to decompress the files for grepping them,
not sure about encryption:
find path/to/userhomes/mdbox/storage -name 'm.*' | \
while read f; do
doveadm dump $f | \
grep -E '^msg.(ext-ref|orig-mailbox|guid)' | \
grep -B2 xx/yy/hash-guid || continue
echo "Match in $f"
done
The dump also contains several other fields you might want to display.
Best,
--
Patrick Cernko +49 681 9325 5815
Joint Administration: Information Services and Technology
Max-Planck-Institute fuer Informatik & Softwaresysteme
smime.p7s
Description: S/MIME Cryptographic Signature
Re: sieve-filter not respecting :create flag of fileinto
> On 16/03/2022 09:19 Aki Tuomi wrote: > > > > On 15/03/2022 01:13 David Arroyo wrote: > > > > > > On Mon, Mar 14, 2022, at 03:46, Aki Tuomi wrote: > > > can you provide output of mail_debug=yes logs during delivery? > > > > Sure, attached. > > > > David > > Thanks. Seems like folder creation is not attempted. Could be a bug, I'll > take a look. > > Aki Seems to be a bug, I'll file it in our bug tracker.. You can workaround this issue by creating the folder by hand. Aki
Re: AW: invalid lz4 chunk size??
Hi, looks a lot like your mail file is corrupted. Not much you can do about it other than maybe delete the file? You can try recover it with `doveadm fetch -u someone text uid 1553 mailbox Sent` and then using `doveadm save -u someone -m Sent` to store it back. Aki > On 16/03/2022 09:35 Joachim Lindenberg wrote: > > > Nobody that can help? > Thanks, > Joachim > > -- > > I am still experiencing the issue. Any suggestion? > As I do have replication between two nodes and only one is showing the issue > - can I rename the mailbox easily on one side and rely on replication to get > the copy replaced? If that makes sense, which commands do you recommend? > Thanks, > Joachim > > -Ursprüngliche Nachricht- > Von: Joachim Lindenberg > Gesendet: Thursday, 3 March 2022 12:06 > An: 'Aki Tuomi' ; dovecot@dovecot.org > Betreff: AW: invalid lz4 chunk size?? > > dovecot --version reports 2.3.17.1 (476cd46418) Joachim > > -Ursprüngliche Nachricht- > Von: Aki Tuomi > Gesendet: Thursday, 3 March 2022 11:56 > An: Joachim Lindenberg ; dovecot@dovecot.org > Betreff: Re: invalid lz4 chunk size?? > > > > On 03/03/2022 12:24 Joachim Lindenberg wrote: > > > > > > Hello, > > when accessing one mailbox via ActiveSync / SoGo / Dovecot I get the > > following error repeatedly in dovecot log: > > imap(somemail...@example.org)<1579><***>: Error: Mailbox Sent: > > UID=1553: read(compress()) failed: read() failed: lz4.read(): invalid lz4 > > chunk size: 1601505441 at 16842752 (read reason=) I can still access the > > mailbox via IMAP though. > > What can I do to resolve the issue? > > Thanks, > > Joachim > > Which version of Dovecot? > > Aki
AW: invalid lz4 chunk size??
Nobody that can help? Thanks, Joachim -- I am still experiencing the issue. Any suggestion? As I do have replication between two nodes and only one is showing the issue - can I rename the mailbox easily on one side and rely on replication to get the copy replaced? If that makes sense, which commands do you recommend? Thanks, Joachim -Ursprüngliche Nachricht- Von: Joachim Lindenberg Gesendet: Thursday, 3 March 2022 12:06 An: 'Aki Tuomi' ; dovecot@dovecot.org Betreff: AW: invalid lz4 chunk size?? dovecot --version reports 2.3.17.1 (476cd46418) Joachim -Ursprüngliche Nachricht- Von: Aki Tuomi Gesendet: Thursday, 3 March 2022 11:56 An: Joachim Lindenberg ; dovecot@dovecot.org Betreff: Re: invalid lz4 chunk size?? > On 03/03/2022 12:24 Joachim Lindenberg wrote: > > > Hello, > when accessing one mailbox via ActiveSync / SoGo / Dovecot I get the > following error repeatedly in dovecot log: > imap(somemail...@example.org)<1579><***>: Error: Mailbox Sent: > UID=1553: read(compress()) failed: read() failed: lz4.read(): invalid lz4 > chunk size: 1601505441 at 16842752 (read reason=) I can still access the > mailbox via IMAP though. > What can I do to resolve the issue? > Thanks, > Joachim Which version of Dovecot? Aki
Re: sieve-filter not respecting :create flag of fileinto
> On 15/03/2022 01:13 David Arroyo wrote: > > > On Mon, Mar 14, 2022, at 03:46, Aki Tuomi wrote: > > can you provide output of mail_debug=yes logs during delivery? > > Sure, attached. > > David Thanks. Seems like folder creation is not attempted. Could be a bug, I'll take a look. Aki
