Re: Minor patches for builds against ancient platforms

[M. Balridge]

David "Show Me The Vintage!" McGuire wrote:

>   I for one am finding this thread extremely entertaining.  I have to
> wonder how you'd sound if you came across a machine that was actually
> OLD. ;)

Well, I am fond of "old" hardware, which may still be on the wrong side of the
New/Old divide for some of you: DECSYSTEM-20s and VAX 11/780s were the first
"big" systems I ran across that I admired a great deal as an assembly language
programmer and software engineer in general. 

[ IBM System/3X0 systems (and EBCDIC) were horrors, though POWER & PowerPC
were very interesting for me. ] My first assembly language was Z80, though,
which spoiled me for the more primitive CPUs like 6502.

I still maintain some respect for the old iron that could seemingly gracefully
handle 200+ users, many of them hammering the system with compile jobs,
Mandlebrot set "renders", or other geek-driven nonsense, without going off
into the weeds the way a 2017 system does when you do something trivial like
enumerate a large directory of files/folders recursively.

Joseph of Tam (I am) wrote:

> If all your concerned is dovecot dot-files, you can place the 
> indices somewhere else other than the user's filesapace.

When I manually created the .subscriptions file(s) in the right places, the
error message went away, and the functionality seems to work in SeaMonkey 
clients.

I wonder if it's a combination of permissions (even though the mail
directories are all owned by their respective users), or dovecot settings... 
On that note, has anyone written a tool that "harmonises" users mail
directories' permissions - ideally reading the dovecot configuration to assess
where *THE* mail directories are actually used by dovecot?  I was surprised by
the pickiness of the group ownership/permissions issues, though reflecting on
things, I can see why you'd at least want some logging by default for those
conditions.

His Timoness boomed:

> On 9 Jun 2017, at 5.03, M. Balridge  wrote:
>> 1) In src/lib/compat.h there is a definition for p(read|write) 
>> that conflicts with the one in /usr/include/unistd.h [...]
> 
> I don't know about this. Anyway, can't apply this patch since it 
> likely fails elsewhere.

Fair enough. I knew this was unlikely to be accepted for multiple reasons,
never mind a ferociously high potential-pain:reward ratio.

I'm happy to help in my insignificant way, re: the second patch.

DO many people use filesystem quotas with dovecot much, you think?

> I think it's just doing a lot of work on the mbox file itself 
>(reading/writing/rewriting). Would be nice of course if it logged 
> more information, but mbox format is a bit too legacy to spend 
> much time on improving.

I suspect the (heavy) use of procmail on Herr Frankbox is contributing to
either some lock "confusion" *OR* triggering dovecot to do "expensive" mbox
re-read/syncs or something?

There are mail-mulching scriplets in the global procmail (tied to spamassassin
results). Some daintily direct the dross to the /dev/null paradise in the sky.
Some "consume" the mail and redirect them to one of two or three folders
within mail/, while the "rest" allow procmail to append it to INBOX. 

My question is:

Is there is smarter way to do the "delivery" so that the dovecot system is
"informed" of an append (or excision), obviating or at least reducing the need
to perform more costly re-syncs (or timeouts awaiting a lock break)?

I anticipate a thundering herd declaiming that procmail is the spawn of Satan,
Hitler, and He-who-shall-not-be-named. As someone who was responsible for much
of them (nearing 10 years ago, now), I don't disagree with that view.

I don't have the budget or mandate to bring slivers of Elysium to this
downtrodden backwater of technology. I would expect that any use of procmail
with dovecot's "special" mail storage formats would *REQUIRE* the use of
"deliver" or some other tool to properly incorporate new mail into a dovecot 
hive.

My thanks as always,
=M= 

Re: Changing the name of a compressed file

[Aki Tuomi]

There is global mail_plugins variable, you just have to be careful not to 
overwrite it somewhere.

Aki

> On June 11, 2017 at 2:46 PM Peter West  wrote:
> 
> 
> It looks as though there is no global mail_plugins variable.  Is this the 
> case?  Or have I misunderstood how global variable are expressed?
> 
> P
> 
> > On 10 Jun 2017, at 9:10 pm, Peter West  wrote:
> > 
> > Ok, I added zlib to imap protocol.
> > 
> > protocol imap {
> >  …
> >  mail_plugins = $mail_plugins zlib
> > }
> > 
> > Now both imap and lmtp protocols have zlib plugin enabled, and both send 
> > and receive mail is compressed.
> > 
> > Peter
> > 
> >> On 10 Jun 2017, at 6:50 pm, Aki Tuomi  wrote:
> >> 
> >> Please check that you are not overwriting mail plugins for lmtp. Or post 
> >> your doveconf -n.
> >> 
> >> Aki
> >> 
> >>> On June 10, 2017 at 11:10 AM Peter West  wrote:
> >>> 
> >>> 
> >>> Not sure what you mean. I’m using lmtp to send messages to Dovecot from 
> >>> Postfix.
> >>> 
>  On 10 Jun 2017, at 6:08 pm, Aki Tuomi  wrote:
>  
>  What's your LDA?
>  
>  Aki
>  
> > On June 10, 2017 at 11:01 AM Peter West  wrote:
> > 
> > 
> > Thanks for that Aki.
> > 
> > Follow-up question.  I tried to initiate compression by adding
> > 
> > mail_plugins = $mail_plugins zlib
> > 
> > plugin {
> >  zlib_save_level = 6
> >  zlib_save = xz
> > }
> > 
> > 
> > to dovecot.conf.  I restarted dovecot and sent one message to the 
> > server, and one message from the server.  Neither was compressed.  I 
> > changed the save type to
> > 
> >  zlib_save = bz2
> > 
> > and repeated. This time the message received (in 
> > /var/vmail///cur) was not compressed, but the message 
> > in /var/vmail///.Sent/cur was bzip2 compressed.
> > 
> > Why is the received mail not being compressed?  Is this the point of 
> > the discussion about compressing old mails?
> > 
> > 
> >> On 10 Jun 2017, at 4:43 pm, Aki Tuomi  wrote:
> >> 
> >> 
> >>> On June 10, 2017 at 5:58 AM Peter West  wrote:
> >>> 
> >>> 
> >>> Concerning Maildir, the wiki page on compression has this:
> >>> 
> >>> All mails must have ,S= in their filename where  contains 
> >>> the original uncompressed mail size, otherwise there will be problems 
> >>> with quota calculation as well as other potential random failures. 
> >>> Note that if the filename doesn’t contain the ,S= before 
> >>> compression, adding it afterwards changes the base filename and thus 
> >>> the message UID. The safest thing to do is simply to not compress 
> >>> such files.
> >>> 
> >>> Further down on the same page is this:
> >>> 
> >>> If the file does exist, rename() (mv) the compressed file over the 
> >>> original file.
> >>>   • Dovecot can now read the file, but to avoid compressing it 
> >>> again on the next run, you'll probably want to rename it again to 
> >>> include e.g. a "Z" flag in the file name to mark that it was 
> >>> compressed (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ).
> >>> 
> >>> These comments seem to contradict each. Or is there a difference 
> >>> between adding the size specifier to the filename and adding a Z flag 
> >>> to the end of the file name?
> >>> 
> >>> --
> >>> Peter West
> >>> p...@pbw.id.au
> >>> And the great throng heard him gladly.
> >>> 
> >> 
> >> Keyword is 'base filename'. From the wiki, "The standard filename 
> >> definition is: ":2,".". Z is a flag.
> >> 
> >> Aki
> > 
> >>> 
> > 
>

Re: Unix socket for quota-status?

[Peter West]

Thanks Aki.

Same user, group and permissions as for the lmtp service?

P
> On 12 Jun 2017, at 12:14 am, Aki Tuomi  wrote:
> 
> 
>> On June 11, 2017 at 4:26 PM Peter West  wrote:
>> 
>> 
>> The example configs for quota-status use inet_listener.  Does quota-status 
>> support unix sockets?
>> 
>> --
>> Peter West
>> p...@pbw.id.au
>> And the great throng heard him gladly.
>> 
> 
> Yes, you can use unix_listener and inet_listener as you please, dovecot 
> supports them both for all services.
> 
> Aki



signature.asc
Description: Message signed with OpenPGP

Re: Unix socket for quota-status?

[Aki Tuomi]

> On June 11, 2017 at 4:26 PM Peter West  wrote:
> 
> 
> The example configs for quota-status use inet_listener.  Does quota-status 
> support unix sockets?
> 
> --
> Peter West
> p...@pbw.id.au
> And the great throng heard him gladly.
>

Yes, you can use unix_listener and inet_listener as you please, dovecot 
supports them both for all services.

Aki

Unix socket for quota-status?

[Peter West]

The example configs for quota-status use inet_listener.  Does quota-status 
support unix sockets?

--
Peter West
p...@pbw.id.au
And the great throng heard him gladly.



signature.asc
Description: Message signed with OpenPGP

Re: Changing the name of a compressed file

[Peter West]

It looks as though there is no global mail_plugins variable.  Is this the case? 
 Or have I misunderstood how global variable are expressed?

P

> On 10 Jun 2017, at 9:10 pm, Peter West  wrote:
> 
> Ok, I added zlib to imap protocol.
> 
> protocol imap {
>  …
>  mail_plugins = $mail_plugins zlib
> }
> 
> Now both imap and lmtp protocols have zlib plugin enabled, and both send and 
> receive mail is compressed.
> 
> Peter
> 
>> On 10 Jun 2017, at 6:50 pm, Aki Tuomi  wrote:
>> 
>> Please check that you are not overwriting mail plugins for lmtp. Or post 
>> your doveconf -n.
>> 
>> Aki
>> 
>>> On June 10, 2017 at 11:10 AM Peter West  wrote:
>>> 
>>> 
>>> Not sure what you mean. I’m using lmtp to send messages to Dovecot from 
>>> Postfix.
>>> 
 On 10 Jun 2017, at 6:08 pm, Aki Tuomi  wrote:
 
 What's your LDA?
 
 Aki
 
> On June 10, 2017 at 11:01 AM Peter West  wrote:
> 
> 
> Thanks for that Aki.
> 
> Follow-up question.  I tried to initiate compression by adding
> 
> mail_plugins = $mail_plugins zlib
> 
> plugin {
>  zlib_save_level = 6
>  zlib_save = xz
> }
> 
> 
> to dovecot.conf.  I restarted dovecot and sent one message to the server, 
> and one message from the server.  Neither was compressed.  I changed the 
> save type to
> 
>  zlib_save = bz2
> 
> and repeated. This time the message received (in 
> /var/vmail///cur) was not compressed, but the message in 
> /var/vmail///.Sent/cur was bzip2 compressed.
> 
> Why is the received mail not being compressed?  Is this the point of the 
> discussion about compressing old mails?
> 
> 
>> On 10 Jun 2017, at 4:43 pm, Aki Tuomi  wrote:
>> 
>> 
>>> On June 10, 2017 at 5:58 AM Peter West  wrote:
>>> 
>>> 
>>> Concerning Maildir, the wiki page on compression has this:
>>> 
>>> All mails must have ,S= in their filename where  contains 
>>> the original uncompressed mail size, otherwise there will be problems 
>>> with quota calculation as well as other potential random failures. Note 
>>> that if the filename doesn’t contain the ,S= before compression, 
>>> adding it afterwards changes the base filename and thus the message 
>>> UID. The safest thing to do is simply to not compress such files.
>>> 
>>> Further down on the same page is this:
>>> 
>>> If the file does exist, rename() (mv) the compressed file over the 
>>> original file.
>>> • Dovecot can now read the file, but to avoid compressing it 
>>> again on the next run, you'll probably want to rename it again to 
>>> include e.g. a "Z" flag in the file name to mark that it was compressed 
>>> (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ).
>>> 
>>> These comments seem to contradict each. Or is there a difference 
>>> between adding the size specifier to the filename and adding a Z flag 
>>> to the end of the file name?
>>> 
>>> --
>>> Peter West
>>> p...@pbw.id.au
>>> And the great throng heard him gladly.
>>> 
>> 
>> Keyword is 'base filename'. From the wiki, "The standard filename 
>> definition is: ":2,".". Z is a flag.
>> 
>> Aki
> 
>>> 
> 



signature.asc
Description: Message signed with OpenPGP

Re: Non-destructive deduplication

[Tristan Miller]

Greetings.

On Sun, 11 Jun 2017 08:25:33 +0300 (EEST), Aki Tuomi
 wrote:
> > 2) Does there exist any tool that will scan a maildir folder to find
> > messages with duplicate Message-IDs, and automatically rewrite the
> > Message-ID headers so that they are (probably) unique?  I could
> > probably kludge together a quick shell script, but if someone's
> > already gone to the trouble of writing such a tool and making it
> > relatively robust, I'd rather use that instead.
> >   
> 
> https://wiki2.dovecot.org/Tools/Doveadm/Deduplicate should let you
> expunge duplicate messages.

Yeah, simply removing duplicates is pretty trivial, and any decent mail
client will also have a nice way of doing this.  The tricky part in my
case is not removing the duplicates, but preserving them and giving
them unique Message-IDs.  If there's no existing way of automating this
then I'll do as you suggest.

Regards,
Tristan

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
  Tristan Miller
Free Software developer, ferret herder, logologist
 https://logological.org/
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


pgpICYKAesG8O.pgp
Description: OpenPGP digital signature