Re: Are there any good maildir manipulation utilities out there?

[N.J. Thomas]

* Chris Green  [2021-04-07 22:01:22+0100]:
> I'm looking for a tool which will allow me to search through a large
> hierarchy of maildir messages and then provide actions to take on the
> matched messages.

You could do this pretty easily with Python, using the mailbox library.

Thomas

Re: Are there any good maildir manipulation utilities out there?

[Cameron Simpson]

On 07Apr2021 18:34, Kurt Hackenberg  wrote:
>On 2021/04/07 18:16, Kurt Hackenberg wrote:
>>On 2021/04/07 17:01, Chris Green wrote:
>>>I'm looking for a tool which will allow me to search through a large
>>>hierarchy of maildir messages and then provide actions to take on the
>>>matched messages.
>>
>>
>>fdm?
>>
>>
>>Or some other filter-and-deliver program -- but not procmail, 
>>because it's a zombie, abandoned by its last maintainers, last 
>>release 20 years ago.
>
>
>Maybe Cameron Simpson's filer, if he publishes it.

It's got some dependencies on some stuff I need to weed out before it is 
easy for someone-not-me to use. It's also not particularly well suited 
to Chris' requirement, which includes preserving the source tree shape 
in the archiving process.

I suspect Chris may need to roll his own. I'd imagine something like:

find message paths using mairix \
| move message files sideways, making sure there's no conflicts

Cheers,
Cameron Simpson 

Re: Are there any good maildir manipulation utilities out there?

[Kurt Hackenberg]

On 2021/04/07 18:16, Kurt Hackenberg wrote:


On 2021/04/07 17:01, Chris Green wrote:


I'm looking for a tool which will allow me to search through a large
hierarchy of maildir messages and then provide actions to take on the
matched messages.



fdm?


Or some other filter-and-deliver program -- but not procmail, because 
it's a zombie, abandoned by its last maintainers, last release 20 years 
ago.



Maybe Cameron Simpson's filer, if he publishes it.

Re: my weekend project: a streaming POP3 fetcher, replacing fetchmail/getmail

[Kurt Hackenberg]

On 2021/04/07 18:03, Cameron Simpson wrote:


Just tried it on the satellite link with an overnight load of messages,
normally a 10 minute exercise with getmail (give or take). 411 messages,
8.5 seconds.


Nice.

...regexps...are appalling for email addresses. When testing 
addresses, my filer does a correct address parse and compares the 
inner component (eg "c...@cskk.id.au" from "Cameron Simpson 
"), and can do set membership tests on that eg "is 
this address in this group?".
I agree that regexps on raw header values are not good enough for 
addresses, and that parsing and some kind of custom matching is the way 
to go. Set membership sounds useful.

Re: S/MIME stopped working

[isdtor]

> >TBH this looks more like a gpg than a mutt problem, and I haven't 
> >figured out how to debug this. Same error for encrypting.
> 
> Yes, it sounds like something changed with either the GPGME version, or 
> perhaps a configuration file.  I can't offer much advice except to check 
> those things. :(

PEBKAC.

Files under ~/.gnupg deleted that shouldn't have been. Luckily I had a backup 
somewhere. Will know better next time.

Re: Are there any good maildir manipulation utilities out there?

[Kurt Hackenberg]

On 2021/04/07 17:01, Chris Green wrote:


I'm looking for a tool which will allow me to search through a large
hierarchy of maildir messages and then provide actions to take on the
matched messages.



fdm?


Or some other filter-and-deliver program -- but not procmail, because 
it's a zombie, abandoned by its last maintainers, last release 20 years ago.

Re: my weekend project: a streaming POP3 fetcher, replacing fetchmail/getmail

[Cameron Simpson]

On 06Apr2021 23:12, Kurt Hackenberg  wrote:
>On Wed, Apr 07, 2021 at 09:43:36AM +1000, Cameron Simpson wrote:
>>My new tool streams the fetches: it issues RETRs for every message up
>>front at maximum network speed - fully buffered and with no waits. A
>>parallel worker thread collects the messages as they come in at full
>>speed (the upstream server likely also gets to fully buffer); it issues
>>DELEtes as each message is saved, also fully buffered.
>
>Slick.  Clearly the right way to handle that high latency.

Just tried it on the satellite link with an overnight load of messages, 
normally a 10 minute exercise with getmail (give or take). 411 messages, 
8.5 seconds.

>Have you ever tried the program fdm?  It fetches, filters, and
>delivers mail, like getmail and procmail combined.  I haven't tried
>it, but it looks interesting.

I have not.

My mail filer is decoupled from my fetcher: it monitors spool Maildirs 
(which also means I can refile a message just by saving it to a spool 
Maildir).  And it has its own syntax to my liking; other tools 
inherently will not :-)

And looking at the conf file, it seems that (like procmail, which I 
abandoned years ago) it matches using regexps. These are appalling for 
email addresses. When testing addresses, my filer does a correct address 
parse and compares the inner component (eg "c...@cskk.id.au" from "Cameron 
Simpson "), and can do set membership tests on that eg 
"is this address in this group?".

>This paragraph in its manual sounds like it might stream fetching like
>your program:
>
>"fdm tries to queue a number of mails simultaneously, so that older
>can be delivered while waiting for the server to provide the next. The
>maximum length of the queue for each account is set by the
>'queue-high' option (the default is two) and the maximum mail size
>accepted by the 'maximum-size' option (the default is 32 MB). In
>addition, the 'rewrite' action requires an additional temporary
>mail. Although fdm will fail rather than dropping mail if the disk
>becomes full, users should bear in mind the possibility and set the
>size of the temporary directory and the fdm options according to their
>needs."

I've been reading the manual. I think it does not. I think it actually 
allows the filing/saving to proceeed while requesting/fetching the next 
message. So a simple form of parallelism, but not one which reduces the 
fetch latency between requests.

>fdm is at github:
>
>
>The paragraph quoted above is at about line 300 in the manual, which is here:
>

Thanks. An interesting read.

Cheers,
Cameron Simpson 

Are there any good maildir manipulation utilities out there?

[Chris Green]

I'm looking for a tool which will allow me to search through a large
hierarchy of maildir messages and then provide actions to take on the
matched messages.

E.g. I might want to archive all messages older than a certain date
but archive them into a similar hierarchy rather than to just a single
archive directory.

Or I might want to search for all messages from a certain person (or
business) and move them into a single folder.

I already use mairix for searching for messages but it doesn't offer
the actions function I'm after.  

Before I start creating something for myself is there anything out
there which can do this sort of thing?

-- 
Chris Green

Re: HTML email?

[John Niendorf]

Thank you - where do you put the python3 script and how do you let mutt know it 
is there?

HTML email?

[John Niendorf]

Hi Folks,

How do you all deal with HTML email?

Thank you,

John

Re: signature and alternates

[Kevin J. McCarthy]

On Wed, Apr 07, 2021 at 01:52:44PM +0100, Steve Karmeinsky wrote:
The only thing not working properly is setting the signature back to 
normal after replying to an alternate.


I didn't see how you overrode the signature, so I can only guess it's 
something like:


  reply-hook  ."set signature=~/.sig-default"
  reply-hook  "~C alias1"  "set signature=~/.sig-alias1"
  reply-hook  "~C alias2"  "set signature=~/.sig-alias2"
  ...

It can be helpful to look at 
 when adjusting these 
things.  Looking at that, you can see reply-hooks fire first, then 
send-hooks, and then three steps below the message and signature are 
generated.


So, replies not covered by one of the aliases will set the signature
back to default, but non-replies still need to reset back.

send-hooks can use the pattern !~Q to be invoked in the non-reply 
case, so you could try adding:


  send-hook  !~Q   "set signature=~/.sig-default"

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature

Re: S/MIME stopped working

[Kevin J. McCarthy]

On Wed, Apr 07, 2021 at 12:36:53PM +0100, isdtor wrote:

My S/MIME setup has died one from day to the next and I cannot find out why.

Symptom: trying to send e.g. signed email, the result is

error signing data: No CRL known?


This an error coming back from GPGME when trying to perform the sign 
operation.


TBH this looks more like a gpg than a mutt problem, and I haven't 
figured out how to debug this. Same error for encrypting.


Yes, it sounds like something changed with either the GPGME version, or 
perhaps a configuration file.  I can't offer much advice except to check 
those things. :(


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature

Re: HTML email?

[User Ribbon]

On Wed, Apr 07, 2021 at 07:22:51AM -0600, John Niendorf wrote:
> Hi Folks,
> 
> How do you all deal with HTML email?

My .muttrc is



set mailcap_path= ~/.mutt/mailcap
auto_view text/html



~/.mutt/mailcap is

text/html; w3m -I %{charset} -T text/html; copiousoutput;

Re: HTML email?

[Francesco Ariis]

Hello John,

Il 07 aprile 2021 alle 07:22 John Niendorf ha scritto:
> How do you all deal with HTML email?

This in mailcap

text/html; w3m -I %{charset} -T text/html; copiousoutput

and then  in mutt itself
—F

Re: HTML email?

[lilydjwg]

On Wed, Apr 07, 2021 at 07:22:51AM -0600, John Niendorf wrote:
> Hi Folks,
> 
> How do you all deal with HTML email?
> 
> Thank you,
> 
> John

I view it in mutt with elinks -force-html -dump -dump-color-mode 1. If
that doesn't work well, I open it in my browser with a script
viewhtmlmsg[1] using a custom keybinding:

macro pager \eh "viewhtmlmsg --fork -w 10" "View in 
webbrowser"

I don't write HTML mails.

[1]: https://github.com/lilydjwg/viewhtmlmsg

-- 
Best regards,
lilydjwg

signature and alternates

[Steve Karmeinsky]

This is my main email, but I have several alternates set-up.

In .muttrc I have

set signature=~/.signature
set realname = "Steve Karmeinsky"
set from = "st...@gbnet.net"
my_hdr X-Organisation: NetTek Ltd

I also have
send-hook . 'unmy_hdr From:'
send-hook . 'unmy_hdr X-Organisation:'
send-hook . 'unmy_hdr Reply-to:'

and then send-hooks with the overrides.

The only thing not working properly is setting the signature back to
normal after replying to an alternate.

Steve

-- 
NetTek Ltd  UK mob +44 7775 755503
UK +44 20 3432 3735  /  US +1 (650) 423 1390
social id stevekennedyuk
Euro Tech News Blog http://eurotechnews.net

S/MIME stopped working

[isdtor]

All,

My S/MIME setup has died one from day to the next and I cannot find out why.

Symptom: trying to send e.g. signed email, the result is

error signing data: No CRL known?

TBH this looks more like a gpg than a mutt problem, and I haven't figured out 
how to debug this. Same error for encrypting.

Everything in the tool chain is at the latest version - mutt, gpg and 
components, openssl. I have verified that my certificate is not expired (years 
in the future) and that the crl is accessible by wget/curl. mutt in debug mode 
does not give any additional information, and I can't make much sense of an 
strace -f log either. The same certificate and process works just fine in 
thunderbird on the same machine.

Any ideas?