On Sat, Jan 15, 2022 at 08:01:05PM +0100, Robert Siemer wrote:
> I need to DKIM sign possibly huge emails (up to 150MB).
No worries, you can do this with a milter, without storing
an extra copy of the complete message.
> Conceptually DKIM needs to go over the email twice: once to calculate
> and sign the checksum and once to write it out with the result of the
> previous step in the headers.ยน
Prepending a header does not require rewriting the message body.
Postfix queue files support efficient header insertion.
> A DKIM signer can do this by either keeping the message in memory (a
> no-go for me) or write it to a file.
Neither is necessary, just compute a streaming checksum, and emit the
signature as a prepended header.
> For the task at hand I want to use a Postfix (filter) mechanism that
> allows me to do that without keeping the message in memory and without
> having it written to disc twice!
A suitably efficient milter that computes a streaming digest will work
fine.
> So far I see that the after-queue content filter mechanism
> (FILTER_README) forces you to write the email to disc again.
Yes, it forces you to buffer the message content if you want to make
body-dependent header modifications. But do you really need to optimise
this to avoid making a copy? Writing 150MB to (SSD) disk or a tmpfs with
a few GB of space is quite fast, and the file can be pre-removed aiding
cleanup.
> The alternative, the before-queue milter (MILTER_README), is
> insufficiently documented for me to see if it avoids keeping the
> message in memory and avoids writing the original mail to file twice.
A milter can compute the desired header in a streaming manner, and then
respond with a "prepend header" action.
Find some good milter API documentation...
--
Viktor.
